How do you handle Apple Pencil input, and how does multitasking change your layout assumptions?

7.8.2 · L5 · Interview

Predict first

In Split View an iPad app is told to run at one third of the screen width. What should your layout do?

// A: use UIScreen.main.bounds, which still gives the full screen
// B: use the view and its size classes, which will report compact width, and adapt
// C: refuse to run
  • A
  • B
  • C

Mental model. The pencil is a fast, precise hand: it writes faster than the screen can show, so you draw a little ahead of it and correct once the real points arrive.

Apple Pencil arrives as ordinary touches with extra information. A UITouch has a type (.pencil versus .direct), force, altitudeAngle and azimuthAngle. Because the pencil samples at a higher rate than the display refreshes, UIKit delivers coalesced touches (event.coalescedTouches(for:)) for smooth strokes and predicted touches (predictedTouches(for:)) to draw slightly ahead and cut latency; predicted points are provisional and must be replaced by the real ones. UIPencilInteraction reports double-tap and squeeze gestures. For drawing or markup, PencilKit (PKCanvasView) gives a full canvas and tool picker without hand-built rendering, and handwriting in text fields is supported by the system (Scribble).

Multitasking means the window is not the screen. In Split View, Slide Over and Stage Manager your app can be resized at any time, down to compact width, and with multiple windows you get several UIScenes, each with its own window and state. Therefore: never read UIScreen.main.bounds, lay out against the view and its size classes, handle every size change (viewWillTransition(to:with:) or trait changes), and keep per-window state in the scene, shared data in a model. Support state restoration so a scene can be rebuilt after being disconnected. This is why the scene lifecycle replaced single-window assumptions in the app delegate.

The window is smaller than the screen, and the trait collection reports it, so adaptive layout handles it without special cases.

Common traps

  • Treating predicted touches as final. They are guesses; remove them when the real points arrive.
  • Using UIScreen.main.bounds for layout. In a resizable window it is simply the wrong size.

They might follow up with

  • Why coalesced touches? The pencil samples faster than the screen refreshes; coalescing gives you the missing points for smooth lines.
  • Where does per-window state live? In the UIScene and its delegate, not in the app delegate.

Related questions

  • 7.8.1 How do you support hardware keyboards, shortcuts, menus and the pointer on iPad?
  • 7.7.1 What is a trait collection, and how do size classes and trait changes drive adaptive layout?
  • 6.2.1 What are the app states, and which callbacks fire as you move between them?

← How do you support hardware keyboards, shortcuts, menus and the…