When do you use SceneKit, RealityKit, SpriteKit or Metal for 3D and games-adjacent work?

Apple offers several layers, and the choice is mostly about what you are building and how long the framework will be supported. SceneKit is a high-level, scene-graph 3D engine (nodes, geometries, materials, lights, cameras, physics, animation) that has been the easy route to 3D in apps since 2012: models in .scn or USDZ, an SCNView or the SwiftUI SceneView, and it still works, but Apple has signalled it is in maintenance mode (soft-deprecated in favour of RealityKit), so avoid starting a long-lived product on it. RealityKit is the modern engine: an entity-component-system design (entities with components, handled by systems), physically based rendering, spatial audio, physics, animation, and first-class support for AR (anchors, plane detection, image tracking through ARKit) and for visionOS (RealityView); it uses USDZ and Reality Composer Pro for content. It is the recommended choice for new AR, 3D model viewers and spatial computing. SpriteKit is the 2D engine (sprites, texture atlases, particle emitters, physics, actions, scenes), good for 2D games, animated effects and educational apps, and mature and stable. GameplayKit adds game logic (state machines, entities and components, pathfinding, randomness, agents). Metal is for custom renderers, performance-critical games, and full control; you build the engine yourself or use a third party (Unity, Unreal, Godot), which are the usual choice for large cross-platform games. ARKit provides tracking and scene understanding (world tracking, planes, meshes, faces, body), usually paired with RealityKit.

How to choose: a simple 2D game or effect → SpriteKit; showing a 3D model, AR, or visionOS → RealityKit; a quick 3D view in an older app that already uses SceneKit → keep SceneKit while planning a migration; a technically demanding or cross-platform game → Metal with an engine; rich UI that happens to need a little 3D (a rotating product) → RealityKit’s Model3D or RealityView, or even Core Animation transforms. Common concerns across all of them: asset pipeline (USDZ, texture compression, level of detail), performance (draw calls, polygon counts, texture memory, overdraw: profile with the Metal debugger and Instruments), frame pacing (a stable 60 or 120 fps matters more than peak speed), thermal and battery limits (sustained load throttles the device), accessibility (give 3D content labels and alternatives), memory (large textures dominate), and device support (not every device has the same GPU or sensors, especially for AR). Before committing to a framework, check its current status in the documentation and WWDC sessions, because Apple’s direction for 3D has shifted toward RealityKit and spatial computing.