How to create a Spatial Tracking Session
Configure and run a Spatial Tracking Session to use ARKit features in RealityKit.
According to the documentation, SpatialTrackingSession aims to simplify the use of ARKit features in RealityKit.
We will use both hand and plane detection in this example. First, we need to add two values to our application info.plist file.
NSHandsTrackingUsageDescription
NSWorldSensingUsageDescription
Let’s create a session for our view
@State var trackingSession: SpatialTrackingSession = SpatialTrackingSession()Then create a function to configure and run it.
func runTrackingSession() async {
// We are using hand and plane anchors
let configuration = SpatialTrackingSession.Configuration(tracking: [.hand, .plane])
await trackingSession.run(configuration)
}We can call this function in a task.
.task {
await runTrackingSession()
}Spatial Tracking Session will enable features on Anchoring Components. We can access the transform of these anchors, enable collisions, and use physics.
These anchors can be useful if we know about the anchors ahead of time. For example, we might know that we want to anchor an entity to an index finger. We can define an anchor that will activate when visionOS detects a finger tip.
We can do the same thing with planes. We can create an anchor and describe its characteristics. These include a horizontal surface classified as a floor of at least a certain size. When a plane is detected with these characteristics, an entity will be anchored to it.
However, these anchors don’t seem to offer control of placement or position. In the video demo, you can see the system creating the floor anchor. The position it uses is entirely arbitrary. It doesn’t place the entity in the center of the floor plane. †If you know a good way to solve this, please leave a comment below. We’ll dive further into this in later examples.
†I can visualize this plane using the Xcode debugging tools
Full example code
struct Example031: View {
// 1. Create a session
@State var trackingSession: SpatialTrackingSession = SpatialTrackingSession()
var body: some View {
RealityView { content in
if let scene = try? await Entity(named: "AnchorLabs", in: realityKitContentBundle) {
content.add(scene)
// 3. FloorEntity has an anchoring component defined in Reality Composer Pro
if let floorEntity = scene.findEntity(named: "FloorEntity") {
if var anchor = floorEntity.components[AnchoringComponent.self] {
// We need to clear the default physics simulation to let our sphere collide with this anchor
anchor.physicsSimulation = .none
floorEntity.components.set(anchor)
}
}
// 4. The hand anchor is created here, with the left hand sphere added as a child
if let leftHandSphere = scene.findEntity(named: "LeftHand") {
let leftHand = AnchorEntity(.hand(.left, location: .indexFingerTip),
trackingMode: .continuous)
leftHand.name = "LeftHandAnchor"
leftHand.addChild(leftHandSphere.clone(recursive: true))
leftHandSphere.position = .zero
leftHand.anchoring.physicsSimulation = .none
content.add(leftHand)
}
// 5. We can listen for anchor state changes
_ = content.subscribe(to: SceneEvents.AnchoredStateChanged.self) { event in
print("**anchor changed** \(event)")
}
}
}
.modifier(DragGestureImproved()) // just here to let me grap the sphere if it gets too far away
.task {
await runTrackingSession()
}
}
// 2. Configure and run the session
func runTrackingSession() async {
// We are using hand and plane anchors
let configuration = SpatialTrackingSession.Configuration(tracking: [.hand, .plane])
await trackingSession.run(configuration)
}
}Support our work so we can continue to bring you new examples and articles.
Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.
Follow Step Into Vision