How to create hand anchors with Anchor Entity

We can use AnchorEntity to access a number of locations and joints on our hands.

The documentation for SpatialTrackingSession explains something interesting:

In visionOS, if you don’t set up SpatialTrackingSession manually, the system anchors your AnchorEntity in a privacy-preserving manner. In other words, the app doesn’t get read access to any transform information from the AnchorEntity.

So we can anchor objects to our hands, but we can’t access transform data. We can’t fire collision triggers or interact with physics.

We can start by adding an entity to track. This will create an anchor for the left index finger

let leftHandAnchor = AnchorEntity(.hand(.left, location: .indexFingerTip), trackingMode: .continuous)

Then we can optionally add an entity to this as a child. In this example we will anchor a sphere from our scene.

leftHandAnchor.addChild(leftHandSphere)
leftHandAnchor.position = leftHandSphere.position

It is important to add this entity to our scene before we can start using it.

content.add(leftHandAnchor)

There are a handful of convenient locations we can use: indexFingerTip, aboveHand, palm, thumbTip, and wrist. We can also specify one of the underlying joints. We will explore these locations and joints in another example.

We can set tracking mode if needed. The default value .continuous is very accurate but it can sometimes lag slightly behind. We can also use.predicted, which can feel much faster but it has a tendency to overshoot during very fast motions. I encourage you to try them both to see what fit you use case best.

Update: An alternative to AnchorEntity is to use AnchoringComponent directly.

Video demo

Full example code

struct Example019: View {
    var body: some View {
        RealityView { content in

            if let scene = try? await Entity(named: "HandTrackingLabs", in: realityKitContentBundle) {
                content.add(scene)

                if let leftHandSphere = scene.findEntity(named: "StepSphereBlue") {
                    let leftHandAnchor = AnchorEntity(.hand(.left, location: .indexFingerTip), trackingMode: .continuous)
                    leftHandAnchor.addChild(leftHandSphere)
                    leftHandAnchor.position = leftHandSphere.position
                    content.add(leftHandAnchor)
                }

                if let rightHandSphere = scene.findEntity(named: "StepSphereGreen") {
                    let rightHandAnchor = AnchorEntity(.hand(.right, location: .indexFingerTip),trackingMode: .predicted)
                    rightHandAnchor.addChild(rightHandSphere)
                    rightHandAnchor.position = rightHandSphere.position
                    content.add(rightHandAnchor)
                }

            }
        }
        .persistentSystemOverlays(.hidden)
    }
}

Download the Xcode project with this and many more examples from Step Into Vision.

Some examples are provided as standalone Xcode projects. You can find those here.