Explore all locations and joints for Hand Anchors

Joseph Simpson Updated on

Unpacking all the locations and joints that we can access with AnchorEntity in RealityKit.

When working with AnchorEntity, we can use two approaches to specify a where to place our tracked entity.

AnchoringComponent.Target.HandLocation includes a handful (haha) of convenient anchors such as .indexFingerTip, .thumbTip, .palm, .wrist, and .aboveHand.

Left Hand Example: Create and customize entities for each primary location. I scaled up some of these and projected the palm anchor a bit in front of the hand. Notice that the `.aboveHand. anchor stays above the hand regardless of hand orientation. This can be very useful for small UI elements.

if let leftHandSphere = scene.findEntity(named: "StepSphereBlue") {
    let indexTipAnchor = AnchorEntity(.hand(.left, location: .indexFingerTip), trackingMode: .continuous)
    indexTipAnchor.addChild(leftHandSphere)
    content.add(indexTipAnchor)

    let palmAnchor = AnchorEntity(.hand(.left, location: .palm), trackingMode: .continuous)
    palmAnchor.addChild(leftHandSphere.clone(recursive: true))
    palmAnchor.position =  [0, 0.05, 0]
    palmAnchor.scale = [3, 3, 3]
    content.add(palmAnchor)

    let thumbTipAnchor = AnchorEntity(.hand(.left, location: .thumbTip), trackingMode: .continuous)
    thumbTipAnchor.addChild(leftHandSphere.clone(recursive: true))
    content.add(thumbTipAnchor)

    let wristAnchor = AnchorEntity(.hand(.left, location: .wrist), trackingMode: .continuous)
    wristAnchor.addChild(leftHandSphere.clone(recursive: true))
    wristAnchor.scale = [3, 3, 3]
    content.add(wristAnchor)

    let aboveHandAnchor = AnchorEntity(.hand(.left, location: .aboveHand), trackingMode: .continuous)
    aboveHandAnchor.addChild(leftHandSphere.clone(recursive: true))
    aboveHandAnchor.scale = [2, 2, 2]
    content.add(aboveHandAnchor)
}

AnchoringComponent.Target.HandLocation.HandJoint contains all the joints that we can access. In ARKit, joints are available as an enum via HandSkeleton.JointName.allCases. Unfortunately, in AnchorEntity, this is not the case. As a workaround I created an array of all joints.

Right Hand Example: Create an entity for each joint

if let rightHandSphere = scene.findEntity(named: "StepSphereGreen") {
    // In ARKit, joints are availble as a ENUM HandSkeleton.JointName.allCases
    // But in RealityKit we are not so lucky. Create an array of all joints to iterate over.
    let joints: [AnchoringComponent.Target.HandLocation.HandJoint] = [
        .forearmArm,
        .forearmWrist,
        .indexFingerIntermediateBase,
        .indexFingerIntermediateTip,
        .indexFingerKnuckle,
        .indexFingerMetacarpal,
        .indexFingerTip,
        .littleFingerIntermediateBase,
        .littleFingerIntermediateTip,
        .littleFingerKnuckle,
        .littleFingerMetacarpal,
        .littleFingerTip,
        .middleFingerIntermediateBase,
        .middleFingerIntermediateTip,
        .middleFingerKnuckle,
        .middleFingerMetacarpal,
        .middleFingerTip,
        .ringFingerIntermediateBase,
        .ringFingerIntermediateTip,
        .ringFingerKnuckle,
        .ringFingerMetacarpal,
        .ringFingerTip,
        .thumbIntermediateBase,
        .thumbIntermediateTip,
        .thumbKnuckle,
        .thumbTip,
        .wrist
    ]

    for joint in joints {
        let anchor = AnchorEntity(
            .hand(.right, location: .joint(for: joint)),
            trackingMode: .continuous
        )
        anchor.addChild(rightHandSphere.clone(recursive: true))
        anchor.position = rightHandSphere.position
        content.add(anchor)
    }
}

Download the Xcode project with this and many more examples from Step Into Vision.

Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?