How to use Physics Interactions with Hand Anchors
We can add a Spatial Tracking Session if we need our hands to interact with physics bodies in the scene.
Building on the example from AnchorEntity with hands, let’s add some ARKit features. In RealityKit, the easiest way to do this is to create a Spatial Tracking Session.
Let’s use Spatial Tracking Session to add physical interactions between our hands and any physics bodies in the scene.
let configuration = SpatialTrackingSession.Configuration(
tracking: [.hand])
let session = SpatialTrackingSession()
await session.run(configuration)Next we need to add some anchors and entities to our hands. The entity named “LeftHand” in our scene has a physics body component with the mode set to kinematic. This will allow our hands to move the entity. They will affect other physics bodies, but will not be affected by them in return.
The key to getting our anchors to collide with the entities is to set the physics simulation to .none. Otherwise, these anchor entities would only be able to interact with physics bodies in their own physics space. They would not be able to interact with the rest of the scene.
if let leftHandSphere = scene.findEntity(named: "LeftHand") {
// 2. Create an anchor for the left index finger
let leftHand = AnchorEntity(.hand(.left, location: .indexFingerTip),
trackingMode: .continuous)
leftHand.addChild(leftHandSphere.clone(recursive: true))
leftHandSphere.position = .zero
// 3. Disable the default physics simulation on the anchor
leftHand.anchoring.physicsSimulation = .none
content.add(leftHand)
}Our scene includes two flat surfaces with static physics bodies. We have three spheres that have dynamic bodies. Gravity has been deactivated for all entities.
Video demo
Full example code
struct Example023: View {
var body: some View {
RealityView { content in
if let scene = try? await Entity(named: "HandTrackingLabsPhysics", in: realityKitContentBundle) {
content.add(scene)
// 1. Set up a Spatial Tracking Session with hand tracking.
// This will add ARKit features to our Anchor Entities, enabling access to transforms, collisions, and physics.
let configuration = SpatialTrackingSession.Configuration(
tracking: [.hand]
)
let session = SpatialTrackingSession()
await session.run(configuration)
if let leftHandSphere = scene.findEntity(named: "LeftHand") {
// 2. Create an anchor for the left index finger
let leftHand = AnchorEntity(.hand(.left, location: .indexFingerTip),
trackingMode: .continuous)
leftHand.addChild(leftHandSphere.clone(recursive: true))
leftHandSphere.position = .zero
// 3. Disable the default physics simulation on the anchor
leftHand.anchoring.physicsSimulation = .none
content.add(leftHand)
}
if let rightHandSphere = scene.findEntity(named: "RightHand") {
let rightHand = AnchorEntity(.hand(.right, location: .indexFingerTip),
trackingMode: .continuous)
rightHand.addChild(rightHandSphere.clone(recursive: true))
rightHandSphere.position = .zero
rightHand.anchoring.physicsSimulation = .none
content.add(rightHand)
}
}
}
.modifier(DragGestureImproved())
}
}Support our work so we can continue to bring you new examples and articles.
Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.
Follow Step Into Vision