SwiftUI Gestures with RealityKit Entities

Learn the core concepts to use SwiftUI gestures with entities in a RealityView.

SwiftUI has a variety of system gestures that we can use in visionOS development. When working with 2D content in a view, we can use these gestures normally. If we want to use these gestures with 3D content created in RealityView, there are a couple of extra steps.

Adding the necessary components

To make an entity interactive, we need to add two components. We can add these in code or in Reality Composer Pro.

1. Collision – Reality Composer Pro will generate a collision shape for the selected entity. Think of this as the “bounds” of the object we can interact with.
2. Input Target – This tells RealityKit that this entity can receive input.

Learn more about Collisions

In a RealityView:

entity.components.set(CollisionComponent(shapes: [.generateBox(size: .init(repeating: 0.25))]))
entity.components.set(InputTargetComponent())

entity.components.set(CollisionComponent(shapes: [.generateBox(size: .init(repeating: 0.25))]))
entity.components.set(InputTargetComponent())

In Reality Composer Pro, select any entities that should receive gestures. Then add these two components

Input Target Component includes an option called Allow Input. By default, gestures will work with direct (hands touching an entity) and indirect (eye+pinch) input.

Create a gesture

Let’s use the TapGesture as an example. To use a gesture with RealityKit entities, we can use targetedToAnyEntity() to modify the gesture. This will tell SwiftUI to target this TapGesture on RealityKit entities. The value in the gesture will include new data not available in the regular version of the gesture. In this example, we set a SwiftUI state variable to the entity that received the tap.

var tapExample: some Gesture {
    TapGesture()
        .targetedToAnyEntity()
        .onEnded { value in
            selected = value.entity
        }
}

var tapExample: some Gesture {
    TapGesture()
        .targetedToAnyEntity()
        .onEnded { value in
            selected = value.entity
        }
}

There are other options for targeting a gesture. We can target a gesture to any entity that matches a query. For example this version will only fire if the entity has the custom AllowGestures component.

var tapExample: some Gesture {
    TapGesture()
        .targetedToEntity(where: .has(AllowGestures.self))
        .onEnded { value in
            selected = value.entity
        }
}

var tapExample: some Gesture {
    TapGesture()
        .targetedToEntity(where: .has(AllowGestures.self))
        .onEnded { value in
            selected = value.entity
        }
}

We can also target an entity directly

// In the RealityView
if let groundEntity = scene.findEntity(named: "Ground_01") {
  self.groundEntity = groundEntity
}

// Target the gesture
var tapExample: some Gesture {
    TapGesture()
        .targetedToEntity(groundEntity)
        .onEnded { value in
            selected = value.entity
        }
}

// In the RealityView
if let groundEntity = scene.findEntity(named: "Ground_01") {
  self.groundEntity = groundEntity
}

// Target the gesture
var tapExample: some Gesture {
    TapGesture()
        .targetedToEntity(groundEntity)
        .onEnded { value in
            selected = value.entity
        }
}

Use the gesture

There are two main ways we can use gestures. Since visionOS 1.0 we’ve been able to add gestures or gesture modifiers to RealityView.

Use the gesture on the SwiftUI view hierarchy that contains our RealityView.

RealityView { content in
    if let scene = try? await Entity(named: "GestureLabs", in: realityKitContentBundle) {
        content.add(scene)
    }
}
.gesture(tapExample)

RealityView { content in
    if let scene = try? await Entity(named: "GestureLabs", in: realityKitContentBundle) {
        content.add(scene)
    }
}
.gesture(tapExample)

We can use this gesture on any entity in our RealityView that meets these criteria.

• The entity has Input and Collision Components
• The entity meets the criteria used in the targeting method that we used

The other way to work with SwiftUI gestures was introduced in visionOS 26. Gesture Component lets us pass gestures directly to an entity instead of adding them to the RealityView.

if let subject = scene.findEntity(named: "ToyRocket") {
    let tap = TapGesture()
        .targetedToEntity(subject)
        .onEnded({ value in
            exampleAction1(entity: value.entity)
        })
    let gesture = GestureComponent(tap)
    subject.components.set(gesture)
}

if let subject = scene.findEntity(named: "ToyRocket") {
    let tap = TapGesture()
        .targetedToEntity(subject)
        .onEnded({ value in
            exampleAction1(entity: value.entity)
        })
    let gesture = GestureComponent(tap)
    subject.components.set(gesture)
}

This can be helpful when each entity needs a unique gesture.

We’ve create an entire series on working with system gestures.

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.