Gesture Component

Joseph Simpson Updated on

Exploring a new way to add gestures directly to entities.

Overview

Prior to visionOS 26, using SwiftUI Gestures in RealityKit was a bit odd. We can create gestures (or gesture modifiers) and add them to the RealityView where we want to use them. These gestures need to be targeted to an entity, entities with a given component, or all gestures.

We’ve published an entire series on this topic which you can find here.

Starting in visionOS 26, we have a new way to create gestures. We can use Gesture Component to add a gesture to an entity directly, instead of adding the gesture to the view. This adds a simple Tap Gesture to the ToyRocket.

if let subject = scene.findEntity(named: "ToyRocket") {
    let tap = TapGesture().onEnded({ _ in
        print("Rocket tapped")
    })
    let gesture = GestureComponent(tap)
    subject.components.set(gesture)
}

Some things to keep in mind.

  • We can only add one gesture per component / entity. If you need multiple gestures, consider composing them.
  • Many gestures rely on the ability to cache initial transform from before the gesture started. This component provides no mechanism to do that, but we’ll look at some options in a future example.

If your gesture needs to read or edit the host entity, there are two ways we can get access to it.

We can use targetedToEntity(), passing in our subject.

if let subject = scene.findEntity(named: "ToyRocket") {
    let tap = TapGesture()
        .targetedToEntity(subject)
        .onEnded({ value in
            exampleAction1(entity: value.entity)
        })
    let gesture = GestureComponent(tap)
    subject.components.set(gesture)
}

Or we can pass in a weak reference to the entity.

if let subject = scene.findEntity(named: "ToyRocket") {
    let tap = TapGesture().onEnded({ [weak subject] _ in
        if let subject = subject {
            exampleAction1(entity: subject)
        }
    })
    let gesture = GestureComponent(tap)
    subject.components.set(gesture)
}

We generally prefer using the targeted method. It provides the entity we need, while also giving us access to several new methods for coordinate space conversion.

Keep in mind that this component is an addition to RealityKit. If you prefer the previous method adding gestures to the RealityView, you can keep doing that. In fact, there may be times when that is preferable. For example, a gesture that needs to act on a of large number of entities would be easier to set up on the view. Adding the same gesture / component to a many objects may slow down creation of your scene.

See also

Video Demo

This simple demo spins the rocket around the Y axis when we tap it.

Full Example Code

struct Example116: View {
    var body: some View {
        RealityView { content in

            guard let scene = try? await Entity(named: "TapGestureExample", in: realityKitContentBundle) else { return }
            scene.position.y = -0.4
            content.add(scene)

//            // Passing in a weak reference
//            if let subject = scene.findEntity(named: "ToyRocket") {
//                let tap = TapGesture().onEnded({ [weak subject] _ in
//                    if let subject = subject {
//                        exampleAction1(entity: subject)
//                    }
//                })
//                let gesture = GestureComponent(tap)
//                subject.components.set(gesture)
//            }

            // Targeted to entity
            if let subject = scene.findEntity(named: "ToyRocket") {
                let tap = TapGesture()
                    .targetedToEntity(subject)
                    .onEnded({ value in
                        exampleAction1(entity: value.entity)
                })
                let gesture = GestureComponent(tap)
                subject.components.set(gesture)
            }


        }
    }

    // See RealityKit Basics: Perform Entity Actions: https://stepinto.vision/example-code/realitykit-basics-perform-entity-actions/
    func exampleAction1(entity: Entity) {
        Task {
            let action = SpinAction(revolutions: 1,
                                    localAxis: [0, 1, 0],
                                    timingFunction: .easeInOut,
                                    isAdditive: false)

            let animation = try AnimationResource.makeActionAnimation(for: action,
                                                                      duration: 1,
                                                                      bindTarget: .transform)

            entity.playAnimation(animation)
        }
    }
}

Download the Xcode project with this and many more examples from Step Into Vision.

Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Leave a Reply to Joseph SimpsonCancel reply

2 Comments

  1. Hey Joseph!
    I wanted to ask if you ever tested this hypothesis:

    Adding the same gesture / component to a many objects may slow down creation of your scene.

    I wonder if I set up TapGestures onto entities synchronously as I add them to the scene, just how much it would slow down the creation time.

    Reply

    1. I haven’t tested this, its just hasn’t come up in practice. Unless you have thousands or tens of thousands of entities, I don’t think you would run into an issue either way. But if you’re adding the exact same gesture to many entities, there’s not much advantage to using this component versus using the old way.

      Reply