Transforming Entities with Gesture Component

Joseph Simpson

We can capture the entity transform when a gesture starts, then use that value when moving the entity as the user drags.

Overview

Using gestures like Drag, Magnify, and Rotate with Gesture Component can be similar to using gestures on RealityView. But there are a few things we be aware of and account for. We’ll use Drag Gesture for this example, but the same concepts apply to other gestures for transforming entities.

There seems to be an issue with Drag Gesture when used with Gesture Component. This bug causes any input below a certain threshold to be ignored. We can work around this by using DragGesture(minimumDistance: 0.001).

The values we get from the gesture are incremental. We need a way to save our initial transform from before the gesture started. We’ll add to this on each update. You can see our approach using gesture modifiers here. We do something a little different here.

  • Create a new component that we can use to capture our initial transform
  • When the gesture starts, set this component and transform
  • On each update, use this initial value in our calculations
  • On end, remove this component (or zero-out the transform)

Let’s create the helper component

struct InitialTransformComponent: Component, Codable {
    var transform: Transform
}

Then create our Gesture Component with Drag Gesture. In this case, we simply add the Y axis component to the transform translation value.

if let subject = scene.findEntity(named: "ToyRocket") {
    let drag = DragGesture(minimumDistance: 0.001)
        .onChanged { [weak subject] value in
            guard let subject = subject else { return }

            // 2) On the first change, capture the starting transform.
            if !subject.components.has(InitialTransformComponent.self) {
                subject.components.set(InitialTransformComponent(transform: subject.transform))
            }

            // 3) Pull the saved start transform.
            guard let initialTransform = subject.components[InitialTransformComponent.self]?.transform else { return }

            // 4) Translate the entity along the Y axis
            let translation = value.translation3D
            var newTransform = initialTransform
            newTransform.translation.y += Float(translation.y)
            subject.transform = newTransform

        }
        .onEnded { [weak subject] _ in
            // 5) Clean up the stash when the gesture ends.
            subject?.components.remove(InitialTransformComponent.self)
        }

    subject.components.set(GestureComponent(drag))
}

This is just one way to solve this. We’d love to see how you tackle this issue in your apps.

Video Demo

Full Example Code

// 1) A tiny component to stash the starting transform (local-to-parent).
fileprivate struct InitialTransformComponent: Component, Codable {
    var transform: Transform
}

struct Example117: View {
    var body: some View {
        RealityView { content in

            guard let scene = try? await Entity(named: "TapGestureExample", in: realityKitContentBundle) else { return }
            scene.position.y = -0.4
            content.add(scene)

            if let subject = scene.findEntity(named: "ToyRocket") {
                let drag = DragGesture(minimumDistance: 0.001)
                    .onChanged { [weak subject] value in
                        guard let subject = subject else { return }

                        // 2) On the first change, capture the starting transform.
                        if !subject.components.has(InitialTransformComponent.self) {
                            subject.components.set(InitialTransformComponent(transform: subject.transform))
                        }

                        // 3) Pull the saved start transform.
                        guard let initialTransform = subject.components[InitialTransformComponent.self]?.transform else { return }

                        // 4) Translate the entity along the Y axis
                        let translation = value.translation3D
                        var newTransform = initialTransform
                        newTransform.translation.y += Float(translation.y)
                        subject.transform = newTransform

                    }
                    .onEnded { [weak subject] _ in
                        // 5) Clean up the stash when the gesture ends.
                        subject?.components.remove(InitialTransformComponent.self)
                    }

                subject.components.set(GestureComponent(drag))
            }

        }
    }
}

Support our work so we can continue to bring you new examples and articles.

Support our Work

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?