Drag Gesture with Pivot

Joseph Simpson Updated on

We can use the location3D values of the gesture to improve dragging when the user turns to face another direction.

Overview

Issue: using the Improved Drag Gesture works well in volumes or for small movements in immersive spaces. But when a user turns to face another direction, this version breaks down. The gesture doesn’t take into account the direction that the user is facing. Let’s see if we can improve on this.

Apple published a project with several examples of system gestures. It has two options for dragging: handleFixedDrag and handlePivotDrag. I recreated the concepts from handlePivotDrag but it had one major issue. I could no longer translate the entity further away or closer to me. I asked how to combine these ideas on a post on the developer forum, but so far no one has replied.

There is another gesture component in an unrelated example project. This version allows translation in space while smoothing the movement in an arc around the user. It’s not quite as good as the gesture Apple uses on Windows and Volumes. However, it is far better than what I’ve been using until now.

Instead of using value.gestureValue.translation3D to move the entity, this version uses value.location3D and value.startLocation3D. We get the current location in the gesture.

let gesturePosition = value.convert(value.location3D, from: .global, to: entityParent)

Minus the start location of the gesture

let deltaPosition = gesturePosition - value.convert(value.startLocation3D, from: .global, to: entityParent)

Plus the initial position of the dragged entity

let newPos = initialPosition + deltaPosition

Video Demo

This shows the fixed drag from this example, then the new improved pivot drag. If you know of anyway to improve this further, please let me know!

Full Example Code

struct Example046: View {
    var body: some View {
        RealityView { content in

            if let scene = try? await Entity(named: "GestureLabs", in: realityKitContentBundle) {
                content.add(scene)

                // Lower the entire scene to the bottom of the volume
                scene.position = [1, 1, -1.5]

            }

        }
        .modifier(DragGestureWithPivot046())
    }
}

fileprivate struct DragGestureWithPivot046: ViewModifier {

    @State var isDragging: Bool = false
    @State var initialPosition: SIMD3<Float> = .zero

    func body(content: Content) -> some View {
        content
            .gesture(
                DragGesture()
                    .targetedToAnyEntity()
                    .onChanged { value in
                        // We we start the gesture, cache the entity position
                        if !isDragging {
                            isDragging = true
                            initialPosition = value.entity.position
                        }

                        guard let entityParent = value.entity.parent else { return }

                        // The current location: where we are in the gesture
                        let gesturePosition = value.convert(value.location3D, from: .global, to: entityParent)

                        // Minus the start location of the gesture
                        let deltaPosition = gesturePosition - value.convert(value.startLocation3D, from: .global, to: entityParent)

                        // Plus the initial position of the entity
                        let newPos = initialPosition + deltaPosition

                        // Optional: using move(to:) to smooth out the movement
                        let newTransform = Transform(
                            scale: value.entity.scale,
                            rotation: value.entity.orientation,
                            translation: newPos
                        )

                        value.entity.move(to: newTransform, relativeTo: entityParent, duration: 0.1)

                        // Or set the position directly
                        // value.entity.position = newPos
                    }
                    .onEnded { value in
                        // Clean up when the gesture has ended
                        isDragging = false
                        initialPosition = .zero
                    }
            )

    }
}

Support our work so we can continue to bring you new examples and articles.

Support our Work

Follow Step Into Vision

Questions or feedback?

2 Comments

  1. Hi!
    Great work!

    Is it possible to chance the rotation to maintain the objectivo facing the user?

    Thanks!
    Keep up the good work!

    Reply

    1. Adding rotation into this can be tricky to get right. We either need access to device pose or the head tracked position from ARKit. I have an example like this on my backlog, but with visionOS 26 we have a better option. Manipulation–in the form of both SwiftUI modifiers and a new RealityKit Component.

      First look at Manipulation Component
      https://stepinto.vision/labs/lab-060-first-look-at-manipulation-component/

      First look at the Manipulable modifier
      https://stepinto.vision/labs/lab-065-first-look-at-the-manipulable-modifier/

      Reply