Lab 033 – Teleportation with SpatialTapGesture

Joseph Simpson Updated on

We can’t move the player/user entity in RealityKit, but we can move the world around them instead.

Overview

When we create teleportation or locomotion features in XR, we often end up setting the transform of an entity that represents the user. Unfortunately, I haven’t found a way to do that in RealityKit. Apple has built visionOS with mixed mode (AR) as the primary use case. With that in mind, it makes sense that we don’t have a way to access the player transform directly. But what if we need to add a way to move a user around a fully immersive (VR) space?

The best solution I’ve found is to invert the math and move the world instead of moving the player. What do we need to perform a teleportation? At minimum, we need a target transform and a coordinate space.

This lab uses SpatialTapGesture to get the first item. We’ll create a gesture that targets a specific ground entity. When we tap on the ground, we can calculate the location of that tap. We can then convert that to a 3D position relative to our space. 

let vectorToTap = value.convert(value.location3D, from: .local, to: sceneContent)

We can normalize the vector to get a direction.

let direction = normalize(vectorToTap)

We can get the distance to the tapped position by getting the length relative to the global space.

let distance = length(vectorToTap)

Then we can multiply these to get a new position. Remember, we can’t use this position to move the player. Instead, we’ll invert this so we can use it to move the world. In this case, that means moving the root entity of our scene.

let newPosition = -direction * distance

Now that we have a new position, we can create a new vector for the teleportation. We’ll ignore the Y axis since we only need to move them on a flat surface for now.

let teleportPosition: SIMD3<Float> = [newPosition.x, 0, newPosition.z]
sceneContent.position = teleportPosition

Note: Breaking this into so many steps may seem redundant, but it gives us a bit more control. For example, we could add an offset to the distance or change orientation separately from position. If all we need to do is change position we could do something like this:

// Get the tapped location
let vectorToTap = value.convert(value.location3D, from: .local, to: sceneContent)
// Invert it
let newPosition = -vectorToTap

This is a pretty barebones teleportation feature. There are a lot of things we could do to improve it.

  • Add a visual indicator
  • Combine SpatialTapGesture with DragGesture to allow the user to move the target indicator
  • Consider using double tap to start the gesture
  • Provide a way to abort the action
  • Provide a way to adjust orientation
  • Teleport relative to the Y axis of entities at different heights
  • Target entities for teleportation by using a custom component

Video Demo

Full Lab Code

struct Lab033: View {

    @State var sceneContent: Entity?
    @State var groundEntity: Entity = Entity()

    var body: some View {
        RealityView { content in

            if let scene = try? await Entity(named: "TeleportLabs", in: realityKitContentBundle) {
                content.add(scene)

                // Get the scene content and stash it in state
                if let sceneContent = scene.findEntity(named: "Root") {
                    self.sceneContent = sceneContent
                }

                // Get the ground entity from the dome. We'll use this to target our gesture on this entity only.
                if let groundEntity = scene.findEntity(named: "Ground_01") {
                    self.groundEntity = groundEntity
                }
            }

        }
        .gesture(teleportTap)
    }

    var teleportTap: some Gesture {
        SpatialTapGesture()
            .targetedToEntity(groundEntity)
            .onEnded { value in

                print("TELEPORTING TO: \(value.entity.name)" )

                guard let sceneContent = self.sceneContent else { return }

                //Calculate the vector from the origin to the tapped position
                let vectorToTap = value.convert(value.location3D, from: .local, to: sceneContent)
                print("TELEPORTING vectorToTap: \(vectorToTap)" )

                // Normalize the vector to get a direction from the origin to the tapped position
                let direction = normalize(vectorToTap)

                // Calculate the distance (or magnitude) between the origin and the tapped position
                let distance = length(vectorToTap)

                // Calculate the new position by inverting the direction multiplied by the distance
                let newPosition = -direction * distance
                print("TELEPORTING newPosition: \(newPosition)" )

                // Update sceneOffset's X and Z components, leave Y as it is
                let teleportPosition: SIMD3<Float> = [newPosition.x, 0, newPosition.z]

                print("TELEPORTING POS: \(teleportPosition)" )
                sceneContent.position = teleportPosition

            }
    }
}

This lab was based on an example I did last year on my personal site. The new version makes use of the location of the tap on a single entity instead of using the global position of the tapped entity.

Support our work so we can continue to bring you new examples and articles.

Support our Work

Follow Step Into Vision

Questions or feedback?