ARKit PlaneDetectionProvider: occlusion material

Joseph Simpson

We can use Occlusion material to hide planes while letting them participate in collisions and physics.

Overview

Let’s build on the example from creating simple planes from anchors. We’ll add collision and physics components, then replace the material with OcclusionMaterial. This will render the passthrough feed in place of the debug materials we were using. This is a great way to include entities for collisions and physics without rendering them in the scene.

func createSimplePlaneEntity(for anchor: PlaneAnchor) -> Entity {

    let extent = anchor.geometry.extent
    let mesh = MeshResource.generatePlane(width: extent.width, height: extent.height)
    let material = OcclusionMaterial()

    let entity = ModelEntity(mesh: mesh, materials: [material])
    entity.transform = Transform(matrix: matrix_multiply(anchor.originFromAnchorTransform, extent.anchorFromExtentTransform))

    entity.generateCollisionShapes(recursive: true, static: true)
    let physicsMaterial = PhysicsMaterialResource.generate(friction: 0, restitution: 1)
    let physics = PhysicsBodyComponent(massProperties: .default, material: physicsMaterial, mode: .static)
    entity.components.set(physics)

    return entity
    
}

We’ll let RealityKit generate a simple collision shape based on the entity using generateCollisionShapes. To generate more detailed shapes see adding collisions and physics.

Video Demo

Full Example Code

struct Example073: View {
    @State var session = ARKitSession()

    @State var planeAnchorsSimple: [UUID: Entity] = [:]

    // An entity with physics that can bounce around the room
    @State var subject : ModelEntity = {
        let subject = ModelEntity(
            mesh: .generateSphere(radius: 0.1),
            materials: [SimpleMaterial(color: .stepRed, isMetallic: false)])
        subject.setPosition([1, 1, -1], relativeTo: nil)

        let collision = CollisionComponent(shapes: [.generateSphere(radius: 0.1)])

        var physics = PhysicsBodyComponent(
            shapes: [.generateSphere(radius: 0.1)],
            mass: 1.0,
            material: .generate(friction: 0, restitution: 1),
            mode: .dynamic
        )
        physics.isAffectedByGravity = false

        let input = InputTargetComponent()
        subject.components.set([collision, physics, input])

        return subject
    }()


    var body: some View {
        RealityView { content in
            content.add(subject)
        } update: { content in

            for (_, entity) in planeAnchorsSimple {
                if !content.entities.contains(entity) {
                    content.add(entity)
                }
            }
        }
        .gesture(TapGesture()
            .targetedToEntity(subject)
            .onEnded { value in
                // Add some force when we tap the subject
                let force = SIMD3<Float>(
                    x: Float.random(in: -1...1),
                    y: Float.random(in: -1...1),
                    z: Float.random(in: -1...1)
                )
                var motion = PhysicsMotionComponent()
                motion.linearVelocity = force * 3
                value.entity.components.set(motion)
            })
        .task {
            try! await setupAndRunPlaneDetection()
        }
    }

    func setupAndRunPlaneDetection() async throws {
        let planeData = PlaneDetectionProvider(alignments: [.horizontal, .vertical, ])
        if PlaneDetectionProvider.isSupported {
            do {
                try await session.run([planeData])
                for await update in planeData.anchorUpdates {
                    switch update.event {
                    case .added, .updated:
                        let anchor = update.anchor

                        let planeEntitySimple = createSimplePlaneEntity(for: anchor)
                        planeAnchorsSimple[anchor.id] = planeEntitySimple

                    case .removed:
                        let anchor = update.anchor
                        planeAnchorsSimple.removeValue(forKey: anchor.id)

                    }
                }
            } catch {
                print("ARKit session error \(error)")
            }
        }
    }

    func createSimplePlaneEntity(for anchor: PlaneAnchor) -> Entity {

        let extent = anchor.geometry.extent
        let mesh = MeshResource.generatePlane(width: extent.width, height: extent.height)
        let material = OcclusionMaterial()

        let entity = ModelEntity(mesh: mesh, materials: [material])
        entity.transform = Transform(matrix: matrix_multiply(anchor.originFromAnchorTransform, extent.anchorFromExtentTransform))

        // We'll let RealityKit generate a simple collision shape based on the entity.
        // For more detailed shapes see: https://stepinto.vision/example-code/arkit-planedetectionprovider-adding-collisions-and-physics/
        entity.generateCollisionShapes(recursive: true, static: true)
        let physicsMaterial = PhysicsMaterialResource.generate(friction: 0, restitution: 1)
        let physics = PhysicsBodyComponent(massProperties: .default, material: physicsMaterial, mode: .static)
        entity.components.set(physics)

        return entity
    }
}

Support our work so we can continue to bring you new examples and articles.

Support our Work

Download the Xcode project with this and many more examples from Step Into Vision.
Some examples are provided as standalone Xcode projects. You can find those here.

Follow Step Into Vision

Questions or feedback?