Step Into Newsletter – December 22, 2024

Welcome to the last newsletter of 2024. Let’s jump right in with the labs, then we’ll talk about hand tracking.

From the Labs

Lab 016 – Entity Spawning and Pooling System – I started working on a component and system to use across labs. Unfortunately, I coded myself into a corner. I’ll tackle this idea again soon.

Lab 017 – Exploring a Skybox Material with Shader Graph – Thanks to @dreamwieber.bsky.social I took my first steps in creating something in Shader Graph. Once I refine this I hope to use this in the Project Graveyard redesign.

Lab 018 – Pulling an entity out of the ground – One of my favorite uses of occlusion material is to hide entities behind it until a certain action or event.

Lab 020 – Exploring Collision Triggers – I haven’t had an opportunity to explore collisions and physics in RealityKit. So I used this lab to learn the basics of trigger (non-physics based) collisions.

Lab 021 – Collision Triggers with AnchorEntity Hands? – Do collision triggers fire from hand anchored entities without a Spatial Tracking Session?

Example code: Hand Tracking

The new series on Hand Tracking is well underway.

ARKit Hand Tracking Basics – I started off with a simple RealityKit scene that uses Hand Tracking Provider to update anchors on our hands. This works, but I quickly found a better way to use ARKit features in RealityKit.

Using RealityKit AnchorEntity with Hands – We can use AnchorEntity to access a number of locations and joints on our hands. We can use this with or without a Spatial Tracking Session (ARKit).

AnchorEntity Locations and Joints for Hands – Unpacking all the locations and joints that we can access with Anchor Entity in RealityKit.

Hand Anchored Collision Triggers – We can combine Anchor Entity and Spatial Tracking Session if we need to track collisions between our hands and other scene entities.

Next in this series I’ll dive into using hand anchors with physics.

Resources and Community

Sizzly – Particle Playground – Explore RealityKit Particles while immersed in your Apple Vision Pro. Save you configuration as a text file to reference later.

Happy Clouds on Apple Vision Pro: Demystifying Shader Graph Materials in Reality Composer Pro – Published by @dreamwieber.bsky.social. Gregory recently launched a Patreon. Please support his amazing work if you can.

Phil Traut launched a new website called Spatial Apps to showcase his favorite work from our growing community.

Tom Krikorian and Tina Debove Nigro lauched an early version Page Blanche, their prototyping tool for Apple Vision Pro.

Thanks to @milanlajtos for showing me how to access all hand joints on AnchorEntity

Thanks to @jessymeow.bsky.social for helping me with an issue with a custom component.

It has been one year since I started working on Project Graveyard for Apple Vision Pro. The initial commit was on Dec 22, 2023 and the first version was released on February 2, 2024. Project Graveyard is a tiny place where you can bury and remember your dead projects. Version two is available now, and I have some major visual changes planned for 2025.

Personal Note

We are approaching the end of the year. I just want to take a moment to thank you for joining me on this journey. We are still in the very early days of the Spatial Computing era. I’m grateful to have found so many inspiring companions. Let’s step into 2025 together and continue building the future of computing.