LiDAR Enabled Templates

In some cases, Lenses can use LiDAR data to enhance the Lens experience--for example on Apple devices with a LiDAR sensor: whether it be to do world mesh reconstruction to enable occlusions by physical objects, have more precise tracking, or understand certain types of surfaces as well as its orientation at a point. 

To see some of the possibilities LiDAR enables, download the World Mesh LiDAR Template.


Guide

Download

Once you have downloaded the zip file above, unzip it, and double-click on the .lsproj file to open it in Lens Studio

Tip: You can push the Lens to your device as you would with any other Lens, but you’ll need a LiDAR enabled device to see the enhanced effect. You can preview how it would look on a LiDAR enabled device in Lens Studio using the Interactive Preview mode in the Preview panel. 


How it Works

The World Mesh LiDAR Template demonstrates two of the key features enabled by LiDAR

  • World Mesh: A mesh that is automatically generated that represents the real world. You can use this to occlude AR effects based on the real world
  • Hit Test Results: Using the ray casting or hit test function, you can learn about the mesh that was hit at a certain point, as well as its position, normal, and the type of surface that is hit. Learn more about this in the API documentation

Note: Like other Lenses where the user should be able to walk around, this template uses the World mode in the Device Tracking Component found in the Camera object.

With the Hit Test Result, you are able to know whether a surface is of type 

  • Wall
  • Floor
  • Ceiling
  • Table
  • Seat
  • Window
  • Door
  • None (of the above)

With these two capabilities, this template allows you to randomly spawn 3D objects around the world based on the surface area where it spawns, then occluding them as they would be in the real world. 

Scene Setup

Each object we want to spawn require three things:

  • The Render Mesh Visuals to spawn
  • A special Graph material for each Render Mesh Visual that enables instantiation
  • A SpawnerSetup script which configures how the object should be spawn

By default, the template comes with several objects and their corresponding spawner for each surface type. 

  • Desert - Seat type
  • Clouds - Wall type
  • Boat - Floor type
  • Forest - Table type
  • Planets - Ceiling type

Tip: You can play around with the Interactive Preview, to see these objects instantiate in your scene as if you were on a device with LiDAR. The classic video and photo preview will demonstrate what the Lens will look like on non LiDAR devices. 

You can find the visuals underneath the Objects Presets object, and their SpawnerSetup underneath the Objects Controller object. 

Each referenced Render Mesh Visual has a special graph material. Since we are populating the world with a lot of these objects, it is more efficient to instantiate them through their material (shader), rather than with additional scene objects.

Note: These Graph materials are unique in that they have their Frustum Culling with Extend option with a large padding so that they don’t disappear when you look away from their origin (since the instantiation can be anywhere), and have parameters that are passed in through the script which describes where they should be instantiated.

At a high level, the SpawnerSetup will tell the special graph material  to instantiate new copies of the Render Mesh Visual, based on the Hit Test Result. 

Spawning Your Own Object

Bringing in your Assets

You can import your own 3D object by dragging them into the Scene panel of Lens Studio. Then, drag them under the Objects Presets objects in the Objects panel. 

Note: Learn more in the Export and Importing 3D content guide. 

Applying the Special Material

With your objects imported, duplicate one of the example special graph materials by right clicking on one in the Resources panel, and then pressing the duplicate button. Then assign the new material to the Render Mesh Visual of the object you’ve imported. 

This template comes with several example materials to help you get started. However, since they are graph materials, feel free to customize them to your needs by double-clicking on your copy in the Resources panel to open it in the Material Editor! 

If you want to instead add the special capability to an existing graph material, you can copy all nodes related to the Vertex Shaders, and increase the Frustum Culling padding of your material. 

Additionally, make all the texture parameters under Controller by Script use the Filtering Mode Nearest (click the V button next to the texture to show the texture’s option). This is important to make sure that the shader can read the precise data passed in by script, rather than a filtered data. 

Tip: You can see an example of a material which has a custom setup in the material used by the water. In the water material, the water texture is made procedurally rather than through a texture. 

Adding Spawner Setup

Finally, with your objects set up, we can spawn them with the SpawnerSetup. 

In the Objects panel, create a new object under Objects Controller. Then select the new object, and in the Inspector panel, press Add Component > Script > Add Script > SpawnerSetup script 

Spawner Setup

The Spawner Setup script contains several options to configure how you want your objects to be spawned.

  • Objects - a list of objects with a render mesh visual on them, with the special graph material described above
  • Group objects - When this option ticked, all of the listed objects will be positioned in the same place. With it unticked, one of the listed objects will be spawned at random. 
  • Surface Type - The type of surface that you want this object to be spawned on. 
  • Ignore World Pos - Use vec3(0,0,0) as the position of the object instance instead of the position of the object in the Inspector panel 
  • Orientation - How you want to generate the rotation for each instance 
  • Placement - Whether you want to spawn the object only when the hit test provides a certain orientation 
  • Spacing - How crowded you want the objects to be. The higher the number, the less crowded. Usually a higher number is better so that you can cover more areas with less objects 
  • Min Scale - Min value of multiplier for the object’s scale
  • Max Scale - Max value of multiplier for the object’s scale
  • Max Instances - The max number of objects to be spawned in the world.

With your Spawner setup, you should now see your objects instantiated around the scene in the Preview panel!

Fallback Mode

Note: By default, the template provides a virtual surface for floor, table, and wall objects. You can skip this section if you don't need a specific configuration.

On devices without LiDAR capabilities you can define how you want your objects to populate the world by describing a “virtual surface.”

You can add your own or modify these definitions in the WorldMeshController script by defining a generator for a fake Hit Test Result.

For example, in the case below, we provide points on a surface between -125 and 125 in the x-axis, -50 in the y-axis, and somewhere between 50 to 250 in the z-axis, all of which have a normal is facing up, to spawn an object for the surface type floor. 

var fakeFloorResult = {
   isValid : function() { return true },
   getWorldPos : function() { return new vec3(Math.random() * 250 - 125, -50, Math.random() * 250 - 50) },
   getNormalVec : function() { return vec3.up(); },
   getClassification : function() { return global.WorldMeshController.surfaceType.Floor; }
}

Then, all we need to do is add this to the list of possible results: 

var fakeResultsPool = [fakeFloorResult, fakeTableResult, fakeWallResult];

In this way, you can define both several types of surfaces, and even a number of surfaces for each type. 

Previewing Your Lens

You’re now ready to preview your Lens! To preview your Lens in Snapchat, follow the Pairing to Snapchat guide.

Related Guides

Please refer to the guides below for additional information:

Still Looking for help?

Visit Support