Maxime FRAPPAT

Hum …no thanks ! – Lordinaire

[Xamarin] Hololens app with UrhoSharp : Spatial Mapping – Part 3

Today, we will try to detect our environment with the spatial mapping. That mechanism allows us to detect real life objects, like a floor or a table, and gets back information in order to build a virtual object that will be the representation of the real object.

Let me do it, please

First of all, we need to add the spatial mapping capability in our project by editing the Package.appxmanifest :

<Package   xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"   
           xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest"   
           xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10"   
           xmlns:uap2="http://schemas.microsoft.com/appx/manifest/uap/windows10/2"
           IgnorableNamespaces="uap mp">
 
 ...  
  <Capabilities>
    <span style="color: #ff0000;"><uap2:Capability Name="spatialPerception"/></span>
  </Capabilities>
</Package>

Override as always

The UrhoSharp framework has already handle basic scenario like spatial mapping in the class HoloApplication (the base class of or app). We only need to override some methods and enable some properties to be able to catch spatial mapping data. Our scenario will be very simple :

  • Start detection
  • Display detected objects (wireframe)
  • Tap to stop detection
  • Hide detected objects
  • Baaaaaaaaaaaaaaalls !

Let’s begin with the Start method :

private bool _isSpatialMappingActive;
 
private Node _detectedSurfaceNode;
 
protected override async void Start()
{
    base.Start();
 
    // Enable the AirTap gesture
    EnableGestureTapped = true;
 
    // Create a new node to store all the detected objects we will create
    _detectedSurfaceNode = Scene.CreateChild();
    _isSpatialMappingActive = true;
 
    // Start the detection
    await StartSpatialMapping(new Vector3(10, 10, 10));
}

For the Update method, just add a new condition :

protected override void OnUpdate(float timeStep)
{
    base.OnUpdate(timeStep);
 
    // Don't do anything if the detection is activated
    if (_isSpatialMappingActive)
        return;
 
    _spawnDeltaTime += timeStep;
    if (_spawnDeltaTime >= _spawnTimer)
    {
        _spawnDeltaTime = 0;
 
        var randomPosition = new Vector3(Randoms.Next(-0.3f, 0.3f), Randoms.Next(-0.3f, 0.3f), Randoms.Next(2, 5));
        CreateBall(randomPosition);
    }
}

The real stuff starting here. We will override the method OnSurfaceAddedOrUpdated like that :

public override void OnSurfaceAddedOrUpdated(string surfaceId, DateTimeOffset lastUpdateTimeUtc, SpatialVertex[] vertexData,
    short[] indexData, Vector3 boundsCenter, Quaternion boundsRotation)
{
    StaticModel model;

    // If the surface already exists get its node otherwise creates a new one
    var node = _detectedSurfaceNode.GetChild(surfaceId, false);
    if (node != null)
    {
        model = node.GetComponent<StaticModel>();
    }
    else
    {
        node = _detectedSurfaceNode.CreateChild(surfaceId);
        model = node.CreateComponent<StaticModel>();
    }

    // Set the position and rotation
    node.Position = boundsCenter;
    node.Rotation = boundsRotation;

    // The model is created with the vertex data
    model.Model = CreateModelFromVertexData(vertexData, indexData);

    // Add a rigidbody for the physic engine
    node.CreateComponent<RigidBody>();

    // Add a collision shape based on the model
    var shape = node.CreateComponent<CollisionShape>();
    shape.SetTriangleMesh(model.Model, 0, Vector3.One, Vector3.Zero, Quaternion.Identity);

    // Add a material for our model (a green wireframe)
    var material = Material.FromColor(Color.Green);
    material.FillMode = FillMode.Wireframe;
    model.SetMaterial(material);
}

To handle the Tap gesture, override the OnGestureTapped method :

public override void OnGestureTapped(GazeInfo gaze)
{
    base.OnGestureTapped(gaze);

    // Stop the detection
    _isSpatialMappingActive = false;
    StopSpatialMapping();

    // Disable wireframe models but keep the rest (RigidBody, CollisionShape)
    // to interact with it
    var childCount = _detectedSurfaceNode.GetNumChildren(false);
    for (uint i = 0; i < childCount; i++)
    {
        var childNode = _detectedSurfaceNode.GetChild(i);
        var model = childNode.GetComponent<StaticModel>();
        model.Enabled = false;
    }
}

That’s it folks !

Previous

[Xamarin] Hololens app with UrhoSharp : Physics – Part 2

Next

[Tips] Can’t share drives in Docker on Windows with Azure AD account

1 Comment

  1. Abra

    Thank you for the very useful Hololens UrhoSharp series.
    Could you please provide the code for the samples ?

Leave a Reply

Powered by WordPress & Theme by Anders Norén

%d bloggers like this: