Maxime FRAPPAT

Hum …no thanks ! – Lordinaire

Tag: UrhoSharp (Page 1 of 2)

[Xamarin] Hololens app with UrhoSharp : Spatial Mapping – Part 3

Today, we will try to detect our environment with the spatial mapping. That mechanism allows us to detect real life objects, like a floor or a table, and gets back information in order to build a virtual object that will be the representation of the real object.

Let me do it, please

First of all, we need to add the spatial mapping capability in our project by editing the Package.appxmanifest :

<Package   xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"   
           xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest"   
           xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10"   
           xmlns:uap2="http://schemas.microsoft.com/appx/manifest/uap/windows10/2"
           IgnorableNamespaces="uap mp">
 
 ...  
  <Capabilities>
    <span style="color: #ff0000;"><uap2:Capability Name="spatialPerception"/></span>
  </Capabilities>
</Package>

Override as always

The UrhoSharp framework has already handle basic scenario like spatial mapping in the class HoloApplication (the base class of or app). We only need to override some methods and enable some properties to be able to catch spatial mapping data. Our scenario will be very simple :

  • Start detection
  • Display detected objects (wireframe)
  • Tap to stop detection
  • Hide detected objects
  • Baaaaaaaaaaaaaaalls !

Let’s begin with the Start method :

private bool _isSpatialMappingActive;
 
private Node _detectedSurfaceNode;
 
protected override async void Start()
{
    base.Start();
 
    // Enable the AirTap gesture
    EnableGestureTapped = true;
 
    // Create a new node to store all the detected objects we will create
    _detectedSurfaceNode = Scene.CreateChild();
    _isSpatialMappingActive = true;
 
    // Start the detection
    await StartSpatialMapping(new Vector3(10, 10, 10));
}

For the Update method, just add a new condition :

protected override void OnUpdate(float timeStep)
{
    base.OnUpdate(timeStep);
 
    // Don't do anything if the detection is activated
    if (_isSpatialMappingActive)
        return;
 
    _spawnDeltaTime += timeStep;
    if (_spawnDeltaTime >= _spawnTimer)
    {
        _spawnDeltaTime = 0;
 
        var randomPosition = new Vector3(Randoms.Next(-0.3f, 0.3f), Randoms.Next(-0.3f, 0.3f), Randoms.Next(2, 5));
        CreateBall(randomPosition);
    }
}

The real stuff starting here. We will override the method OnSurfaceAddedOrUpdated like that :

public override void OnSurfaceAddedOrUpdated(string surfaceId, DateTimeOffset lastUpdateTimeUtc, SpatialVertex[] vertexData,
    short[] indexData, Vector3 boundsCenter, Quaternion boundsRotation)
{
    StaticModel model;

    // If the surface already exists get its node otherwise creates a new one
    var node = _detectedSurfaceNode.GetChild(surfaceId, false);
    if (node != null)
    {
        model = node.GetComponent<StaticModel>();
    }
    else
    {
        node = _detectedSurfaceNode.CreateChild(surfaceId);
        model = node.CreateComponent<StaticModel>();
    }

    // Set the position and rotation
    node.Position = boundsCenter;
    node.Rotation = boundsRotation;

    // The model is created with the vertex data
    model.Model = CreateModelFromVertexData(vertexData, indexData);

    // Add a rigidbody for the physic engine
    node.CreateComponent<RigidBody>();

    // Add a collision shape based on the model
    var shape = node.CreateComponent<CollisionShape>();
    shape.SetTriangleMesh(model.Model, 0, Vector3.One, Vector3.Zero, Quaternion.Identity);

    // Add a material for our model (a green wireframe)
    var material = Material.FromColor(Color.Green);
    material.FillMode = FillMode.Wireframe;
    model.SetMaterial(material);
}

To handle the Tap gesture, override the OnGestureTapped method :

public override void OnGestureTapped(GazeInfo gaze)
{
    base.OnGestureTapped(gaze);

    // Stop the detection
    _isSpatialMappingActive = false;
    StopSpatialMapping();

    // Disable wireframe models but keep the rest (RigidBody, CollisionShape)
    // to interact with it
    var childCount = _detectedSurfaceNode.GetNumChildren(false);
    for (uint i = 0; i < childCount; i++)
    {
        var childNode = _detectedSurfaceNode.GetChild(i);
        var model = childNode.GetComponent<StaticModel>();
        model.Enabled = false;
    }
}

That’s it folks !

[Xamarin] Hololens app with UrhoSharp : Physics – Part 2

In the previous post, we manage to display a ball each second at a random position. Our goal, if you accept it, will be to add some physical properties to the ball (mass, gravity, collision).

Balls are falling !

The first step is to define a mass for our balls. Without that, the physical engine can’t apply to the object because the mass is null. In the CreateBall method, you can add the code snippet below that adds a RigidBody component and sets his mass to 0.1. It’s important to note that we don’t create a new node to attach the RigidBody because the physical property must be apply to the entire object (=the root node).

// Add a rigidbody
var rigidbody = ballNode.CreateComponent<RigidBody>();
rigidbody.Mass = 0.1f;

You can use a RigidBody2D if your project is in 2D. That affects the physical engine because some axis are ignored during calculations.

Collision detected sir.

To handle the collision between objects, we need to add a CollisionShape that can be for example a sphere, a cube or a more complex shape. In our case, we will use a sphere that will fit our ball.

// Add a collision shape
var collision = ballNode.CreateComponent<CollisionShape>();
collision.SetSphere(1, Vector3.Zero, Quaternion.Identity);

In term of pure performances, it’s better to use a simple shape (or an addition of them) than a complex shape that represents exactly the model. An approximation is often used in real scenario : a cube can be a simple solution to handle a head for exemple.

So, our balls can collide so let’s add a fake floor to show what happens.

private void CreateFloor()
{
    var floorNode = Scene.CreateChild();
    floorNode.Position = new Vector3(0, -1, 0);
    floorNode.SetScale(20);

    var floorModelNode = floorNode.CreateChild();
    var plane = floorModelNode.CreateComponent<Urho.Shapes.Plane>();
    plane.Color = Color.White;

    var rigidbody = floorNode.CreateComponent<RigidBody>();
    rigidbody.Kinematic = true;

    var collision = floorNode.CreateComponent<CollisionShape>();
    collision.SetBox(new Vector3(20, 0.1f, 20), new Vector3(0, -0.05f, 0), Quaternion.Identity);
}

That method seems pretty similar to CreateBall except the use of the property Kinematic for the RigidBody. The physical engine will not apply the gravity to this object, and it’s right what we need for a floor. Don’t forget to invoke the method at the beginning of the application, inside the Start method.

protected override void Start()
{
    base.Start();

    CreateFloor();
}

urhosharp_balls_on_the_floor

In the next episode, we will use the spatial mapping to detect the floor automatically.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén

%d bloggers like this: