Interesting facts about Azure Digital Twin Service Preview

Currently the Azure Digital Twin Service is still in preview. There are some interesting facts and small issues you need to know before starting to try this out. To understand what Azure Digital Twin offers, it would be best to read my extended post about an introduction to Microsoft Digital Twin. So let’s start!

Availability of Digital Twin Preview

Or the lack of it. You are lucky if you already had a created Azure resource based on the Digital Twin service. At the moment Microsoft states :

Thank you for your interest in the Azure Digital Twins preview program. Due to overwhelming demand, the preview program is temporarily closed as we prepare for the upcoming release of new capabilities. As a result, you may not be able to create new Azure Digital Twins resources right now. Please continue checking back for new information.

So be careful not to throw your resource away. A good chance is that you are not able to create one again. I tried several locations worldwide without luck. Luckily I already had one.

Simulated device(s)

Microsoft has an interesting example which allows you to play around with the Azure Digital Twin service without having an actual device. It uses a separate project which communicates through the IoT hub to provide the Digital Twin simulated data.

IoT Hub… where is it?

Talking about IoT Hub. Wonder where that one is? It is there, but you cannot access it. It is automatically generated when the Azure Digital Twin was created. It will not appear in the list of resources in your Azure environment. Even worse… you can’t access it via Azure Powershell.

Digital Twin viewer

There is a great Digital Twin viewer. But you will get into an issue, when you have done the example before the viewer. The redirect URL of the example is the same as the viewer which causes issues by logging into Azure via the browser using the example. The best way is to change the port number of the redirect URL of the project. Add an additional URL to the App registration under the Azure Active Directory to support this.

Limitations

There are several limitations using the Azure Digital Twin Preview. You are allowed to run only one Azure Digital Twin service at the time. There are rate and message limitations for Azure Digital Twin Management API, user-defined functions and device telemetry. Read about them here.

Conclusion

Playing around with Azure Digital Twin Preview is just great! As long as you understand the limitations you can try several things yourself without having an actual IoT device. It gives you a good understanding and ideas where you can use it for. We need to wait for general availability to start using it in businesses. There are some other interesting findings posted in the feedback hub which you can find here.

The post Interesting facts about Azure Digital Twin Service Preview appeared first on 365 XR Blog.

Microsoft Guides, the next step in training & guidance with Mixed Reality

It is more than 4 years ago that Microsoft came with the first release of Microsoft HoloLens device in the world of Mixed Reality. It took some time before Microsoft released several out-of-the-box applications for businesses. And they are ruling the Mixed Reality landscape since then. Their focus stays continuously on providing solutions which addresses the most common challenges for businesses.

Business challenges

One of these business challenges is training personnel. Training personnel in organizations require a lot of effort from experts on the workforce. And with a large staff turnover, something which happens a lot on the factory floor, is training a costly part of their business. But even keeping your workforce up-to-date with the latest requirements for training, or to prep students on school for what is coming or getting more insights by combining it with for example a Digital Twin will improve, support and saving costs for your organization allowing you to drive better business.

It is important for organizations to understand the business value of Microsoft Guides. Eventually organizations want to empower their workforce, optimize digital operations and deliver new services internally or externally to their customers. And that will result in a different way of thinking within manufacturing, training, support and guidance of employees.

Microsoft Guides

Microsoft Guides is a Mixed Reality business application which allows you to create a tailor made guide for employees on the factory floor. It allows you to create holographic work instructions which support a whole or part of the work flow. There are absolutely no coding skills requires for creating these Mixed Reality guides.

Microsoft Guides is part of the Microsoft Dynamics 365 family and therefor actually called Microsoft Dynamics 365 Guides. It uses the base storage system of Dynamics 365, its underlying Common Data Service and Power Apps to store its flow and content.

The solution works for Microsoft HoloLens 1 and 2. But understandable it is extended based on the new functionality provided by Microsoft HoloLens 2 like natural gestures and other. The natural gestures are currently only used for the author when building guide.

Since then the Microsoft Team responsible for Microsoft Guides is extending the functionality of the application and its platform on a monthly basis. They are keen on getting feedback from customers, MVP’s and consultants. And they take the feedback seriously.

How does it work?

In short a guide is created by an author. The author starts outlining a guide into different tasks using a desktop application. Outlining is based on requirements of the customer and the guidelines of building a guide. Then the author switches to the Microsoft HoloLens 2 device. There the guide is attached to for example an object in the real physical world by editing each task of the outlines. This is accomplished by moving, placing, sizing and rotating the holograms around in the real physical world. Adding additional holographic instructions like arrows, hands and more.

An operator logs in Microsoft Guides and select the guide. The operator needs to synchronize his training using a tag or positioning a holographic object on the object in the real world. After this the guide is executed through the outlines and tasks. The operator experiences the cards with the steps explained floating around. A tether (dotted line) is used to indicate where the step needs to be executed on the real assets. Finally the author is able to monitor and analyse the progress and execution. It allows the author to improve the guides and support the operator in improving their skills.

Different approaches

Microsoft Guides allows you to use different approaches.

  • Real-life assets – This is the best way of having hands-on training for employees. The guide is synchronized on a real-life object using a tag or holographic object.
  • Virtual assets – guides with virtual assets allow you to have education or training without the actual real-life asset. The asset is digitally placed as a holographic object in each task. This is specifically interesting in situations where you are not able to have the actual asset. It is also a more passive way of training since you are not able to execute the tasks on the real-life asset.

Before you start

One of the biggest mistakes is thinking that creating a guide is simply creating a few outlines and task. It is much more comprehensive. Before you start you will need to gather as much content as possible to create your guide. This is accomplished by having for example workshops and inspiration sessions with your customer or target group. There are a few things which are important for creating your guide. You need to understand the space where the guide is build for. You need to understand the procedure and workflow of the actual work. These things will influence in how you build your guide. If possible try to involve an expert of the organization. Actually it is a must to create a good guide. Keep that in mind!

Also try to understand the objectives and sub-objectives of your guide. This will influence the amount of outlines and tasks you want to use. You would think why? But it is important to not lose your operator using the training into doing too much tasks within the same outline. Something we will explain further in detail. And the final one is taking into consideration for which role you are creating a training. The role determines what you need to highlight from the asset and the depth of information you want to provide.

Anchor

Guides makes use of an anchor. The anchor is used to position the guide at the right location in the real world. This is accomplished by using a QR code or a Circular code. The later one is a predefined tag by the Microsoft Guides team. It is also possible to use a holographic object as a marker. But that requires to position the holographic object at the exact position which is in most cases very difficult. This is mostly interesting when it is not possible to place a tag on the actual asset.

Structure of a guide

The structure of guides exists of one or more cards placed in an outline. Your guide can have more than one outline. the outlines and cards are executed in a logical sequence. There are rules for creating fabulous and great working guides. Information about making guides great can be found here. Not going too much in detail, but it explains from begin to end and has a large number of great tips and tricks. Examples are that each outline must be a specific task within the work flow. It should have a clear beginning and end. It explains how to get around with holographic objects and to be consistent in using models, styles, texts and more.

The outlines and tasks are created via the desktop application. The desktop application allows you to add content. Content can be anything like images, sounds, video and holographic models. It offers out of the box several holographic helpful parts like arrows, generic tools, hands, numbers, symbols and animated zones.

Actions

A really cool feature which is just added two months ago is having actions in a card. At the moment there are only two actions. But i would expect more in the future. And who knows… maybe even add your own actions. But for now we have the action website link an Power App.

The website link allows you to add one specific link per card which can be opened during the presentation of the card. Imagine using a link to more information about what the employee is doing. Or maybe real-time information coming from a sensory device shown at the real object or in the virtual model.

Power Apps is a major cool feature. It allows you to show a Power App during a card in the outline. Just like the website link you can have a different Power App per card. Imagine a Power App with questions to be answered by the employee. You could for example use a single Power App with several questions. Each question can be reached by using a parameter in the URL to the Power App. And the results are stored in a database based on the credentials of the logged on user. There are so many more things you could do with this. And just like Microsoft Guides, Power Apps has a creator interface for power users. No additional coding and technical skills are required.

Roles and rights

At the moment Microsoft Guides knows two roles. We have the author and the operator.

The author is allowed to create, rename and edit guides. The author can also activate or deactivate guides. Just like the operator the author can also operate guides.

The operator is allowed to view and operate assigned guides via the Microsoft HoloLens.

Improve efficiency by analytics

The performance of the operator is measured due to data collection during the operation of a guide. Each gaze and commit interaction on buttons is measured. Time related information about the run, guide task and step are stored. This allows the author to view usage statistics and detailed time-track information via Power BI reports. These metrics can be used by the author to optimize the created guide. But it can also be used to see how the operator’s performance is improving doing the work. Sharing the result using Power BI Reports requires an appropriate license.

There are two default reports available. The process time-tracking report and the Guides usage report. These reports can answer several questions like

  • Is daily guide usage changing?
  • What is the most frequently used guide?
  • What is the average run time per guide?
  • How long is a guide run in minutes?
  • How long did each task or step take?

Power Apps & Power automate

Microsoft Guides can be integrated into existing process and workflows using Power Apps and Flow. You can start a workflow or even use the “Create work record” event. This integration is mainly on the back-end of Guides and differs therefor on the Power App action discussed earlier.

Dynamics 365 Field Services

There is also an integration with Dynamics 365 Field services. It allows you to attach guides to Field Services tasks. This gives you the ability to complete work orders via Microsoft Guides.

Conclusion

Microsoft Guides is the tool for building great guides in Mixed Reality. While the tool is just released last year, it offers an extensive amount of functionality which is extended every month. Dynamics 365 and Power Apps are the base for Microsoft Guides. And do not get mistaken. Power Apps is going to be big and very important through the Microsoft landscape. The same is for Microsoft Guides as tool to build guides and more.

The post Microsoft Guides, the next step in training & guidance with Mixed Reality appeared first on 365 XR Blog.

An introduction to the Microsoft Azure Digital Twin service

Digital Twin? That’s a buzzword for something isn’t it? Could be. But it is actually an interesting type of service which allows you as an IT company explain a business solution to your customers without going into depth about Machine Learning, Mixed Reality, Internet of Things, Sensory data and many other technologies. It is a business solution which can easily solve several use cases for your customer. And they aren’t particularly interested in what technologies you would need to use, as long as it does the job.

What is a Digital Twin?

So what is a Digital Twin? A Digital Twin is a digital representation, also called a copy, of physical entity(s) in the real world. A physical entity can be anything from people, places, machines, factories, devices, systems and even processes. One of the characteristics of a Digital Twin is that the replica represents a, if possible a true, copy of one or more actual physical entities. Such a copy is visualized using a 2D form like a web browser or a 3D form using for example a Microsoft HoloLens device or mobile device supporting augmented reality. These physical entities generate real-time data which is feed back into the model of the Digital Twin. In some cases a Digital Twin is compared to concepts like cross-reality environments and mirror models.

The idea is creating a “living” simulation model. The model continuous update and change based on the actual values of real–life entities. Such a simulation model can be used in several different use-cases. Use cases for example

  • Simulation – Simulate different scenarios which you normally never would do an the actual asset without damaging it or causing collateral damages.
  • Analyse & Optimize – optimize systems, machines or processes by tweaking values in the simulation model. This allows you to see the effect on those entities without changing the actual configuration. When a better and more optimized configuration is found it can be applied to the actual real-life entities.
  • Training – simulation models allow you to train new personnel without using the actual machine or control room. Some scenarios You could also use it in education at schools
  • Testing – use the simulation model to find the utter edges of the system before it fails.
  • Visualization – get more and clearer insights in current statuses of systems and machine over multiple locations spread.
  • Complex models – Bring complex scenarios with lots of sensory devices divided over structured locations into a single model. Combine and use the data to create easy and simple to understand views.
  • Professional services – Use sensory data from devices at different locations to provide services to your company. The Microsoft example which is available for Digital Twin is a great example. In this example different types of sensory data, temperature, movement and other, is used to determine if a room is available and suited for you.

You can understand that there are far more examples in which a Digital Twin can offer benefits.

Microsoft Azure Digital Twin Service

The Azure Digital Twin service is an IoT service which helps you in creating models of physical environments. It uses something called a spatial intelligence graph. The spatial intelligence graph allows you to model a structure of relationships and interactions between people, places and devices. This allows you to query real-time data from devices which are bound into a structured environment instead of a single device without any relationships whatsoever. The service is part of Azure and supports high scalability and re-usability so called “spatially aware experiences”. With other words duplicating real-time experiences around assets (e.g. machines, processes or other) into a model which knows where exactly the data from sensory devices is coming from. More information can be found here.

Keep in mind that Azure Digital Twin service does not deliver a complete Digital Twin solution. It still requires to connect to sensory devices using for example a IoT Hub. That means you will need to setup a IoT Hub. You also need to think about how you want to visualize and interact with your model. Where are you going to output the results? And how do you want to visualize the structured model of people, places and device. So what does Azure Digital Twin service offer? It offers

  • Modelling the relationships and interactions between people, places and devices using the spatial intelligence graph
  • Use of predefined and domain specific object models
  • Secure scalability and reuse for multiple tenants
  • Custom functions which can be used for changing incoming data or executing checks against incoming data from sensory devices
  • Automation of device tasks using advanced Azure compute capabilities.

Using the Azure Digital Twin service

Azure Digital Twin is at the moment of writing in preview. it is even extremely difficult to get a service installed via the Azure portal since Microsoft has limited the number of instances allowed per region. Meaning that you will need to wait till someone removes a Azure Digital Twin service from his tenant before you can add one yourself. Hopefully this will be resolved as quickly as possible with more availability or when Azure Digital Twin goes into general availability.

Microsoft has an extensive amount of documentation about how to implement Azure Digital Twins. There are concepts, references and resources available in the following documentation.

There is an interesting tutorial about monitoring a building with a Azure Digital Twin. This is the Microsoft example i spoke earlier about. The tutorial helps you to configure and deploy a already made solution into your Azure Digital Twin service. The first two steps of the tutorial are the most important one. They are roughly laid out below.

  • Deploy the Azure Digital Twin service by creating a new resource in your Azure portal.
  • Create an app registration to access the Azure Digital Twin REST API
  • Grant the right permissions to the app registration
  • Download the sample code. It consists of two projects. One is used to configure and provision a spatial intelligence graph. The second one is used to simulate devices and sensory data
  • Configure and provision a spatial intelligence graph
  • Define conditions to monitor
  • Create an user-defined function
  • Simulate device and sensory data using the second project
  • Run the simulation data
  • Run the building service to see if there is a room available based on the current sensory data

The tutorial is thorough and self explaining. Therefor i’m not going to explain each step. You will to follow the following tutorial.

There were some things which require a little bit more attention. Those things i will mention here.

You are building an application which will access the Azure Digital Twin Service through the Azure Digital Twin API. This requires to have an app registration in Azure Active Directory which is given read/write permissions to the Azure Digital Twin service. This requires to have administrative rights in your Azure portal. To get it to work i had to specify an additional platform in the app registration under [Your app registration] > Authentication > Add platform. Add the mobile and desktop applications platform. Make sure you add a redirect URI called http://localhost:8080/. In a later stadium you will be changing the appSettings.json file. That configuration file contains a AARedirectURi defined with the same URI.

Based on some organization settings you will require to have administrative consent. You will need to add delegated permissions for read/write access to Azure Digital Twin. Make sure that the Azure Digital Twin permission appears correctly in the list. If not, then use “Grant admin consent for organization” to give the admin consent.

Keep in mind that the demo uses a simulation sample which simulates sensor data and send that to the IoT Hub which is provisioned by the Azure Digital Twin service. No actual devices and sensors are used in the example.

Azure Digital Twin pricing

The Azure Digital Twin service has no upfront costs or termination fees. You only pay per node and message.

A node is a single component in the spatial intelligence graph. Below the Microsoft example showing a Digital Twin for sensory devices in rooms in a building.

Each Tenant, building, floor, room, device and sensor in this Microsoft demo is a node.

Each API call to the Azure Digital Twin API counts for a message. Each communication sent to a device and sensor are counted as message. And messages send from the Azure Digital Twin to other systems count. You can get discounts when using the service extensively. More information about pricing can be found here.

Conclusion

Azure Digital Twin is a useful tool to build Digital Twin solutions for customers. It is mainly self explanatory when using the different use-cases without going to much into technical detail. Azure Digital Twin service delivers an important part of storing the spatial intelligence graph which is a replication of your real world environment. The tutorial is a great example in how to use the Azure Digital Twin Service in combination with simulated IoT data.

The post An introduction to the Microsoft Azure Digital Twin service appeared first on 365 XR Blog.

Speaking at Microsoft 365 Virtual Marathon about Remote Assist and Guides

I’m honored to have two talks at one of the largest Microsoft 365 online events in the world called Microsoft 365 Virtual Marathon. An 36 hour event happening on May 27th till May 28th 2020. The event is a joined effort between SharePoint Conference and members of the Office 365 community. And that’s not even all. There are keynotes from key Microsoft employees and from thought leaders and members of the community like Jeff Teper, Bill Baer and Naomi Moneypenny. A truly great online event for everybody with an incredible amount of content. Over 300 speakers will deliver more than 400 sessions during this event. And it is all for free!

My sessions are focusing on solving business problems around training, guidance and remote assistance. Microsoft has two major Mixed Reality applications which are part of the Microsoft Dynamics 365 stack. In a time like this these applications are becoming more and more important for organizations.

Getting your employees ready for business using Microsoft Guides

Microsoft offers a broad range of out-of-the-box solutions for the Mixed Reality market. One of them is creating customized training modules for new workers on the factory floor. Using Dynamics 365 and Microsoft HoloLens we are able to create a specific training which allows new workers to be more quickly in learning their daily job. The session contains explanation of the different functionalities of Microsoft Guides and live demonstration of the application.

Empower your workers using Microsoft Remote Assist

Microsoft offers a broad range of solutions modernizing field services with Mixed Reality for technicians. It empowers them by offering modern tools like Mixed Reality devices, Video calls, Annotations and File Sharing capabilities. These tools allow field service workers to solve complex problems even faster, collaborate together with experts and gives them easy access to work orders. During this session we will show you a global overview of Dynamics 365 Remote Assist using Dynamics, Teams and HoloLens.

Join my online sessions

Hopefully you can join one or both of my sessions and learn more about the world of Mixed Reality, Microsoft HoloLens 2 and Microsoft’s great applications to support many industries and sectors. Sessions will be recorded and available at a later time.

The post Speaking at Microsoft 365 Virtual Marathon about Remote Assist and Guides appeared first on 365 XR Blog.

Upgrade your HoloLens development edition to a Commercial Suite edition and keep it on a specific operating system

More control over your device its operating system

There are situations when you want to have control over your device in terms of operating system. In our case we have been building applications which have functionality depending on accuracy using WorldAnchors. We noticed that WorldAnchors are stable using the RS1 HoloLens device operating system version in combination with a specific Unity version which is used building the Mixed Reality app. As soon as we moved to a newer version of the operating system, WorldAnchors became less stable hence the application did not work properly.

Automatic updating

But the problem with a HoloLens development edition is that it updates automatically to a newer version (currently Microsoft has RS5 released) of the operating system. And you can’t do anything about. Yes, you could block specific update URLs in your network to have the device not being updated by Windows Update. But that is not recommended and gives also other problems.

Production apps require Commercial Suite edition

As many of you do, the HoloLens development edition is often used when a Mixed Reality project is started with the customer. But as soon as you go into production with your app, you will be needing a Commercial Suite edition. This is a recommendation by Microsoft. A Commercial Suite edition only differs in software from a Development edition. The hardware is exactly the same. It gives you a ton of extra software options, warranty and support from Microsoft. You can read here more about the differences.

Solution

Depending on your scenario you will need some or all of the steps below to achieve your goal.

Getting your device to the right operating system

You will need to execute this step first if you want to have your devices operating system at a specific version. In our case we want to have it to stay at RS1. The only way to accomplish this is flashing your device. For that you will need the Windows Device Recovery Tool and the image of the specific operating system. You can download the tool here. You can download the HoloLens Anniversary Update recovery package here.

Run the tool on your Windows 10 computer. Startup your HoloLens and connect it via the USB cable to that computer. It will be recognized by the tool.

Select your device. The model and current version of the operating system is shown in the tool. Select “Manual package selection” and browse the RS1 recovery page on disk.

Now select “Install software”. the tool will tell you that the version you are going to install is older. Select yes to install the package.

Keep in mind that flashing the device will remove everything from the HoloLens device and does a factory reset. Meaning it is like if it comes out of the box as when you bought it.

Upgrading to a Commercial Suite edition

If you downgraded your device to RS1 make sure that you do NOT execute the configuration flow of the device. Leave the device as it is for now.

It is possible to upgrade an existing Development edition to a Commercial Suite edition. You will need to create a provisioning package which needs to be added to the device. You also need to buy a upgrade license from Microsoft. They will provide you a XML file which you will need to apply.

There is some Microsoft documentation explaining about the provisioning package process. But there is a catch. I noticed if you follow the documentation it fails to add the provisioning package. I will explain below the best steps to get the package provisioned successfully.

To create a package you will need to use the Windows Configuration Designer Tool. The installation process is described here. If you are running Windows 10 like me (which i presume since you are developing for HoloLens) you can download it from the store.

Run the tool. There are some predefined templates present. Do NOT use the one for HoloLens. That one will cause issues and does not allow to define a lot of custom specifications. Select the “Advanced provisioning” instead.

Enter a name and select a folder. When you create the package it will use the folder to store all necessary files. It needs to be a folder. Selecting for example the root or the desktop as output will not work. Click Next to continue.

To apply the new Commercial Suite license file you got from Microsoft requires to set the EditionUpgrade > UpgradeEditionWithLicense. Browse for the xml file and make sure it is added to the selected customizations at the right column of the tool.

Apply policies to prevent updating

If you also wish to add policies for preventing the update mechanism on the HoloLens device you will need to add the following customizations.

Set the policy Policies > Update > AllowAutoUpdate to “Turn off automatic updates” and the policy Policies > Update > AllowMUUpdateService to “No”. In our case since we are setting the policies for a RS1 version we will not see the policies being reflected in the settings screen later on.

Create the provisioning package

When you have finished all your changes to in the Windows Configuration Designer tool it is time to create the provisioning package. Click on the menu item Export > Provisioning package to create it.

Fill in a name and leave all other settings to their defaults. Press the Next button to continue.

Do not change any of the security settings. The package will fail if you select encryption for  the package. Press the Next button to continue.

Finally leave the location as specified and press the Next button the generate the provisioning package.

After the package is generated you will find a set of files generated by the tool. The Upgrade-Policies.ppkg is the actual provisioning package which is needed to apply to the HoloLens device.

Apply the provisioning package

The final step is applying the provisioning package to the HoloLens. There are two ways of adding the package to the device. If you are running a RS5 version you will be able to add them via the settings windows on the device. The second method allows you to add the package before the device is configured.

Important! The downside of the second method is that the device need to flashed or needs a factory reset. Both will remove the upgrade license to Commercial Suite edition. Meaning you have to include the license again in the provisioning package.

In our case we need the second method since that is the only way for a RS1 version.

You will need to do the following:

  • Make sure the device is not configured yet
  • Connect the device via the USB cable
  • Start up the device and wait for the blue square which is used to position the HoloLens correctly on your head
  • Press the power button (on the back) and the volume button down (at the right and close to the back of the head) together once
  • Wait for the device to be recognized
  • Open file explorer and browse to the root of the device
  • Copy the provisioning package file in the root of the device

  • Press again the power button and the volume button down together once
  • Now you will get a message asking “Is this package from a source you trust?”
  • Select “Yes add it”

If everything went well you will see a message telling you that the provisioning succeeded. Press the Continue button to go into the configuration of the device. Finish it till you get the start menu.

Check if upgrade is executed

You will need to go into the settings window of the device to check if the upgrade to Commercial Suite edition has succeeded. If it contains VPN Connection functionality under network you can be sure it has worked. In case of a RS1 version that will be the only way to check if it worked.

 

 

 

 

 

The post Upgrade your HoloLens development edition to a Commercial Suite edition and keep it on a specific operating system appeared first on 365 XR Blog.