Virtualization Technologies: Visualizing Data to Improve Operations

Artificial and virtual reality help manufacturers digitally transform and train new workers.

Augmented reality (AR) and virtual reality (VR) technologies are conduits of information to the plant floor. They are answers to the questions, “What do I do with all this data I’ve collected?” and “How do I attract and train new workers at a much faster rate than in the past?”

AR and VR use similar technology but are very different. Both have been popularized by gaming culture, and both are focused on changing what is visually presented to the user, but their industrial use cases are completely different.

Augmented reality, which uses devices such as smart eyeglasses, smartphones, or tablets, changes what a user sees in his or her real environment. AR is very popular with games and entertainment applications, of which perhaps the most famous example is Pokémon Go. This mobile game lets users see Pokémon creatures in their own real-world environment when looking at their phone screens. Combined with global positioning system technology, the game allows players to find Pokémon associated with a specific real location.

Augmented reality is the most popular term to refer to such technology, but it is also called “mixed reality” or “computer-mediated reality.” The devices that have helped it become popular are Google Glass and Microsoft Hololens smart glasses. Although Google Glass had a short life, it helped other companies, such as Vuzix, create their own products using the Android operating system for smart devices.

Smart glasses use AR to help operators get machine and system information that would usually require them to access another interface, such as a human-machine interface (HMI). Wearing smart glasses, operators can just look at an asset like a machine and get its entire real-time context—its current status, speed, last downtime event, current job and so on. And this is just one of the possible industrial use cases.

Virtual reality is also referred to as “alternate reality.” Unlike AR, which mixes reality with computer-assisted information presented to the user’s eyes, VR immerses the user in a completely virtual environment. Although it can be experienced on devices such as a laptop or a phone, the most popular and useful way to use VR is with virtual reality glasses or goggles. While using a VR device, the user has no vision of the real world, and his or her view range can reach only what is being shown by the VR lenses.

As with AR, games and other entertainment applications are helping to take VR mainstream. Meta (Facebook’s parent company) Oculus Quest 2 is one of the most popular VR devices. It was created to provide a complete experience for the user without requiring a connection to a computer or powerful video game console, which was a big breakthrough.

So, while it makes sense to wear AR smart glasses to take a walk through the shop floor, the same cannot be said for VR, as they take complete vision and attention. The use cases are different, such as for training new employees on facilities and equipment they have never been exposed to before, or even to allow colleagues to join in a virtual environment and collaborate by sharing computer screens and whiteboards, which is possible with an immersed app.

Why are AR and VR so important right now?

AR and VR have been in the market for quite a long time. According to Wikipedia, the U.S. military started the first AR tests in the early 1990s, while VR with immersive devices was introduced in 1979 at NASA. During the 1980s, several companies, including ATARI, started substantial investment in research and development. In the 1990s, many gaming VR headsets were available from companies like SEGA and Nintendo.

The fact is that the technology at that time was not robust or cost effective enough to allow people to really feel immersed in a world-like environment. Just like handheld devices for business and daily tasks became ubiquitous only after the first iPhone was introduced, the same can be said for AR and VR devices.

As little as five years ago, the best VR devices required an expensive computer or a video game console. Today, devices like Oculus Quest 2 are available at a viable price point and comfortably deliver lifelike simulated worlds: If a colleague calls to you from behind, you hear the voice as if the person was right at your back. The headset comes with its own processing unit and graphical processing unit (GPU), and you can use it hands free. And the battery, always an issue, now lasts for four hours.

Now that these gadgets are being taken more seriously, and not just for games, there are many possibilities for applications on both AR and VR in business, especially manufacturing. Manufacturers need to digitally transform, and AR and VR can help them in this process in a cost-effective way.

Current trends related to AR and VR bode well for its adoption by industrial companies. Devices are getting cheaper and more physically robust. In addition to Google Glass and Microsoft Hololens, dozens of new devices have been launched at various prices, notably RealWear, Vuzix, Hololens 2, and new versions from Google and Apple. In a major shift, most of the AR/VR platforms can run on the Android operating system now. Manufacturers can hand people inexpensive tablets or leverage the phones in their pockets. 

Applications abound

With so much data available on the shop floor, AR glasses help contextualize the processes for operators by providing information that layers over the assets. This information might be something as simple as an andon could show, or something that would require visiting spreadsheets, PDF project documents, and other HMIs. With a well-customized system on the back end, the smart glasses do not only present the information from these various systems, but they also present it in a processed and contextualized way.

Here is an example: By looking at an enterprise resource planning system HMI, operators can see which work order is next and then look at a PDF document to check the correct tool. With AR, they can look at the machine and immediately receive the information that the current job is done and the next work order is ready to start, with a list of the required tools. Microsoft presented a Hololens use case at Toyota in which inspection time was reduced by 20 percent. The time for manual and repetitive jobs can also be reduced by allowing the user to check for next steps right from the glasses, without having to look in paper documents.

How can AR and VR help in a smart factory?

Today, we are seeing virtual training applications where people can practice spraying coatings without causing issues in production. We are seeing underground nuclear facilities with secure data and hands-free workflows. There are fully processed 3D models that allow executive board members to walk around their facility without going into the field. Tomorrow’s applications will only be limited by the demands and our imaginations.

Companies are being forced to adopt technologies at a fever pitch. The lack of skilled resources is forcing companies to leverage technology to complete their work and attract new talent. The ability to upskill and train employees in a mock environment has never been more important, and younger employees want interesting technology to learn and grow with over the course of their careers. AR and VR are two of those technologies with extremely low costs of entry. 

This is now a reality. AR/VR is the path to putting the correct information in the hands (or should we say, eyes) of workers when they need it.

This feature originally appeared in InTech magazine's August issue, a special edition from ISA's Smart Manufacturing and IIoT Division.