News and notes from Imagine! and the Imagine! Foundation.
Learning Job Skills through Virtual Reality
May-09-2022
Services
Imagine! Employment Services work one on one with clients who request assistance with landing a meaningful job. Imagine! job coaches provide counseling, skill assessments, interview practice, resume building, and on the job coaching after they secured employment. Even after implementing these structures for support, it can be difficult for an individual with a developmental disability to enter a new environment and start learning new skills. Seeking solutions to bridge this gap, Imagine! partnered with Reality Garage in Boulder and created a Virtual Reality job simulation that allows clients to gradually learn specific job skills before doing so in a busy and chaotic environment.
“I don’t think the general public understands some of the obstacles our clients have to cope with,” said Tyler Little, Imagine! Assistant Technology Supervisor. “There is severe anxiety and stimulation overload involved, which can turn anything into a negative experience. We want to gradually introduce them to a working environment so they can get comfortable with this new venture in their life.” Along with Tyler, Imagine! AT Services is managed by Assistive Technology Supervisor Meagan Wittle.
Thanks to a grant that Imagine! received over a year ago specifically for this project, Imagine! AT Services drummed up the idea to build an entirely virtual world that helps simulate job responsibilities and environments. To start, they focused on restaurant work and partnered with The Sink in Boulder to create this world.
Bob and Brenda are the owners of Reality Garage and jumped right in when presented with this project. They went to The Sink before operating hours, used 360 cameras to capture the interior, then plugged that media into their VR software. Next, they used LiDAR (Light Detection and Ranging) to create the virtual world, overlaying the LiDAR footage with the VR simulation.
“Bob and Brenda have been incredible partners and super accommodating,” said Tyler. For example, when using a VR headset, users typically hold a controller in each hand for making selections. Understanding that using motor skills isn’t always an option for our clients, Bob and Brenda programmed an “eye-gaze” feature, allowing clients to make a selection simply by looking at it.
How does it work? After putting on the headset, you will see a recording of Tyler welcoming the user and reciting a small intro. Users then dive in and are given prompts to clear a table and then take the bin full of dishes back to the kitchen. Users go through the literal motions of these tasks and the simulation detects their movement. As a spectator, you see the user wearing a headset, walking back and forth, and opening and closing their fists. As a user, you are fully immersed in an animated world that represents The Sink and prompts you to complete activities related to practicing job skills.
A complimentary feature to the VR simulation is the tracking of progress per each client’s performance during each simulation. This gives the AT team solid data that informs them of possible next steps to encourage the client’s growth within these training courses. If Meagan and Tyler see that a client is excelling and mastering the job tasks in the VR simulation, they can add artificial audio in the background to simulate potential distractions.
After the VR activity, students practice clearing a real table and unloading dishes into real bins, focusing on the end goal of mastering these tasks in real life situations. Premeditated distractions can be added here as well, such as having someone sit at the table and talk up a storm.
This technology is a strong step forward in supporting individuals who have intellectual or developmental disabilities with assimilating into the workforce successfully. The Imagine! AT team and Employment Services, along with Reality Garage, have aspirations of building more work environments that will focus on different skillsets and occupations. This technology is also just the tip of the iceberg. With Augmented Reality slowly being infused into popular tech devices, such as Google Glasses and the Pokémon Go app, there is potential for this tech to be channeled into applications that could assist clients while on the job.
This is the start of what we believe to be a long and fruitful journey, bridging the gap between chaos and understanding, finding healthy solutions for individuals with disabilities to excel in the workplace and find meaning through their careers.
If you would like to learn more about this technology or how to get involved with Imagine! Employment Services, please contact ATservices@imaginecolorado.org.
