HIT team wins Innovation award for Mixed Reality training project
Using Mixed Reality for training purposes
The prestigious CIEHF establishment awards recognise teams and individuals who demonstrate exceptional commitment to and application of human-centred design practices in research and development projects. It is therefore a great achievement for Prof. Stone1, a key advisor to Blue Abyss, and his HIT team2 to receive this internationally recognised innovation award for their involvement in the MERT project.
MERT trainer team, from left to right: Prof. Bob Stone, Dr. Robert Guest, Dr. Vish Shingari.
The project is sponsored by the UK’s Defence Medical Service via the Royal Centre for Defence Medicine and aims to enhance pre-deployment training for medics due to be deployed into war zones to provide immediate medical attention to casualties whilst enroute to a more comprehensively-equipped surgical unit. With the use of virtual reality (VR) and blue screen chroma key, this Mixed Reality trainer allows trainees to see and interact with real physical objects, such as medical equipment and casualty mannequins, while experiencing the environment within a military platform, such as an RAF Chinook Helicopter or Royal Marines Landing Craft. This project is set to enter a third phase and move towards commercialisation, where the Mixed Reality solution could be exploited in other training applications. This is particularly relevant to space-related training and holds great potential in contributing to Blue Abyss’s goal of offering the most advanced and realistic space training programmes possible. Prof. Stone says,
We could, in future, extend the Mixed Reality research we were undertaking for the MERT projects to possible space (and subsea) training, developing Mixed Reality reconstructions of spacecraft interiors, or planetary modules, for example.
The MERT project in action. The title article image shows the execution of the training programme by a volunteer surrounded by blue screen chroma key enclosure and wearing a camera-equipped VR headset while ‘’operating’’ on a ‘’casualty’’. The image above demonstrates the VR environment showing the trainee’s view of the real mannequin as if within the rear cabin of a Chinook Helicopter. The camera-equipped headset enables trainees to see their real arms and hand whilst “treating” the “casualty”. The vital lifesigns monitor is virtual and controlled by the training instructor outside the enclosure.
The HIT team
Prof. Stone and the HIT team belong to the University of Birmingham within the College of Engineering & Physical Sciences. Here, the team has been developing interactive media and robotic technologies in the UK since 2003, building on three decades worth of experience in the domains of VR, augmented (AR), and Mixed Reality (MR), Simulation, and Telerobotics/Telepresence. With their pioneering research, the team specifically focuses on exploiting Human Factors knowledge, such as realistic visuals, sounds, haptics and smells, to develop applications that take into account the needs of the end-users in order to provide an immersive and authentic experience.
Knowledge about Human Factors techniques and processes underpin every human interface project we’ve ever been involved with.
Involvement in the space domain
The involvement of Prof. Stone and his HIT team in the space sector has a long history. Between 1980 and 2003, the European Space Agency (ESA) – ESTEC and British Aerospace sponsored human and automated spaceflight research in which members of the team were actively involved. The focus of this research involved experimentation with a teleoperation and control test bed based on a modified industrial robot to observe issues such as transmission time delays and impacts on remote operator control ability. Additional projects addressed Human Factors issues related to human-involved rendezvous and docking, as well as helmet-mounted display concepts for extra-vehicular activities. These engagements set the scene for the HIT team to be involved in other space projects, such as designing the human-system interface for NASA’s Orbital Manoeuvring Vehicle, and supporting the Russian Gagarin Cosmonaut Centre in developing the first ever VR model of the Mir Space Station. This led to the introduction of VR into the Cosmonaut Training Programme, for which Prof. Stone received prestigious recognition by the 2000 Chief of the Centre, General Pyotr Ilyich Klimuk at the 4th International Scientific-Practical Conference on Manned Space Missions, and subsequently was made an Honorary Cossack as well as an Honorary Professor at the South Russia State Technical University.
Since 2003, Prof. Stone and the HIT team still place a heavy focus on fundamental human factors, telerobotics and simulation research in the space domain. The team’s main line of research encompasses issues related to space exploration, particularly remote driving and manipulation simulators and using VR, AR and MR test beds for command, control, and remote situational awareness. The most recent output from the team, in collaboration with researchers led by former astronaut Jay Buckey from Dartmouth University, has been the use of VR to display nature scenes for the improvement of human health and well-being. This was used by space physiology researchers in Arctic trials to explore how such virtual scenes may benefit humans in space during long-duration spaceflights and planetary exploration3.
Collaboration with Blue Abyss
Prof. Stone and his team are well-placed to form an influential contribution to Blue Abyss’s mission of providing state-of-the-art facilities for deep-sea and space extreme environment research, training, and test centre capabilities. With the Mixed Reality efforts entering the next phase of the project pipeline towards commercialisation, this work will provide unprecedented opportunities to apply this technology to training programmes. This could pave the way for space exploration with a well-prepared human taskforce that would be trained to face the technical challenges of space exploration, while being supported in their mental well-being when faced with long-duration missions. This joint effort is already underway. For example, in 2018, the HIT team put together a VR demo called ‘’Mars Survival’’4 for Blue Abyss, showing the potential for this sort of project to extend the Mixed Reality research for MERT to space and subsea training, such as developing reconstructions of spacecraft interiors or planetary modules.
A more comprehensive summary of the MERT project was recently published during an international VR conference. Please see p88-90 here.
A recent in-depth interview with Prof. Stone was conducted by Ed Kramer – a highly experienced and recognised digital artist who has been involved in a multitude of science fiction films of the past. Prof. Stone’s interview can be seen here.
Share this story