Senior commanders in the Royal Air Force (RAF) have been shown by experts at the AMRC how the same Industry 4.0 digital technologies that have drastically changed how NASA astronauts perform duties in orbit could transform how the RAF operates and trains its workforce.

University of Sheffield Advanced Manufacturing Research Centre (AMRC) engineers worked with officers at RAF Leeming in North Yorkshire to demonstrate how augmented reality (AR) headsets could revolutionise day-to-day maintenance, repair and overhaul (MRO) by streamlining inspections, providing access to remote expert assistance and rapidly upskilling the workforce.

“This could completely change the way we work,” said Squadron Leader Marcus Ramsden, Engineer Officer (AeroSystems) at RAF Leeming. “Because of its use in gaming, this is technology that new RAF recruits are completely familiar with; we need to grasp it and embody it in our operations.”

Since 2015, Microsoft HoloLens’ have been employed by astronauts on the International Space Station when they need hands-free, remote assistance from NASA scientists on Earth, who are able to see precisely what the astronaut is seeing and advise accordingly.

After identifying mixed reality, specifically the Microsoft HoloLens 2 AR headset, as having the potential to streamline inspections within the RAF, by replacing traditional pen-and-paper checklist work instructions with simple, step-by step holographic instructions displayed in an engineer’s field of view, a team from RAF Leeming visited the AMRC’s Factory 2050 in Sheffield.

Sqn Ldr Ramsden said: “At Factory 2050 we saw mixed reality devices being used to do MRO on a kit car and it set my mind into overload – if it could work on a car, why couldn’t it work on an aircraft?

“The University of Sheffield has a Hawk T Mk 1 on its campus, the AMRC has the expertise, and a relationship with Microsoft was already there. It was all perfectly aligned so we commissioned the AMRC to turn one of our paper-based systems into a mixed reality solution, to see what it could do.”

To demonstrate the technology, the AMRC chose to create a mixed reality solution for a wheel change and brake pack assembly on the Hawk T Mk 1, a two-seat training aircraft familiar to many as the aircraft of choice for the RAF’s aerobatic team, the Red Arrows. Developing the mixed reality solution involved learning the current manual method for the wheel change and brake pack assembly, reverse engineering the paper-based instructions and then measuring the individual components to create computer-aided design (CAD) assets.  Work instructions were created from those assets and generated in AR on the HoloLens 2 headset using the Microsoft Dynamics 365 Guides application.

Mike Lewis, Digital Theme Lead at the University of Sheffield AMRC, said: “The major element of the solution we demonstrated is the overlaying of the 3D CAD assets onto the aircraft to provide holographic instructions to the operator. On the headset, an engineer can see components superimposed in 3D on the workpiece and, using spatial perception on the HoloLens 2, be told exactly where they need to be working.

“An AR headset can store far more information than a paper instruction document and it reduces the margin for error considerably because there is a 3D representation of the instruction.”

Sqn Ldr Ramsden says the AMRC, part of the High Value Manufacturing (HVM) Catapult, has demonstrated how the technology could be integral in three core areas of the RAF’s operations: remote assist, understanding data and training.

“Right now, when an aircraft is moved to the front line, a team of engineers with individual expertise and their many 400-page manuals move with it. Those paper-based systems not only use up valuable space and weight, they are susceptible to wear-and-tear and can be very cumbersome in the environments we work,” said Sqn Ldr Ramsden.

“Added to that, most of our processes were written down in the 1970s, which means they have been updated inconsistently, can be tough to understand and are hard to translate from the page into the real world.

“Mixed reality would completely change the way we work; one engineer can have the entire up-to-date dataset right in front of them and can be supervised remotely by subject matter experts potentially located on the other side of the world.

“Then, when we consider training, not only can our engineers learn in a safe environment where maintenance notes are flagged up in front of their eyes, they can be guided through simulations of different tasks before physically performing that process – all entirely hands-free. This technology has the potential to streamline our processes, boost our efficiency and improve our safety.”

RAF Leeming is home to RAF eXperimental (RAFX), an innovation hub which experiments ‘at the edge’, allowing for concepts to fail early, fail safely and learn rapidly to embrace capabilities that will shape the future air force.

Sqn Ldr Ramsden says RAFX, a concept brought over from the USA, is a space to generate ideas but also a sandbox for concepts that can be presented to senior commanders as part of the Astra programme.

Astra, inspired by the forces’ motto Per ardua ad astra (Through adversity to the stars), is the campaign plan for building the next generation air force with the core themes of people, support and equipment, training, and infrastructure. Cutting across each of the themes are emphasises on replacing outdated systems, digitalisation and exploiting cutting-edge technologies.

“We pitched the integration of mixed reality, demonstrated by the AMRC, to senior commanders at an Astra Technology Exposure Day,” said Sqn Ldr Ramsden. “There was positive feedback from the Chief of the Air Staff, Air Chief Marshal Sir Mike Wigston, and they seem keen to take this technology forward.”

Mike says using AR for MRO could be just the start. 

“The example we have chosen is a really quick way to get value from these kinds of devices but once an organisation starts to adopt this kind of technology in a wider sense, you can start to harness so much more power,” he said.

“Applying AR to work instructions opens the door to wholesale integration and every data-producing system communicating with each other, which any operator can access. The scope for its application is massive.”

Professor Rab Scott, Head of Digital at the University of Sheffield AMRC, said: “Augmented reality is going to shape the way people work in the 21st Century. It is going to allow us to perform tasks faster, more accurately and with a much richer basis of knowledge from which to work; as importantly, it also has a powerful role to play in the training of staff and the upskilling of workforces.

“This project is at the heart of what the AMRC has been doing for the last 20 years - helping organisations adopt technologies more quickly through proof of concept and applying horizontal innovation by transferring best practice in one sector to another.

“Industrial digitalisation is happening now, it is happening at the AMRC and it’s great to see it being embraced by Sqn Ldr Ramsden and his team at RAF Leeming.”