Arun Navneethakrishnan, Aviation Solution Advisory Americas, Ramco Systems explains how the future of MRO/M&E IT will make for a much more efficient future

As you might expect of any laboratory, the MRO Lab in Singapore, which is co-funded by Singapore Government and powered by Air France Industries & KLM Engineering & Maintenance, has been working on incubating a lot of ideas in the time since it was created. In this article, I hope to offer readers a glimpse into some of the ideas and technology applications that have emerged from the Lab: more importantly, I want to share some ideas on how the technologies with which the Lab in Singapore and our Global R&D center in India have been working, will be able to add value, improve work processes, and raise standards and quality in MRO and M&E facilities.

But, before we delve into the world of tomorrow’s technology today, there is one thing I’d like to cover first. It’s a realization, borne of experience. We used to preach the whole mantra of business process driven improvement and technology enabled change. But what has become apparent in the last couple of years has been the fusion of new technologies (augmented reality, virtual reality, artificial intelligence, machine learning). In this, it is particularly important for technology leaders in the industry to really take a leading position on this topic and use these tools in more innovative ways; in ways that are only limited by the power of our imagination.

This article, as I’ve already said, will offer just a glimpse of what is possible. But the real value that I hope you will gain will be when, you are able to consider all the business scenarios, which you might not have previously considered or even conceived of, and which could provide a step change to your business process improvements, by applying some of these new technologies to your work environment and to your business processes.

The key topics that I want to share in this article will include…

• Virtual and augmented reality using Microsoft® HoloLens wearable technology.
• BOTS – Heading towards ZERO UI – including conversational UIs; Chatbots with artificial intelligence (AI) behind them that can talk to the admin system to provide guidance and intelligent responses to questions, when you interact. It is not even necessary to log into the system.
• Mobility: specifically focusing on some very neat things built from a line maintenance perspective.
• Paperless Maintenance: task digitization.

A new era of technological revolution is all set to disrupt the business landscape once again. The HoloLens from Microsoft promises to transform business operations by simplifying the routine complex day-to-day activities through interactive holograms. Enabling comprehensive collaboration and communication, HoloLens allows in-depth exploration of functionalities through specialized sensors and optics. An experience of augmented and mixed reality is delivered through an amalgamated digital interaction. The MRO Lab Singapore has adopted the concept of HoloLens and is working on customizing it to generate effective solutions and use cases for the Aviation and Logistics industries.

It will be easier to understand this revolutionary concept through looking at two MRO-specific use cases for HoloLens.

Use Case 1: MRO Training

HoloLens provides immense opportunities to deliver in-depth MRO training for engineers and operators. For example, 3D models of jet turbines can be converted into detailed holograms and rendered on HoloLens, resulting in an immersive training experience, simulating every scenario an engineer might encounter as part of their operational responsibilities. Every component of the jet turbine can be navigated intricately and, on selecting a specific component, the engineer can gain access to detailed videos and related visual information about the component. Similarly, it is also possible to expand extensively from this perspective and digitize an entire aircraft from corresponding 3D models. These holograms are delivered as training apps on HoloLens and they provide a precise and exhaustive view of every aspect of the aircraft from an MRO perspective; ensuring operators, mechanics, and engineers are trained in detail.

Use Case 2: MRO – Remote Support

One of the primary applications of HoloLens is in providing comprehensive remote support from the Maintenance Control Center/Technical Services to engineers and mechanics in remote locations. A simple Skype call with the control center, enabled by the HoloLens, worn by the user (mechanic/technician), can deliver much-needed assistance in real time.

HoloLens provides the complete visibility of the issue required by the control center through direct feed images and videos from the user’s glasses; with those, the control center can then use the HoloLens interface to pin specific instructions and guidance as annotations (for example, an inspection checklist) for the user to follow. These annotations are visualized by the HoloLens user as augmented details positioned on the corresponding component location, on the HoloLens view. Eventually when the user moves to another view and comes back to the same location, the annotation would still remain visible anchored to the location, and will remain there as a permanent referral point, until the application is reset.

This feature is extremely useful for inspections of aircraft components where specific checklists can be checked through verbal interactions; documents, manuals, and videos related to the specific component can be called to the HoloLens for reference; and Skype-based annotations can be used to interact in real time with experts in the control center.

With many more exciting use cases on HoloLens well on their way, HoloLens is clearly at the leading edge of a huge technological revolution that celebrates the magic of augmented/mixed reality.

The era of hands-free computing is here. From Bots to Gesture based computing powered by mixed reality, the future holds unimaginable scope for interactive communication between humans and the virtual world.

Powered by AI and Natural Language Processing (NLP), Bots act as assistants for any of the blue collar jobs in MRO/M&E and provide an unparalleled engaging and interactive experience for users to complete simple, repetitive tasks (e.g., ordering food, applying for leave, organizing meetings, providing weather updates) through conversations. It’s a zero UI (user interface) concept where users can use their mobile phone and other devices to chat and get intelligent responses to questions, with respect to maintenance and operations. The objective for bots is to move towards zero UI, not only through capabilities such as gesture computing, but also offering the possibility of being platform independent. The aviation bots that Ramco has developed, for instance, could be added to any communication apps so that the bot will be a conversational chat medium for users and will use its in-memory learning capability to learn how they are communicating with it. The bot will analyze the pattern of the user’s searches, enquiries and activities and then provide an appropriate response. They represent the next level of system use, whereby having their speed of operation and the conversational mode of interaction (the “what can I do for you” approach) make bots one of the most intriguing technological innovations of recent times.

Applying this to an ERP, users no longer find the need to wade through complex screens of menus and options – just a few minutes of pleasant, intelligent interactions with their virtual assistants can get their work done. Their ability to provide alerts and forecasts, list all possible options based on a cognitive assessment of all variables, and their buddy-type interaction make bots the ideal technology capability to increase productivity in many areas of enterprise application – especially in the complex business of Aviation Maintenance.

Where every minute counts: Bots for AOG recovery

Aviation bots have a particularly valuable contribution to make in the area of aircraft maintenance, where the potential for financial and reputational loss is very high – aircraft on ground (AOG) events. AOG events are straightforward, you can’t plan for them and they’re always going to be a feature of airline operation. As it’s not possible to predict every AOG event, the best policy is to plan for the response to them. As the AOG motto of United Airlines states ‘Five minutes makes the difference’: in short, a response is needed within minutes. An airline’s AOG desk role is significant in the recovery. AOG, which is primarily a maintenance event, also triggers a cascade of supply chain events such as sourcing, logistics, payments and even reverse logistics of unserviceable items.

The AOG desk faces a continuous streaming of information from different entities involved in the AOG recovery (Ref Fig 1). Not all are easily available and several minutes are spent with multiple systems, e-mails, phone calls for getting information on availability of airline stock at AOG, or a nearby location, partner stock or pool stock availability at AOG location, sourcing through locator services, OEM stock availability, identifying the right shipping carrier, tracking ASN from supplier, tracking AWB till POD and so on.

The questions that arise are: can technology aid in building a connected Eco system? Can inter-related systems be integrated for processing the relevant information and in turn suggest recommendations to the AOG desk? Today, intelligent MRO systems are expected to provide Maintenance Control with instant recommendations such as:

• The possibility of swapping from any other grounded aircraft or aircraft currently under base maintenance.
• The possibility of expediting the Inspection and Certification, if the Item is WIP at Receiving Bay or Shop etc.

These recommendations should consider the AOG Tail # and Part # effectivity to avoid the wrong part ending up in the hands of the mechanic.

To reduce investment on spares parts but also achieve the availability of spares parts, airlines are entering into formal agreements with partner airlines, AOG parts support based on PBH, bilateral pool agreements or IATP agreements. An AOG event trigger should instantly process the demand and recommend the best option to meet the demand based on the preset agreements. This is where bots can help with AOG events. With real-time access to partners’ stock availability in the AOG location, a request for a part can be automatically triggered through text or email messaging systems. If the parts support is provided by pool agreements, order notification can be triggered well in advance through any of the communication channels (depicted in figure 2), followed by paper work.

In locations where the AOG part support is not available or expected lead time is high, one option is to check with a part locator service. An integrated eco system will aid the AOG desk to send the RFQ through chat windows within their MRO system and get an instant response on availability, lead time, price, certification and payment terms. A decision to buy can be advised through a messaging chat to auto create a purchase order and transmit through B2B EDI, if the supplier is enabled, or auto transmit the purchase order copy through mail.

In circumstances when the airline needs to expedite an order or place a new order with a supplier to meet the AOG requirements, the onus on arranging the best possible shipment from the OEM to the AOG location lies with the AOG desk. The MRO system is expected to connect, retrieve and process the real time data stream from various sources including its own airline flight schedule systems, other airlines flight schedule systems, shipping carrier systems, and provide the best option to ship considering the HAZMAT /Oversize nature of the Part #. The system will also expect to learn from the AOG desk if the suggested option is selected or rejected.

Through an interface with the OEM system, an advance shipping note can be received along with the AWB # of the shipping carrier. Large shipping carriers provide full visibility of the AWB # tracking status. Continuous streaming of AWB status is expected to get processed and any deviation from the original delivery time, which may affect the AOG recovery, shall be escalated to the AOG desk through a messaging system.

While relationships between the AOG desk and the rest of the system still matter, the most effective AOG desks leverage technology to provide them with decision assisting capability, for driving down their AOG recovery time. It’s no more the case that information is available to AOG desk and waiting for a response; in the future, MRO systems will be expected to take a decision on behalf of the AOG desk and keep learning to refine their decisions.

As well as the undoubted benefits of technology such as HoloLens, the MRO Lab is also working on areas such as Gesture computing, so that, if a mechanic has their hands dirty or, for other reasons, cannot use a keyboard or touchscreen, they can simply employ gesture control where the system will perform all the intended actions.

The next level of mobile technology and capability will see everything brought into the mobile environment. For instance, PDFs whether a document or a scanned image, can be introduced and converted into process-able documents for task use and sign-off requirements. The question we have been asked in this is how to take a scanned document and turn it into a process-able form. It involves taking any scanned document and having the sign-off block set on top of the document, while the system automatically reads who has to sign-off on the particular task card. So, the moment the task card comes in, the mechanic can instantly sign-off using the electronic sign-off capability. The system is totally dual authentication enabled; the moment a mechanic signs-off, he’ll be able to register his sign-off and follow the task card to the respective team with all the information available on-line right away: the team will be able to look at the ‘dirty fingerprints’ in real-time.

As we see the mobile application today, the objective is for a user to be completely unchained from their desktop, not just mirroring a capability into the mobile app and then having to return to a desktop or laptop to perform the required activities. It needs to be completely unchained; if a mechanic is going to use the app they should be completely reliant on it. As part of this, Ramco has introduced voice recognition whereby the mechanic can report a discrepancy by simply talking to the mobile app – a capability aimed specifically at cabin defects, where the mechanic can walk through the aircraft, keep talking to the application and the system will report the discrepancies.

These are just a few of the key technology innovations that we believe will transform the way MRO/M&E personnel work in the future.

Arun has worked over 10 years as project manager, senior business analyst and solution advisory consultant in the area of maintenance, supply chain, production planning, flight operations and financials for various industry branches. He has been associated with Ramco for over seven years and has been involved in several phases of product development. Arun heads the solution advisory group and supports sale teams in the Americas for several new accounts and existing projects. He has a Masters in Aeronautical and Airport management

Designed to be accessible on cloud and mobile, Ramco Aviation Software continues to add technological innovations with ‘Anywhere Apps’, redefining the power of Mobility, to significantly reduce transaction time both during AOG conditions and critical aircraft turnarounds. The software helps aviation companies ensure zero tolerance to error, with higher safety standards. Ramco is changing the paradigm of enterprise software with ZERO UI powered by cool new features such as Chatbots, Mail bots, HUBs and Cognitive solutions. Furthering this, the company has also developed application compatibility on Microsoft HoloLens to bring Augmented Reality which improves efficiency in maintenance operations. With 75+ Aviation leaders onboard, Ramco is the solution of choice for several large airlines, and top heli-operators and multiple MROs in the world.