FFG NGS “Next Generation Safety”
Successful project finalisation
This project has received funding from the Austrian research funding programme Take Off. Take Off is a research, technology and innovation funding programme of the Republic of Austria, Ministry for Climate Action. The Austrian Research Promotion Agency (FFG) has been authorised for the programme management.
The FFG NGS “Next Generation Safety” Project aimed to provide AI-based automation in ATM, releasing controllers from simple repetitive tasks to focus on decisions requiring higher levels of concentration.
The project researched and developed new AI algorithms and methods in the areas of:
- Speech-2-Text for transcribing radio communication
- Context-sensitive dialogue automation of air traffic controllers
- Text-2-Speech to provide a clear air traffic controller voice speaking over radio
- Decision automation system for ATM process automation
- Simulation environment for validating the communication process automation.
Together with our partner Ondewo, provider of the ONDEWO Call Center AI (CCAI) Platform voice and text automation software with machine learning and deep learning algorithms for natural language understanding, human speech to text transcription and text to human speech synthesis, we established a common ATM simulation environment, built on the Frequentis MosaiX platform and Frequentis X10 VCS, to evaluate the automation of two-way communication between pilots and controllers.
A key element was the transcription and interpretation of the radio messages as a baseline for further automation. This included the optimisation of Speech-2-text (S2T) and natural language understanding (NLU).
The project included technical and operational evaluation, with more than 100 test runs and the collection of feedback from 12 ATCOs.
The project was supported by an advisory board consisting of Austro Control, Skyguide and Austrian Airlines to add an operational viewpoint.
The key insights from this evaluation are as follows:
- Fast recognition of callsigns (average time to highlight ~0.6 sec)
- Callsign recognition rates are between 90% and 100%, depending on the similarity of callsigns within the area (e.g., AUA55, AUA56)
- The Speech-2-Text transcription processing was very fast with an average time of ~1.6 sec to process an average of 5-8 seconds of input audio. The radio transcription has been optimised down to a WER 13-14% and CER 8-10% (based on 20 hours of training material) compared to Google WER ~16% (2021).
- The format of numbers and text should be digits and capital letters for waypoints, and lower-case letters for better readability of other text – for example: “A U A 5 5 request taxi way A 3”.
- Pre-processing of radio transmissions with a noise-reduction algorithm provides no significant improvement in the recognition and error rates.
- To enable fully automated pilot and ATCO conversations, intent recognition is needed. The intent recognition for the project’s use case was brought to a value of 76%, meaning that we were able to automate 76% of the conversations.
- The use of automatic ATCO answers should always include human confirmation and could be ideally suited to repetitive tasks (e.g., startup clearance).
Value Proposition:
- Callsign highlighting on the radar screen
- Saving time through automatic clearance deliveries (e.g., CFL, route clearance) - and automatic sending of messages (e.g., startup clearance)
- Intent and value recognition of ATCO for pre-filling of information
- Transcription of radio transmissions and ATCO communication for further automation.
We would like to thank Ondewo, Advisory board members and FFG for their valuable cooperation in bringing AI-based automation to ATM.