Nova Intelligent Copilot (NIC)

The problem

Radar maintenance teams at the Federal Aviation Administration (FAA) work with large, fragmented technical manuals — covering fault isolation, troubleshooting procedures, and equipment maintenance for different radar systems. Finding the right procedure within these manuals can be slow and cognitively demanding, particularly during time-sensitive troubleshooting when the correct sequence of steps matters.

Existing keyword search tools frequently returned broad or irrelevant results, increasing the risk of delay or procedural confusion. A search might surface content from the wrong radar system entirely — for example, returning maintenance guidance for one radar family when the technician was working on another. Frequently receiving broad or noisy results increased the risk of delay or procedural confusion.

The solution

Nova Intelligent Copilot (NIC) is an independently built, in-house experimental decision support tool, an AI-powered search and guidance tool that supports radar maintenance technicians in finding relevant procedures quickly and accurately across multiple technical manuals. It is a technical reference tool that improves source-grounded retrieval, citation traceability, and domain filtering, and it does not replace official manuals, required procedures, supervisory review, or qualified technician judgment. Always, the user needs to verify the sources first before making a decision.

How it works
Rather than relying on simple keyword search, NIC uses a hybrid Retrieval-Augmented Generation (RAG) approach — meaning it searches a library of technical documentation and then uses a Large Language Model (LLM) to synthesise a clear, citation-backed answer from the relevant passages. Every answer is linked to its source in the original manual, so technicians can verify the guidance against the official documentation.

The system is built with several layers designed to ensure accuracy in a technical environment. First, the documentation is broken into smaller, structured sections — a process known as chunking — so that searches can return precise passages rather than entire chapters. Second, the system uses domain classification to identify which radar family a question relates to, and applies compatibility filtering to prevent results from one radar system appearing in answers about another. For example, content from the ASR-8 radar system will not appear in answers about NEXRAD, a weather radar network, even if the language in the manuals is similar.

When the system cannot generate a confident answer, it does not attempt to fill the gap. Instead, it falls back to an extractive mode — returning the most relevant source passages with direct links, rather than producing an unsupported response. This means technicians always receive traceable, source-linked information, even when the AI-generated summary is not available.

NIC was built in-house as a research prototype, using existing and open-source tooling, with no additional procurement cost. The developer maintained a large automated test suite of 1,772 tests to ensure that improvements to retrieval and safety features could be shipped quickly without breaking existing functionality.

The impact

1. Faster, more accurate retrieval
In simulated research testing, NIC shifted troubleshooting from searching through manuals to guided, citation-backed answers, with each response pointing directly to the relevant section in the source documentation. During evaluation of the proof of concept, technicians were able to find the guidance they needed faster and with fewer irrelevant results, especially after domain hardening updates, changes made to tighten the system's ability to recognise which radar family a question relates to and filter results accordingly.

2. Cross-domain noise reduced
Cross-domain retrieval noise was reduced — for example, maintenance guidance for the ASR-8 system had been appearing in answers about NEXRAD. After tuning the classifier — the part of the system that identifies which radar family a question relates to — and introducing incompatibility rules that prevent content from one radar family appearing in answers about another, domain confidence on representative NEXRAD queries improved from approximately 0.60 to 0.97 out of 1.0. In testing, this meant the system returned information from the correct radar family in almost all of these cases.

3. When the system isn't confident, it says so
The system now also degrades more safely: when generation fails, users still receive extractive, source-linked results — the most relevant passages from the original documentation with direct links — instead of unsupported answers.

4. Built-in quality checks allowed fast, safe updates
From an engineering quality perspective, the developer maintained a set of 1,772 automated tests — checks that run every time a change is made to the system to verify that everything still works as expected. This meant that improvements to search accuracy and safety features could be made quickly and with confidence, without the risk of accidentally breaking core workflows or something that was already working.

In testing, NIC improved response quality, traceability, and operator trust, while highlighting clear areas for continued precision tuning.

Lessons learned

• Enforce domain boundaries from the start. Early versions of NIC suffered from cross-domain noise — content from one radar system appearing in answers about another. In retrospect, the team would enforce strict domain constraints at every stage of retrieval from day one, rather than adding them incrementally.
• Calibrate relevance signals early. Initial relevance scores were inflated, meaning the system appeared more confident in its results than was warranted. Calibrating these signals earlier in development would have made it easier to identify where the system was underperforming and prioritise improvements.
• Make fallback behaviour visible to the user. When the system fell back from AI-generated answers to extractive retrieval, this was not always transparent to the user. Building visibility into the interface — so that technicians can see when they are receiving a direct source extract rather than a synthesised answer — is important for maintaining trust, particularly in safety-critical environments.
• Citation-grounded retrieval builds trust. Linking every answer to its source in the original documentation was one of the most effective features for building operator confidence. In a maintenance environment where following the wrong procedure can have serious consequences, traceability is not optional.

We operated within a United States (US) public-sector aviation operations context, supporting radar maintenance knowledge workflows tied to FAA and National Weather Service (NWS) technical documentation, including NEXRAD maintenance and equipment handbooks.

While NIC is a technical reference and documentation retrieval research prototype — not an autonomous control system — we treated it as a high-assurance environment. This meant source-grounded outputs, citation traceability, explicit fallback behaviour, and strong human-in-the-loop verification — where a qualified technician reviews and approves any action before it is taken — for all operational decisions.

We also designed the tool to adhere to standard operational safety practices already used in this domain, such as lockout/tagout procedures for maintenance work. Lockout/tagout is a safety procedure that ensures equipment is properly shut down and cannot be restarted during maintenance. AI outputs are intended to assist technicians in navigating documentation, not to replace official procedures or supervisory approval.