GOV.UK Chat

This information was sourced from "GOV.UK Chat" by AI.GOV.UK, https://ai.gov.uk/knowledge-hub/tools/gov.uk-chat/?directTo=library. Licensed under Open Government Licence v3.0.

The challenge

GOV.UK hosts over 700,000 pages, with new content added regularly. The sheer volume renders traditional search and navigation methods less effective. This makes it hard for users to quickly find the information they need, especially in cases involving more complex queries.

The solution

GOV.UK Chat uses a Retrieval-Augmented Generation (RAG) approach, allowing users to ask natural language queries and receive relevant information. Safeguards have been put in place to protect user privacy. Personal data is removed from the GOV.UK pages accessed by the LLM. The tool is developed in compliance with UK data protection legislation, and a thorough Data Protection Impact Assessment (DPIA) was conducted. Additionally, red teaming exercises were conducted to identify and mitigate potential vulnerabilities.

The results

Tested by hundreds of users in a controlled environment, the tool demonstrated strong potential to improve access to government information. These experiments have helped the GOV.UK Chat team understand the types of user queries and the most common failure modes. It has enabled the team to assess the feasibility of integrating such a system more broadly.

Learnings / insights

• Given the frequent updates to GOV.UK content, Retrieval-Augmented Generation (RAG) provides better accuracy and relevance in responses compared to fine-tuning a large language model (LLM).
• Automated quality assurance should be a key consideration to consistently maintain high response quality when scaling the solution.