Search across all content
projectA prototype combining Optical character recognition (OCR) and prompt-based language models cut early-stage complaint screening time by 78.6 per cent at Thailand's anti-corruption agency, while keeping human officers in charge of final decisions.
Behind every complaint received by Thailand's National Anti-Corruption Commission (NACC) is a citizen who believes something has gone wrong and has turned to the state to seek accountability. Before any investigation can begin, however, that allegation must first pass through an administrative gateway: it must be read, interpreted, digitised and categorised. In practice, this early-stage screening process relies heavily on manual review. Officials examine written narratives, often lengthy, sometimes handwritten or scanned, and determine how each case should be classified and routed.
The work is essential but time-consuming.
During controlled testing, trained personnel required an average of 333.39 seconds (approximately 5.6 minutes) per complaint to complete the initial screening tasks.
Five minutes may not sound significant. But intake rarely involves a single case. When multiplied over hundreds or thousands of incoming complaints, those minutes add up quickly. What feels manageable at the level of a single allegation becomes a sustained administrative burden at scale, slowing throughput, increasing workload, and placing ongoing pressure on staff responsible for triage.
Over time, even modest per-case delays at the front end of the system can constrain the organisation's ability to route allegations promptly for further investigation.
It was this front-end bottleneck that prompted Issara Sereewatthanawut and colleagues to conduct a closer examination, publishing a peer-reviewed study exploring whether Artificial Intelligence (AI) could support the complaint intake process at Thailand's NACC. In collaboration with the NACC, the researchers built and tested a prototype AI system to improve the speed and consistency of early-stage allegation screening.
Rather than automating the entire investigative process, the prototype focused specifically on the administrative gateway: turning incoming documents into structured, readable and classifiable cases. The prototype combined three functions into a single workflow:
Each stage was designed to reduce manual handling time while preserving human oversight.
Step 1: Optical Character Recognition (Document Digitisation)
The first challenge was practical. Many complaints arrived as scanned files or handwritten submissions. Before any analysis could take place, these documents needed to be converted into machine-readable text.
To do this, the prototype used Optical Character Recognition (OCR), software that converts images of text into editable digital text.
The researchers tested the OCR module on 100 documents chosen to mirror the variety of submissions the NACC actually receives.
This means that the dataset deliberately included:
This design choice matters. Real public sector intake rarely involves clean, standardised forms. By including messy and inconsistent documents, the testing better reflected the reality of documents that the AI system would have to read and interpret in practice.
Performance results
Overall OCR performance:
In plain terms, this means the prototype correctly captured most relevant text, though not perfectly.
For printed Thai text specifically:
Performance was lower for handwritten submissions, highlighting an important limitation. While Artificial Intelligence can significantly reduce digitisation time, handwritten or degraded documents remain technically challenging. For public bodies relying heavily on informal submissions, this remains an operational constraint. Therefore, even if the prototype could meaningfully reduce time spent manually transcribing or retyping complaints, human review would still be required, particularly for handwritten cases.
Step 2: Natural Language Processing and Structured Summarisation
Once text was digitised, it needed to be made usable.
The prototype applied Natural Language Processing (NLP), a branch of Artificial Intelligence that enables computers to analyse written language, to organise and structure complaint narratives.
The prototype generated structured outputs designed to standardise presentation, sparing staff the task of reading long, unformatted paragraphs. This reduced reading burden and created a more consistent intake format before classification.
For intake officers, this stage functioned as a preparatory layer. It did not make decisions, but it reduced variability in how information was presented. This means that time spent scanning unstructured narratives could be redirected toward judgement and prioritisation rather than document parsing.
Step 3: Artificial Intelligence-Based Classification Using Large Language Models
The final stage of the prototype addressed a crucial question: once a complaint has been digitised and structured, how does it get routed to the right place?
At the NACC, every complaint must be assigned to one of a set number of misconduct types. These categories determine which department handles the case and what legal framework applies. Getting this classification right is not a minor administrative step. It shapes how quickly and effectively the allegation moves forward.
To support this task, the prototype used two Large Language Models (LLMs): Gemini-1.5-Flash and Gemma-27B.
A Large Language Model is a type of Artificial Intelligence trained on vast amounts of text. It works by learning patterns in language, allowing it to analyse, summarise, or categorise written content.
However, the researchers did not retrain these models using sensitive internal complaint data. Instead, they used a technique called prompt-based inference. This means the model was given carefully written instructions telling it exactly what task to perform. For example, the prompt might instruct the model to:
These prompts were deliberately structured to reduce ambiguity and prevent open-ended responses. The prototype was not asked to generate creative text. It was asked to make a constrained, category-based decision.
1. Major efficiency gains at the front door
The most immediate result was a significant reduction in time spent on initial screening. Under manual review, the average time to complete early-stage complaint screening was 333.39 seconds (approximately 5.6 minutes) per case. With the AI prototype, this fell to 71.47 seconds, a 78.6 per cent reduction.
In practical terms, this meant intake officers could move through routine administrative steps far more quickly. Tasks that previously required careful retyping, formatting and preliminary categorisation were partially handled by the prototype, allowing staff to focus sooner on assessing substance rather than structure.
This does not eliminate the complexity of corruption investigations. But it does change the tempo of the first interaction with each case. The administrative lift at the front end becomes lighter, and the flow of cases into the wider system becomes steadier. For teams working under sustained caseload pressure, that shift in pace can be meaningful, not because the work disappears, but because it becomes more manageable.
2. Faster summaries without losing legal substance
Speed alone would not be sufficient if it compromised quality. For that reason, the Natural Language Processing summarisation module was separately evaluated.
After a complaint was digitised, the prototype produced structured summaries to help intake officers quickly understand key elements. These summaries were reviewed by a panel of legal experts and NACC officials.
They were assessed on:
The summaries received an average score of 4.74 out of 5.
Crucially, reviewers found the summaries reliably surfaced the legally important facts in each complaint: the person accused, the conduct alleged, and the laws or regulations said to apply. This matters because intake officers are not looking for shorter text; they are looking for reliable understanding. The evaluation found that the summaries substantially reduced preparation time while remaining easy for officers to interpret.
The result is not automation of legal reasoning, but acceleration of information preparation.
3. Classification: promising, but not autonomous
The final performance test assessed how reliably the prototype matched each complaint to the correct misconduct type on the predefined list.
The strongest-performing model achieved:
What does that mean in practical terms?
An accuracy rate of 57.5 per cent means that in roughly 58 out of 100 cases, the prototype selected the correct category.
The macro-F1 score is a more balanced measure. It ensures that the prototype performs reasonably across all categories, including less common types of misconduct. This is important because corruption complaints are not evenly distributed. Some categories appear far more frequently than others. A prototype could appear accurate simply by repeatedly predicting the most common category. Macro-F1 prevents that distortion.
Compared with baseline multilingual systems tested in the study, which achieved only 15 to 25 percent macro-F1, the structured prompt-based Large Language Model approach performed substantially better.
However, 57.5 per cent accuracy is not sufficient for autonomous decision-making in a high-stakes anti-corruption context. What it is sufficient for is decision support.
At this level of performance, the prototype can:
The human decision-maker remains central. The Artificial Intelligence prototype acts as an assistant, not a replacement.
Performance depends heavily on context. Classification accuracy varied across misconduct categories. The prototype performed best when complaints used clear and distinctive language. For example, explicit references to bribery were easier to identify and categorise accurately. It struggled more when categories overlapped or relied on subtle legal distinctions. Where wording was similar across misconduct types, or where meaning depended heavily on context, consistent classification became more difficult.
Two factors largely explain this. First, some misconduct categories appeared far more frequently in the test data than others. Rare categories offered fewer cases for measuring performance, and errors were more common in those less frequent case types.
Real-world data matters, and so do data constraints. Because corruption complaints contain sensitive information, the prototype was tested using semi-synthetic data, designed to imitate real submissions while keeping live operational files out of the test environment. This enabled structured evaluation but limited the extent to which the test environment reflected full operational complexity. In addition, the Large Language Models were not fine-tuned on domain-specific Thai legal material. They were guided through carefully designed prompts instead. For a prototype, this approach reduced governance risk. But for long-term operational deployment, deeper contextual adaptation would likely improve performance. The lesson here is practical: early pilots can demonstrate potential, but production-level systems require secure access to real-world data environments, ongoing evaluation and investment in domain-specific refinement.
Human oversight must be designed in, not added later. Staff feedback made one point very clear: In high-stakes integrity and regulatory contexts, over-reliance on automated outputs can pose risks. At the same time, ignoring Artificial Intelligence entirely can mean missing the opportunity to maximise efficiency. Artificial Intelligence outputs must not be treated as definitive. Users emphasised the importance of:
Multilingual and informal contexts require caution. Finally, the study highlights the challenge of applying general-purpose language models to complex administrative settings. Citizen complaints are rarely written in standardised legal language. They may include informal expressions, mixed terminology, incomplete information or regional phrasing. In multilingual environments, translation nuances add another layer of complexity. These factors can reduce consistency in automated classification. For public institutions operating in linguistically diverse contexts, this suggests that careful prompt design, ongoing monitoring and transparent communication of limitations are not optional extras. They are necessary safeguards.
Launch year: 2026
This case study was written with assistance from artificial intelligence.
Connect with 500,000+ public servants solving your hardest challenges.
Connect with 500,000+ public servants solving your hardest challenges.
Help public servants worldwide learn from your work, what worked, what flopped and what you'd do differently
Share your project
Log in or sign up to continue the conversation