Search across all content
projectWith monetary damages from voice phishing crimes in Korea standing at 3 trillion won over the last five years, citizens have suffered massively, and socioeconomic costs for crime prevention have skyrocketed.
This information was sourced from "Development and Operation of the 「Korea Voice Analysis Model (K-VoM) for Voice Phishing」, designed to capture ‘Criminal Voices’" by Observatory of Public Sector Innovation (OECD), https://oecd-opsi.org/innovations/development-and-operation-of-the-%e3%80%8ckorea-voice-analysis-model-k-vom-for-voice-phishing%e3%80%8d-designed-to-capture-criminal-voices/. Licensed under Creative Commons Attribution-ShareAlike 3.0 IGO (CC BY-SA 3.0 IGO).
With monetary damages from voice phishing crimes in Korea standing at 3 trillion won over the last five years, citizens have suffered massively, and socioeconomic costs for crime prevention have skyrocketed. Comparing suspect voices to known criminal voices and specifying offenders is a key step in the criminal investigation process, and had been handled by the National Forensic Service (NFS). The NFS had conducted voice analysis with models developed overseas, including the UK and Russia, but the models that had learned data based on foreign languages, by nature, had limitations in precisely distinguishing Korean users. Furthermore, as voice phishing criminal organizations generally operated by forming groups (playing the role of investigators and prosecutors), there was a dire need for functionality that could identify clusters of identical criminal groups by chaining voice tracking, but this too did not exist in the existing model.
The Integrated Data Analysis Center at the MOIS, alongside the NFS, implemented the development of the first AI-based voice analysis model that could more precisely identify criminal voices and form clusters of criminal organizations. More than a million Korean and overseas voice datasets from approximately six thousand speakers were utilized. The voice analysis model, K-VoM, was gradually established by repeating the processes of teaching the AI model and verifying its performance with the latest AI deep learning technology.
Once the K-VoM’s performance was tested, an approximate 77% increase ( 51% from the existing 28%) in read rates for precise identification of criminal voices was confirmed in comparison to the existing overseas model. The new model has been applied to the NFS’s voice phishing audio analysis process beginning in February, 2023. Going one step further, a police investigation version of the model was also developed so that police forces in charge of criminal investigations could verify criminal voices with speed and ease in the field. The police investigation version was distributed to police forces nationwide beginning July, 2023.
Due to its increased performance relative to existing overseas voice models, the K-VoM raised the credibility of voice analysis being utilized in criminal investigations. As the K-VoM was utilized not just by the NFS, but also directly by the police, investigations were sped up and criminal arrest rates were raised. This marks an outstanding turning point in reducing criminal damages to Korea’s citizens. Recently, the Korean police have managed to capture 51 members of a voice phishing criminal organization utilizing the K-VoM.
The K-VoM largely possesses two innovative elements in comparison to existing voice analysis models developed overseas. Primarily, the model can identify human voice with 77% higher precision, thanks to the application of AI deep learning technology. Next, the model, for the first time in the world, implemented functionality that chain-tracked criminal voices, discovered individuals involved in crime, and formed clusters, considering the core nature of voice phishing crimes being conducted by groups of criminals.
In order for K-VoM to be actively utilized in the field for police investigations, user education has been conducted for police officers, beginning in July, 2023. Regular education for police officers are planned to be conducted at least twice a year moving forward. There are also plans to share this new analysis model with the international community, which continues to experience damage from voice phishing crime. Specific discussions are currently underway with the Inter-American Development Bank for this expansion. Moving forward, K-VoM will be further advanced to be continuously utilized against voice phishing crime, which continues to evolve with each passing day.
The model was developed with solid collaboration between the MOIS’s Integrated Data Analysis Center and the NFS. Seoul National University’s voice analysis researchers and investigators of the National Police Service provided a vast trove of expertise. By thoroughly applying the actual needs of potential users of the model in the field, the model achieved high on-site compatibility and performance. Collaboration was a core resource in enhancing both the quality and quantity of this innovation.
With this innovative voice analysis model, the NFS was empowered to perform voice identification with high fidelity, 77% enhanced from existing models, and execute focused analysis that formed data clusters of individuals involved in voice phishing crimes. Furthermore, as police officers on the front lines in the field could rapidly perform voice analysis, faster criminal arrests became possible. All these changes serve as factors to reduce the criminal damage affecting our citizens.
State of the art technology and participation from experts allowed this high performance model to be born. Actual criminal voice data was utilized in two rounds of performance verification. Verification tests wielded read rates for identifying criminal voices at 51%, higher than existing foreign-developed models with 28.7% rates. Furthermore, for the first time in the world, technology that formed data clusters for individuals connected to each respective case was established. Police, utilizing this model, managed to perform an arrest of 51 criminals in October, 2023. With this model widely utilized in the field, we expect our citizens to experience substantially less damage.
In the initial stages of development, we experienced the difficulty of the fidelity of the new model not being significantly improved in comparison to existing models. A solution to the issue was discovered after rounds of discussions with specialized research institutes, including the NFS and Seoul National University, with knowledge of the voice analysis field. There were further challenges acquiring the necessary channels for police forces nationwide to utilize the K-VoM. After discussions with the National Police Agency, the existing ‘Investigation Support System’ operated by the police was first upgraded, after which the K-VoM was installed and actual service could be activated.
Innovation requires courage. Problems must be identified as problems, and a willingness to improve them should exist. Many of us know problems exist, but chose to adapt to them and live on. An organizational culture that permits its members to be courageous is also crucial. Various resources, inside and outside the organization should be utilized to improve existing problems in the most efficient way. The approach taken for K-VoM, where the latest advanced technology was utilized, and various stakeholders were guided to participate, just for the sake of reducing the damage caused by voice phishing, is a prime example of this idea.
We are focusing various efforts to share our knowledge with the international community. First, in 2023, by conducting training programs for investigative agencies of developing nations such as Mongolia and Indonesia, we shared methods for audio analysis. Development results for K-VoM have been shared at international specialized forums, including the ‘Asian Forensic Sciences Network Symposium’ (September, 2023), with strengthening international communication in the field of voice identification. Through collaboration with international agencies, including the Inter-American Development Bank, there are ongoing discussions to test expansion of the model in South America, beginning in 2024.
Innovation of high completeness cannot be achieved by singular efforts. Better methods and directions for good innovation can come from thorough communication with various stakeholders. It may be difficult to meet various stakeholders and apply their opinions. However, the process of listening to the ideas of those who differ from us is also an element of innovation. We must never forget that when different opinions and ideas collide, better concepts are formed, eventually fuelling innovation.
Budget: Not disclosed
Connect with 500,000+ public servants solving your hardest challenges.
Connect with 500,000+ public servants solving your hardest challenges.
Help public servants worldwide learn from your work, what worked, what flopped and what you'd do differently
Share your project
Log in or sign up to continue the conversation