Development and Demonstration of Generative AI-Based Barrier- Free Content Production Technology for Persons with Disabilities, the Elderly, and Others

Overview

Media-vulnerable groups such as persons with disabilities and the elderly find it difficult to understand recent broadcasts—which feature rapid plot developments and emotional shifts—using standard subtitles that convey only the speaker's voice. The government mandated that 132 broadcasters provide hundreds of thousands of hours of TV captions for the disabled. However, existing human-based captioning services face issues like cost. Introducing generative AI technology can automate most captioning tasks and also incorporate background, atmosphere, and external sounds into the content, enabling improvements in public services.

Project description

For AI training, 1,000 hours of video and 8 million sentences are collected from broadcasters like KBS and JTBC. Through training, the AI generates text from the speaker's voice in broadcast videos and extracts background, atmosphere, and ambient sounds from the screen. These results are then converted into barrier-free subtitles and inserted into broadcasts.

This service provides user-friendly subtitles for media-disadvantaged viewers while simultaneously supporting broadcasters by reducing costs and enabling diverse revenue streams. These include story/audiobook services, short-form content production, scene search services, and translation services.

Key technological features

• Multimodal Captioning (AI) Model (CLIP): Screen Feature Extraction, Sound Feature Extraction
• Generative LLM (LLaMA3 (8B)): Outputs barrier-free subtitles and screen description text
• Text-to-Speech (Fastspeech2): Screen descriptions for the visually impaired trained on user data, then executed with minimized errors

Expected outcomes

By automating broadcasters' subtitling services—which previously relied entirely on human labor— through AI technology, this initiative is expected to contribute not only to public services but also to the overseas export of high-quality K-content. The developer, the WiseNut Consortium, plans to expand the Barrier-Free Subtitle Service beyond its 2024 pilot at Mokpo MBC to include nationwide regional MBC stations, KBS cable broadcasts, cable VOD services, terrestrial OTT platforms, and advertising by 2025.

Launch year: 2024

Attribution

Case study courtesy of the National Information Society Agency, Republic of Korea, Apolitical's Content Partner. Sourced from Use Cases of Public AI Service, Vol.1,  https://eng.nia.or.kr/site/nia_eng/ex/bbs/View.do?cbIdx=31975&bcIdx=28997&parentSeq=29007