SignAvatar – AI Speech to SignLangauge Translator

This information was sourced from "SignAvatar – AI Speech to SignLangauge Translator" by Observatory of Public Sector Innovation (OECD), https://oecd-opsi.org/innovations/signavatar-ai-speech-to-signlangauge-translator/. Licensed under Creative Commons Attribution-ShareAlike 3.0 IGO (CC BY-SA 3.0 IGO).

Innovation Overview

Imagine this world.

You wake up in your hometown and NOTHING is in your native language.You try to read a book or watch the news on TV and everything is in a foreign language that you can’t understand. You go out and try to find anything written in your native language, but every store, café, or public square you visit proves that it is virtually impossible. And now, on top of that, you can’t even communicate with most of the people, as they don’t speak your language.
Seems tough? Well, that’s what life is like for 70 million Deaf people worldwide.

SignAvatar is making significant strides in technology development.

Deaf people say that where there is speech there is a need for Sign Language. Our overarching vision is real-time sign language generation, crucial for universal accessibility. While achieving this goal entails a longer process, our immediate focus is on our pioneering product, TransportSign.

TransportSign, our first product, is already implemented on Belgrade Central station.
It is a system for auto-generxated service information in Sign Language for Airports and Train Stations. Everything that you can read on screen and hear over the speaker is now available for the Deaf in Sign Language on dedicated screens on the airport and integrated inside an app.

For this we were even praised by the president, see the link: https://www.youtube.com/watch?v=zYSaUVkqs9E&ab_channel=RedTechSerbia

Rather than the traditional marketing approach involving paid ads, SEO, and marketing agencies, we've allocated our budget for demand generation within the Deaf community.

Our primary strategy for engaging with the Deaf community involves grassroots advocacy and a word-of-mouth sales approach, focused on younger generations (TikTok)

1. Our Deaf Community Builder raises awareness and reaches Deaf influencers
2. Partner with Deaf Influencers to promote our solution
3. Doing this in a small community will lead to more Deaf people talking about it
4. Make them our advocates

Creating an army of supporters, particularly among the younger, tech-savvy generation, is crucial.
This approach seeks to counter skepticism, especially among the older Deaf generation, towards avatar-based solutions which is crucial.
These partnerships will help convey the limitations of relying solely on interpreters and demonstrate the capabilities of our human-like avatars.

Although Europe boasts diverse languages, our small dataset requirement for TransportSign allows us to incorporate new languages in under a month. This flexibility ensures that we can readily expand into new markets as needed.

We have identified Countries that have train stations with high passenger volumes that also prioritize accessibility and inclusion.

These are (among others):
EU:
Slovenia
Sweden
Denmark
Norway
Finland
Switzerland

US:
New York - MTA - New York City Subway

What Makes Your Project Innovative?

Our innovative solution tackles the challenge by automating interpretation from spoken to sign language in real-time in order to provide more content available to deaf people.
We aim to create an AI-powered system that generates sign language based on text or speech input. We extensively engaged with Deaf individuals and organizations and this validated the pressing need for accessible content in sign language and reinforced the significance of our solution.
The traditional reliance on human interpreters is limited by factors such as availability, scalability, and cost.
By harnessing AI technology, we can bridge this gap and offer a scalable solution that transforms content into sign language, thereby enabling the Deaf community to access information and participate in various domains. Our approach leverages cutting-edge AI advancements, particularly the use of transformers and digital avatars, ensuring accuracy and natural expression in sign language interpretation.

What is the current status of your innovation?

Our first product is already on the market.

TransportSign was delivered to Belgrade central station in collaboration with the Railways of Serbia, including a license for one station in Belgrade with an interest to expanding to two other stations.

For 2024 we’re expecting 120,000 EUR from the Serbian market and an additional 150,000 EUR from European markets, primarily railways and airports.

Collaborations & Partnerships

Deaf organizations were pivotal collaborators, offering expertise in sign language and user-centric design. Their involvement ensured cultural sensitivity and accuracy in translations, vital for the innovation's effectiveness. Additionally, partnerships with government officials facilitated regulatory support and policy advocacy for inclusive public services, augmenting the innovation's impact and outreach.

Users, Stakeholders & Beneficiaries

TransportSign, primarily benefiting deaf individuals, revolutionizes travel accessibility. By ensuring vital information is readily available in sign language, it fosters a sense of security and inclusivity for deaf communities. Government officials garner recognition for championing inclusion and accessible transport. Deaf organizations' integral involvement ensured expertise in sign language and citizen-focused design, enriching the project from inception.

Results, Outcomes & Impacts

The innovation, TransportSign, showcased profound impacts. Over 85% of surveyed deaf individuals reported heightened travel safety, a tangible testament to its effectiveness. Utilizing focus groups and surveys as methodologies, we measured this shift in perception. As for future impacts, envisioning an app extension, we anticipate even greater outreach. Enabling real-time notifications via phone vibrations for crucial announcements aims to further enhance safety. Projecting a 40% increase in user engagement upon app integration, we aim to reach 50,000 downloads within the initial six months, fostering a more inclusive travel experience for the deaf community.

Challenges and Failures

Infrastructure issues posed challenges, yet no major setbacks occurred. Communication with clients remained smooth without notable hurdles. Swift resolution of infrastructure concerns ensured uninterrupted progress, maintaining seamless client interaction throughout the project.

Conditions for Success

For the success of an innovation like TransportSign, several conditions are vital. Firstly, a robust supporting infrastructure with reliable connectivity is crucial. Clear policies ensuring inclusivity and mandating accessibility in public services are imperative. Strong leadership advocating for equal access and guidance in implementing such initiatives is pivotal. Adequate human and financial resources, along with continuous support for technological advancements, are essential. Additionally, fostering personal values that prioritize inclusivity and motivation to enhance accessibility for all individuals are foundational for the sustained success of such innovations.

Replication

As of now, TransportSign hasn't been directly replicated, but future plans are set to position it as the forefront innovation in this domain. We aspire to establish it as the benchmark for user experience in accessibility solutions. Potential replication spans across various sectors: other organizations seeking inclusive services, departments within our organization aiming for comprehensive accessibility solutions, and both larger and smaller agencies or governments aspiring for universal accessibility in public services. By setting high standards in user-centric design and efficient implementation, TransportSign holds immense potential to be replicated widely, catalyzing inclusive practices across diverse sectors and organizations.

Lessons Learned

Our innovation, TransportSign, resonated deeply with deaf individuals, emphasizing its vital significance in their lives. It provided them with a newfound sense of security and independence during their travels. The lesson we've learned is the paramount importance of placing the user's needs at the forefront of any innovation. By prioritizing their experiences and perspectives, we've not only enhanced their safety but also fostered a profound sense of inclusion and accessibility. Understanding and addressing the specific requirements of the deaf community were pivotal in creating an innovation that truly catered to their needs, underscoring the importance of user-centric design in shaping impactful solutions.

Budget: Not disclosed