Since AISIN introduced the YYSystem, a communication support app for people with hearing impairments, it has spread rapidly to various areas of society. The number of users has been on a steady incline, and reached one million downloads by November 2024. Efforts are underway to create a society in which people with hearing impairments can communicate with confidence.

This article presents a behind-the-scenes perspective with the project leader and project promotion members who developed this app, including its history, its evolution with its users, and the thought that went into it.

Continuing to evolve through development that is close to users

The YYSystem voice recognition app series, which includes YYProbe and YYTranscription, was developed to support communication for people with hearing impairments and those who have difficulty hearing sounds in everyday life due to aging. The app series is capable of displaying the contents of your conversation in real time on a smartphone, display panel, and the like.

YYSystem has been extremely popular since its release. Its biggest feature is its highly accurate voice recognition. In addition to simply transcribing, the service is designed to convey the reactions of those around you, the emotions of the speaker, and the atmosphere of the conversation.

For example, a person with a hearing impairment may not notice applause from behind him or her while making a presentation at a meeting, but if he or she is using YYSystem, he or she will see “clap!” visualized on the screen. This allows the person to share the reaction to his or her own comments with those around him or her, enabling deeper communication.

Another major feature is that the project members use social media to actively collect requests from individual users, such as “I want this kind of function” or “I’m having trouble with this,” and then develop the product in a co-creation style that allows them to reflect these requests flexibly and extremely quickly.

Development project leader Masaki Nakamura had this to say about the co-creation style development:

“We communicate with our users on a daily basis, building relationships so that they feel comfortable coming to us for advice on even the smallest of issues. Some examples of listening to that input include the creation of a function that displays expressions such as “lol” or “hahaha” when laughter occurs during a conversation, as well as a function that uses AI to summarize long, difficult to understand stories. The app YYSiren-Alert, which visualizes and notifies users of emergency siren sounds, began in response to feedback from users who were having trouble noticing the sirens of ambulances and police cars while driving.  YYSiren-Alert was developed together, with our users, and this closeness to users is what differentiates YYSystem from other apps.

By co-creating together, we receive support and encouragement from our users while greatly enhancing our product which translates into business revenue and allows us to continue our activities.”

The co-creation style involving users has been highly praised, and in 2023, YYSystem brought AISIN the Good Design Award operated by Japan Institute of Design Promotion. In 2024, the Best Design Award was won in the Integration Design category of the Golden Pin Design Award, which selects outstanding designs from around the world.
* The Golden Pin Design Award was started in Taiwan in 1981 to select outstanding designs from works around the world. This year, 2,339 works were submitted from 21 regions around the world, of which 642 were selected for the Golden Pin Design Award. Moreover, 32 particularly outstanding works were awarded the Best Design Award, and YYSystem was selected as one of the prestigious winning works.

“Seeing how YYSystem can be useful to everyone makes me truly realize why I want to be an engineer, and it gives me emotional support. As a developer, I thought I was supporting the lives of users, but now I feel like I’m being supported by everyone even more than that,” added Nakamura.

The Noto Peninsula earthquake has made it possible to use the service offline

Nakamura also said that he wants to develop YYSystem into an essential tool that can ensure users’ safety and security even in times of emergency. After the Noto Peninsula earthquake that occurred in January 2024, Nakamura received a huge amount of information from users and found that many earthquake alerts were only delivered as audio, which was a problem for them. He immediately began broadcasting the earthquake alerts shown on television in text form over the Internet, informing many hearing-impaired people of the situation.

In addition, since there were time restrictions on usage for individual users, Nakamura lifted all time restrictions one hour after the earthquake occurred. “I had heard that during the Great East Japan Earthquake, many people became victims of the tsunami because they did not hear the evacuation announcements, so I made the decision immediately,” he said.

During the Noto Peninsula earthquake, there were times when signals from mobile phone companies were not working. Therefore, Nakamura and the development team put all their efforts into developing a voice recognition engine that would function offline and released it in November of2024. Future tasks include connecting with local governments and NPOs to create communities that are prepared for emergencies. They are considering creating a system that will fill in the gaps that technology cannot cover by combining YYSystem with analog support.

Aiming for normalization

Meanwhile, Kiyomi Kusaka, who works with YY System as an implementation support consultant, points out that there are still many challenges to overcome when it comes to understanding and communicating with people with hearing impairments.

“I had a consultation with a workplace where a hearing-impaired employee was taking every Thursday off. At first I couldn’t figure out the reason, but after listening more closely, I realized it was because the employee was unable to keep up with the work meetings on Thursdays. The colleagues around the employee assumed that he understood what was being said based on the speaker’s mouth movements, but in fact this was not the case. I thought that by putting words into writing, it would be possible to communicate information in real time and enable him to participate in discussions, so I proposed introducing YYSystem. As a result, not only did everyday communication become smoother, but the effects were greater than we had expected; he became able to laugh together with his colleagues, gained confidence, and was motivated to take on new tasks.

This is an example from a workplace, but I think there are many people in society who face similar misunderstandings and problems. What we aim for is the normalization* of society as a whole. We would like as many people as possible to use YYSystem, and while carefully listening to user feedback, we will continue our activities to bring us closer to a world where everyone can live together happily.”
* A concept that aims to create a society in which people with and without disabilities can live equally

Convenient use in a wide range of situations in society

In addition to transcribing text, YYSystem can instantly translate the displayed text into more than 20 languages. As a result, there has been an increase in inquiries from our customers working at hotels and train station counters, where there is a need to assist an increasing number of foreign customers at the counter.

Satoshi Ozawa, who is in charge of implementing the system for corporations, speaks passionately about wanting to establish YYSystem as a system that all people can rely on when they have communication problems.

“There is an increasing number of foreign visitors to Japan. Hotels and station counters now need to be able to handle a variety of languages, including not only English but also Chinese and Korean, which has led to many communication problems. We have received positive feedback that the introduction of the system has enabled both counter staff and customers to have stress-free, comfortable conversations. YYSystem is a system that can remove communication barriers, regardless of whether or not a person has a disability or of nationality. By working with everyone who uses it to create an easy-to-use system and operating methods, we hope to create a world where everyone around the world can communicate without difficulty.”

It has also been decided that the system was fully introduced at all branches of Tokyu Stay, a major hotel chain, in 2024, and will be introduced at Hankyu Umeda Station in Osaka in 2025, a train station in the run-up to Expo 2025. Thus, the number of counters where the system can be used is expected to gradually increase.

We will continue to evolve YYSystem for the people of the world

Lastly, we asked Nakamura about his vision and goals for the future.

“By implementing new technologies such as AI in society and having everyone use them, we aim to solve various social issues, including communication-related ones. There are countries in the world where there are almost no schools for the deaf. People born in such countries are likely to miss out on an education and are destined to be poor, which is unfair. YYSystem is a solution that contributes to addressing these social issues, and we will work together with everyone to improve its functionality and roll it out so that people all over the world can use it.”

AISIN remains committed to helping build a sustainable society to create a safe and comfortable future where no one is left behind. We are committed to the journey ahead.