Whether someone has difficulty hearing or doesn’t speak the native language, essential information fails to reach the people who need it more often than we might expect. Through YYSystem, a speech recognition app that AISIN continues to develop as a communication aid for people who are deaf or hard of hearing, we are working toward a society where everyone can communicate with greater ease.

Downloads across the YY app series have now surpassed three million, and use has spread beyond individuals to local government service counters, hotels, railway stations and more.

This growth has been driven by a development approach rooted in co-creation: refining the technology with input from the people who actually use it.

Where YYSystem began

Today’s YYSystem voice recognition app series traces its roots to YYProbe, a speech recognition app developed by AISIN in 2019.

YYProbe was originally designed to use highly accurate speech recognition to capture and analyze conversations in work settings and meetings to help preserve operational know-how and improve productivity.

Time lag and misrecognition are common frustrations with transcription tools. To address this, the app was built on AISIN’s proprietary algorithm, which lets it recognize speech quickly and accurately, even amid the noise of a manufacturing plant.

Another distinctive feature is its ability to detect laughter. An AI model trained on samples of many kinds of laughter recognizes when people laugh during a conversation and displays “(Lol).”

*1 When laughter is detected, “(Lol)” is displayed, and the color of the text bubble changes to pink.
*2 When a speaker laughs while talking, “(Lol)” is added at the end of the sentence.

Junki Yamakawa, a member of the YYSystem development team, explains the thinking behind making laughter visible:

“We wanted to convey not only what the speaker says, but also their emotions and the atmosphere of the conversation. Real conversations are full of cues, like laughter and hesitation, that reveal the mood of the moment and how the speaker feels. Yet that information usually gets lost in records such as meeting minutes. By making the nuances of a conversation visible, we set out to deliver new value to users. Visualizing laughter is just the first step.”

Junki Yamakawa, YYSystem Development

Supporting employees who are deaf or hard of hearing: where co-creation with users began

 Aisin Wellsmile Co.,Ltd.,an AISIN Group subsidiary that promotes employment for people with disabilities, came up with the idea of using the app to support communication in workplaces that include employees who are deaf or hard of hearing.

In March 2021, AISIN launched a project that distributed devices with YYSystem installed to about 300 employees who are deaf or hard of hearing. In daily use the app accurately recognized speech despite noise from operating machinery and other sources across the plant, and it correctly displayed in-house jargon and technical terms, proving its performance in real-world conditions.

Users said the app made it easier to take part in meetings and improved the quality of their work by helping them follow conversations more fully. It has proved to be a highly effective tool for breaking down communication barriers.

Through this real-world use, we have continued to refine YYSystem’s features and uses based on the challenges users face and the feedback they share.

YYSystem expands beyond AISIN

Building on this in-house track record, AISIN began offering the app outside the company in 2021.

At local government service counters, including in the town of Abu in Yamaguchi Prefecture, where the population is aging, the app was introduced to display town officials’ speech as real-time captions on a tablet. At the time, a staff member of the Abu Town Health and Welfare Section, who used it at the information desk, said: “The app has made it easier to communicate with residents who have hearing difficulties and has helped us improve our services. Other staff have also responded well, because it converts speech to text accurately and is simple to use.”

Since then, use has expanded to government offices, companies, hospitals, schools, and other settings. YYSystem is also being rolled out at railway stations, with installations underway at roughly 40 locations across Japan.

This expansion has been driven in part by growing inbound tourism and by an April 2024 revision to Japan’s Act on the Elimination of Discrimination against Persons with Disabilities, which extended the requirement to provide reasonable accommodations to private businesses.

Making station announcements visible at Meitetsu Nagoya Station

In April 2026, a new demonstration project began in partnership with Nagoya Railroad (Meitetsu).

Meitetsu Nagoya Station can serve up to 280,000 passengers a day, with about 900 trains arriving and departing on just two tracks, as often as every two minutes. With the 20th Asian Games (Aichi-Nagoya 2026) being held in fall 2026, large numbers of visitors from across Japan and overseas are expected to pass through the station.

At Meitetsu Nagoya Station, trains bound for different destinations arrive in rapid succession. Station staff make announcements live, but because each is delivered only once and only in Japanese, the information can be hard to catch for people with hearing difficulties and for international travelers.

This led to an initiative to transcribe staff announcements in real time, display them on monitors, and translate them into English, Chinese, Korean and other languages.

Although the app had already been introduced at other railway stations, this is the first project to transcribe and translate live platform announcements in real time.

Deploying the system on a live station platform, however, posed a real accuracy challenge. Yuki Maeda, who was responsible for implementing YYSystem at Meitetsu Nagoya Station, reflects on the experience:

“Station announcements are packed with station names, train types and distinctive phrasing, one after another, so this was far from easy. General-purpose speech recognition couldn’t transcribe them properly, and early on, raising recognition accuracy was a constant struggle.”

Yuki Maeda, YYSystem Implementation

“My Engine:” Speech recognition tailored to each speaker

The breakthrough came from “My Engine,” an AI-powered feature unique to YYSystem. This speech recognition engine uses AI to learn an individual’s speaking style and vocal characteristics, optimizing recognition accuracy for that person. It was originally created to support communication for people who are deaf or hard of hearing.

The speech of some people who are deaf or hard of hearing can be difficult for general-purpose speech recognition to capture, which is why optimizing for each individual voice is necessary.

“My Engine” was developed and refined in collaboration with Microsoft Japan. YYSystem’s speech recognition runs on Microsoft’s Azure AI, and working with Microsoft Japan on My Engine helped achieve a high level of accuracy.

The effort has also drawn international attention: Microsoft Chairman and CEO Satya Nadella highlighted AISIN’s work during his keynote at Microsoft Build 2025.

Maeda recalls how the team used “My Engine” on the Meitetsu project:

“Over six months, we trained the system on station names and train types, as well as the speech patterns of the roughly 40 staff members who make the announcements. As a result, it can now accurately transcribe Meitetsu’s unique station and train names, its distinctive phrasing, and even rapid-fire announcements. Recognition accuracy improved from about 78-percent before training to about 97-percent.” (Based on AISIN internal measurements.)

The future of YYSystem, built together with its users

What has guided YYSystem’s development from the start is co-creation — building it together with the people who use it.

In demonstration projects, we intentionally share examples of misrecognition alongside successful results and continuously make improvements based on feedback received on social media. This approach has been recognized as an example of co-creative inclusive design, and an AISIN patent related to the speech recognition technology received the Aichi Invention Award.

AISIN will continue enhancing YYSystem to meet diverse needs and expand its reach, working toward a society where everyone can communicate smoothly and with confidence.