Google Assistant Is Working On A Feature Called ‘personalized Speech Recognition’

Google began using federated learning on Android in March 2021 to improve "Hey Google" hotword accuracy. An upcoming "Personalized speech recognition" feature will now assist Google Assistant in becoming "better at recognizing users' frequent words and names."
SIA Team
May 29, 2022

Google began using federated learning on Android in March 2021 to improve “Hey Google” hotword accuracy. An upcoming “Personalized speech recognition” feature will now assist Google Assistant in becoming “better at recognizing users’ frequent words and names.”

According to Google, they decompiled the most recent version of a Google Play Store application.  Various lines of code have been found within the said files (called APKs in the case of Android apps) that hint at possible future features when they decompile them. Google reminded users that they may or may not ever release these features, and their interpretation of what they are may be flawed. However, those that are closer to completion will be able to show users how they will look if they do ship. 

According to strings in recent Android versions of the Google app, “Personalized speech recognition” with the following description will appear in Google Assistant settings:

It will keep audio recordings on this device to help Google Assistant improve its recognition of what the users say. The audio remains on the device and can be deleted at any time by disabling personalized speech recognition.

The upcoming feature aims to extend machine learning-based improvements beyond “Hey Google” to users’ actual Assistant commands, particularly those with names (for example, using users’ voice to message contacts) and frequently spoken words. Audio from previous utterances will be saved on the device and analyzed to improve transcription accuracy in the future.

Google already uses a machine learning chip on devices like the second-generation Nest Hub and Mini to locally process users’ most common queries “for a much faster response time,” and that concept may now be expanding beyond smart home devices to Android.

Given Google’s stance on Assistant and voice privacy, this will almost certainly be an opt-in feature, similar to how Assistant settings > “Help Improve Assistant” is currently. According to the description available today, “audio stays on this device” and is deleted when the capability is disabled. Meanwhile, if users disable Personalized speech recognition, Google warns:

If users disable this feature, the user’s Assistant will be less accurate at recognizing names and other words users frequently say. This device will delete all audio used to improve speech recognition for users.