Google Puts SL2T on Pixel 11: Sign-Language Input Meets Real-World Error Conditions

One number that would materially affect adoption is still missing: SL2T’s sentence error rate when Deaf or hard-of-hearing users operate a Pixel 11 in dim rooms, harsh outdoor backlight, or while moving. Google DeepMind has published its own benchmarks and examples of failures, but no independent field test. Over the next three to six months, error rates broken down by lighting, skin tone, clothing contrast, and signing variety would test whether this deployment can move from a controlled demonstration into routine communication.

Google DeepMind released its sign-language-to-text model SL2T on August 12, 2026. The first version translates American Sign Language into English and ships in Gboard and Live Transcribe on Pixel 11 at no additional cost. A Deaf user can sign toward the phone’s camera to produce text in a search box, message, document, or Gemini prompt. Live Transcribe also lets a user sign a response during a conversation instead of typing it. Google says more devices and languages will follow, but gives no schedule.

Video stays on the phone while coordinates go to the server

The data path starts with MediaPipe Holistic on the device. It tracks pose landmarks on the face, body, and hands, then sends geometric coordinates to the server for translation; the original video can be discarded immediately. SL2T produces English directly from that coordinate sequence instead of relying on the intermediate “gloss” labels common in earlier sign-language translation systems. This reduces the transfer of raw imagery and avoids a fixed label vocabulary that can lose facial expression and spatial grammar. It does not make the feature fully offline, because the coordinates still require server-side translation.

The training corpus contains more than 100,000 hours across over 50 sign languages, with roughly one quarter in American Sign Language. Google reports a zero-shot score of 70 BLEURT on FLEURS-ASL and says this exceeds previously published results. These are vendor-reported figures. A benchmark recorded under defined conditions cannot cover every case involving camera shake, occlusion, rapid signing, or multiple people in frame, so 70 BLEURT should not be treated as a universal user accuracy rate.

Pixel 11 moves model limitations into ordinary input fields

Google’s published examples show that SL2T can turn the rapidly fingerspelled word “prey” into “grey.” It can also drop passive constructions or classifier depictions and lose tense when context is absent. The development team says it worked on hallucinated text when nobody is signing, fairness for the 10% of signers who are left-handed, and one-handed signing while the other hand holds a phone. Those priorities show that the team identified practical risks, but Google has not disclosed the frequency of each error class.

Independent technology outlet SiliconANGLE confirmed the Pixel 11 rollout, the 100,000 hours training scale, the coordinate-upload mechanism, and the 70 BLEURT result. A joint impact report from Google and the AI Sign Language Advisory Committee describes SL2T 1.0 as an assistive tool for ordinary communication rather than a replacement for qualified interpreters in high-stakes settings. Both applications show a text preview before transmission so the user can correct it. That safeguard itself assumes enough English literacy to detect a mistranslation.

The next measurable results are the share of Pixel 11 translations that users edit, the false-trigger rate on scenes without signing, and the dates when additional devices and sign languages arrive. If Google expands availability without publishing error rates separated by lighting, skin tone, signing speed, and language variety, outside observers will still be unable to determine how much time SL2T saves in daily input or how much verification work it transfers back to users.

Sources:

• Google DeepMind: Putting sign language AI into users’ hands
• SiliconANGLE: Google debuts SL2T, an AI model that’s designed to understand sign language