The Twitch Streamer Using Face Recognition to Make Video Games More Accessible

Squirelo’s passion for accessibility is twofold. He grew up around friends and family members living with disabilities and also had ADHD. Beyond that, he has a strong desire to help people. The release of PlayAbility, which began rolling out late last year, is the culmination of years of collaborative efforts with his Twitch community to create a program that pushes the boundaries and understandings of accessible software.

“I always had a community on Twitch of people with disabilities and we were already trying to find stuff,” he says. Those players have been helping Squirelo from the start, offering advice, testing new features, and ultimately giving people tools to help them play.

Squirelo’s creation attempts to target the individualistic nature of the disabled experience. While adaptive hardware like the Xbox Adaptive Controller and PlayStation’s Access Controller offer customization through different buttons, switches, sticks, and button placement, disabled players are still limited to aspects like desk space, individual strength and energy levels, and even their finances. A basic PlayAbility account is free, and lets players control pretty much any game available on PC through whatever facial motions they have.

For Eduard Poch, Squirelo’s vision is clearly working. As a kid, Poch’s strength and dexterity allowed him to use standard controllers. Yet, as he grew older and his disability progressed, restricting his hand movements, he needed to rely on adaptive equipment. While accessible hardware was beneficial, his individualistic experience prevented him from fully using devices on the market.

“That’s why I bought a computer, because it was easier for me to use the keyboard and mouse,” Poch says. “Even so, I have trouble reaching certain buttons, since I can only have one hand on the left part of the keyboard. Thanks to PlayAbility, I can use a larger number of buttons immediately with a simple movement of the face.”

Poch’s experience of buying numerous devices to find the right solution speaks to an issue every disabled person encounters in gaming: the cost of playing. For those who rely on fixed incomes, they can expect to spend upwards of $300 for a complete Xbox Adaptive Controller setup, and approximately $250 for a complete set of Access Controllers. Combine that with the need to purchase additional switches, buttons, and sticks from third-party vendors, and adaptive equipment can cost the same as a console.

PlayAbility costs significantly less. Even the unlimited premium plan is just a one-time payment of €129 ($132). The only equipment needed is a standard web cam. And while the cost of PlayAbility is an expense that disabled players must account for, it pales in comparison to finding an adaptive hardware setup that may not even work as an individual’s disabilities progress.

In some of the program’s promotional materials, PlayAbility refers to the technology used as “cutting-edge AI,” but Squirelo admits the technology behind the software isn’t the high-powered stuff people think of when they hear “AI” these days. “We are using computer vision in a way, but we can call it AI,” he says. Instead, PlayAbility functions using algorithms that translate facial movements into game actions. This was especially beneficial when it came to the cost of developing the program.