Being Pokey Rule: Speaking software into existence
Last night I injured my shoulder at the gym. I'm OK - I'm prone and know how to treat it. I’m “writing” this with dictation using a Google Docs tool called Voice Typing because it hurts to move my shoulder right now.
Determining the beginning, middle, and end of anything I write is an organic process; I copy and paste things a lot. If I could use my voice to copy/paste segments of this blog post to dance a little jig with ordination, I’d be one happy blogger. Google’s Voice Typing does not allow these kinds of commands, only typing.
What if I told you Pokey Rule could command a computer to do a heck of a lot more than re-order a blog post? As if from a sci-fi movie, Pokey Rule, software engineer and Cursorless founder, uses his voice to design software and applications. He even uses Cursorless to develop fixes and new versions of Cursorless itself, which now has thousands of downloads and active users at many major tech companies. In fact, Cursorless was built almost entirely using Cursorless itself.
Pokey, like Captain Kirk, controls every aspect of the software and hardware that powers any device he’s using. The biggest difference between Captain Kirk and Pokey is that Pokey can crochet while he’s commanding machinery. How? Enter: ingenuity.
His loss of mobility due to Repetitive Strain Injury (RSI) motivated Pokey to develop a new way of using the computer for himself, and also for the world. A repetitive strain injury is “damage to your muscles, tendons or nerves caused by repetitive motions and constant use,” according to Clevelandclinic.com.
He calls it Cursorless and it is one of the most exciting things I've seen in accessible computing. Pokey created Cursorless, a spoken programming language that lets users eliminate the need for a cursor entirely, while experiencing intense pain in his wrists. A career filled with long hours of coding software left Pokey with compromised wrists. RSI itself can debilitate its sufferers and if unmanaged, it can develop into something more severe like carpal tunnel syndrome.
Pokey was motivated by his injury. And yet, a truly aspirational goal to create a computer language that would let him and any user (even those without injury) code even faster than by typing solidified his commitment to create Cursorless. Today Cursorless is a highly efficient way of commanding a device and coding. And it is totally open source aka free to all. Pokey succeeded!
Let’s get into the details of how the language works. Most surprisingly, Pokey taught me that the “cursor” in “Cursorless” doesn't actually refer to the mouse pointer! Instead the name refers to the text input cursor where you type and select text. Rather than moving your cursor around to edit text, you leave your cursor where it is and verbalize what you want to have happen, based on its location on the screen. This article on caret browsing from Wikipedia can help clarify the text input cursor and includes a visual.
In other words, by bypassing the cursor, a Cursorless user can rapidly edit multiple places in the document in a single breath. They can still use a cursor when convenient, and what’s more - Cursorless actually lets its users interact with multiple cursors at once.
Cursorless enables many cursorless operations (aka operations that use no cursor at all), yet it also can execute operations that change or reference one or many cursors. A user would rely on multiple cursors to edit in many locations simultaneously.
The spoken computer language entirely eliminates the need for a mouse and keyboard when writing code and while using any feature of a computer. Instead of physically engaging with a mouse to navigate and edit code, art files, text, etc., Cursorless allows developers to speak their desires rather than “point and click them.” Pokey primed Cursorless with spoken language commands to interact with a device. Once installed, users learn the programming language and rely on its syntax as a distinct spoken language in order to make commands like these: "pre next line" and "chuck word.” Streamlined and expressive, Cursorless also proves easy to learn and use.
Other features Pokey packed into Cursorless to make it as powerful as it is include:
A fully customizable vocabulary to adapt to the individual needs / accent of any user
The ability to create new, custom commands
The ability create many cursors at once to allow you to edit in many locations simultaneously
Context-sensitive commands
Support for editing code in 20+ programming languages and growing
“When I first started programming by voice, it was incredibly frustrating. I struggled to get to 80% of my previous coding speed. I spent a couple days throwing together a quick hack to let me jump around a document. Two years on, that “quick hack” has blossomed into a rich and expressive spoken language for transforming code. The community that has grown up around this language has been incredible,” Pokey explained. “If there’s anything I’ve learned from this experience, it’s that a dominant narrative can suffocate innovation: the notion that a cursor is the way to edit text has gone unquestioned for decades. To move forward, we need to actively seek out and amplify the voices of those for whom the status quo is not working.”
I don't know if Pokey identifies as being disabled. I do know that he managed to uplift people around him by creating a solution that could help many people and not only himself. In a previous blog post, I touched on a pattern of behavior where people with disabilities look to uplift all of the people around them.
Even before connecting with Pokey over Zoom, questions raced in my head. I’ve been so excited by many accessibility challenges that Cursorless lets its users overcome that I wanted to really identify the advancements or evolutions most important to Pokey. Pokey, joining Zoom from London, shared with me that he did not inherently observe the fact that Cursorless is a way to connect folks with pre-existing mobility challenges to a lucrative career path. Whether that be software development or using Cursorless in a business role like social media marketing or human resources, commanding a computer by voice quickly and accurately will help include tons of marginalized communities of people who would formerly face discrimination about how they’d perform these jobs.
In my conversations with various people experiencing life with disability, I’ve observed a shared desire among them and myself. We all want to make a contribution to society rather than simply participate. Holding a meaningful job lets any citizen contribute; Cursorless allows people with disabilities to become more eligible for dignifying jobs. Another way to think of it - learning to use Cursorless could make it nearly impossible for companies to discriminate at hiring
When I asked what if a startup required using it? Pokey shared with me that requiring use of Cursorless could invite an accessibility issue all its own. For example, a user with vision loss may not be able to see what are known as “hats” when using Cursorless. The hats are what maps out the screen allowing users to verbalize commands based on the location of items (text, code, images, popup windows, etc.) onscreen.
The fact Cursorless addresses so many accessibility issues remains celebratory. I’m actually obsessed with how many hugely impactful ways that Cursorless will help our society. You might think of using Cursorless as a healthier way to code. The spoken computer language is a form of preventative healthcare. Using voice - whether 100% of the time or partially - would reduce strain on a person’s wrists and vastly decrease their risk of developing debilitating mobility challenges.
I started to ask Pokey about educational programs and it was fun and inspiring to think about how Cursorless could be adopted. If lessons on the spoken computer language were weaved into mainstream high school and higher education programs, students would benefit in these ways:
Students would become aware at a younger age that humans are not invincible. We use our bodies to execute our desires and unfortunately no human is guaranteed a healthy life. We achieve our best chance at living a healthy life through prevention of disease, injury, and disability.
They’d be able to code more quickly. Yes - even users of Cursorless who are free from disability code faster with their voice.
Students (and all people) with ADHD can increase the levels of dopamine and norepinephrine in their brains through movement. These chemicals are perpetually in short supply for sufferers of ADHD. According to John Ratey, M.D. and author of the book Spark, even small, simple movements like fidgeting increase the levels of these neurotransmitters in the way ADHD medications do. Those with ADHD experience subconscious urges motivated by their brain and nervous system’s desperation to release dopamine and norepinephrine. Think of Pokey crocheting! Activities like this conducted in tandem with coding by voice can balance neurotransmitters in the brain for these busy humans and promote focus and attention to detail.
It gives young people hope. In today’s world of smart people making technology for their own benefit (typically as it relates to making money), we can be inspired by Pokey’s decision to build an open source program that only helps its users.
Voice coding can be an answer to the ever-increasing demands placed on students (and workers) to do more and more. Cursorless just might be capable of easing the burden of capitalism on young Americans.
The most interesting aspect of Pokey is his humility. I love speaking with brilliant people and it’s a huge bonus when they’re also as altruistic as he is.
I was not paid to write this post nor is it an official endorsement. What I’m certain of: increasing the rate of adoption of Cursorless benefits our society by 1) preventing injury and 2) eliminating employment discrimination faced by those already living with a disability.
The Cursorless community uses GitHub - check it out.
