Students build apps for Grace

“A few years ago I saw the current state of the art for people with cerebral palsy, quadriplegics, and paraplegics, in order to make interfacing with the computer better. I thought, it’s kind of cool… but I know we can do better.” – Dr. David Chesney

Students in Dr. Chesney’s EECS 481 Software Engineering class demoed their senior projects at a showcase event on April 22. The projects were designed as assistive technologies for Grace, a young woman with athetoid cerebral palsy who has shared her time with Dr. Chesney’s class for the past two semesters.

While Grace can read, understand spoken language, and signal “yes” or “no,” she can’t speak or use extensive sign language to communicate because of her limited muscle control. The projects that students have created in Dr. Chesney’s class have been designed to help Grace to communicate or to engage more easily in social activities.

Some of the teams leveraged a unique touchscreen operating system that was developed as an original project in EECS 481 previously. Now called ASK Interfaces, it has been further evolved as an OS platform for people with disabilities and is being developed and commercialized by original team member Chris McMeeking, who is now Dr. Chesney’s teaching assistant. ASK Interfaces scans through a list of possible selections and essentially turns the entire tablet’s screen into a button. By making successive choices, the user can quickly drill down to the desired final choice.

One team that built upon the ASK platform created JamMaster Grace, which allows a user to compose music on a tablet device. The program displays columns of beats into which the user can add a note or chords. Successive columns allow for additional notes, chords, or rests. The app also allows for choices of instrument tones. When asked, team member Rachael Miller confirmed that Grace had used the app successfully. “Because the whole screen becomes a button for selections, the ASK interface works really well for people like Grace who have fine motor control challenges.”

Other teams developed projects for Grace that employed technology such as Microsoft Kinect, which tracks body movements, and Google Glass. One team built their own customized controller for Grace, using an Arduino board and integrating the same type of pushbuttons that are found in devices that Grace uses at home. The controller box is integrated to with a tablet device and allows for single or multiple user play of the team’s implementation of the popular Hasbro game Sorry. The students who developed this project were Nicole Grieble, Nicholas Augustyniak, Maxwell Warner, Stephen Woodruff, and Zoltan Nemeth.

Some teams used a device from MIT called the MaKey MaKey, which can turn anything that conducts electricity into a USB input device for a computer, to create novel input devices for Grace. One of these, KeyUp, was built by Thomas Dorsey, Steve Dougherty, Jonathan Kim, Ruixuan Wang, and Rishi Zaveri and uses parsing software to help people with disabilities to type more quickly. Because the MaKey MaKey needs to complete an electrical circuit to operate, the team developed an optional aluminum-foil hat to make the project more fun.