Singing Speech Synthesizer With Theremin-Like Controls

The embedded video is a collection of soundbytes that give a good general background of the Glove-TalkII system from Sidney S. Fels and Geoffrey E. Hinton.at The University of British Columbia. But the brief samples of the system in operation that it includes are, frankly, not a great advertisement for its capabilities or its potential. The best videos don’t seem to be available as embeds, yet, and to really appreciate how Glove-TalkII works, I recommend following this link to download a short AVI clip from the project homepage, showing operator Sageev Oore “singing” the alphabet song using hand gestures.

As with a Theremin, pitch is controlled by hand position in space. Closed-finger gestures, with one hand, create voiced consonant sounds, while open-finger gestures with the same hand give sustained vowel sounds. Hard stops, like “puh” and “buh” and “tuh,” are controlled with the fingers of the opposite hand. Looks like lots and lots of fun. [Thanks, Laura!]