GetLoFi has a nice rundown on how to get your bending with an inexpensive surplus “Saw III” voice recorder -
The device costs $2 and includes as set of 2 AAA Batteries! When it comes to bending there are a few interesting points and also some annoyances. The drawbacks include: no Loop Mode when play button is held, there is also a gap between each playback, and a audible beep at lower speeds before the audio starts. Despite the shortcomings there are a some very nice features. The recorded sample is stored even if batteries are taken out and the standard speed length of the sample is about 30 seconds.
The batteries alone could cost you more – gotta love those surplus deals! – Circuit Bending a SAW III Sampler (Thanks, Doctor Popualr!)
DIY Dub Siren for cheap