Ask Slashdot has posed an interesting question looking for a cheap (under $50) word processor for banging out notes on the go. I think my favorite one is the $9 (Ebay) DreamWriter NTS 325. Seems like you can get the data off fairly easily too. Link.
Cheap and Portable Word Processors