frankly I love the little baby Korg Monotron synths. They have 3 or 4 different models. And they are all very grungy and the exact opposite of synth pop. It's almost impossible to play in TTET on the membrane keypad so there is that atonal thing going for it.
A couple smaller synths from Yamaha are the DX100 which is kind of a cult synth for some ravers or something, but there are a million of them. If you get a DX patch editor you can create some very messed up sounds.
Also the CS01 which is one of the most basic analog synths possible. It's about the size of a casiotone but is an analog monosynth.
I've gotten a lot of mileage out of the Roland SH09. I picked up mine for $500 USD. I think it may have some vintage collectorness to it. But is probably overshadowed by the hipper looking SH101 or SH7 which has more features. It's got a lot of neat things it does, but mostly I like that you can plug the headphone into the aux input and make it feedback through the filter.
It also has a suboctave, which is nice.
The Casio SK1 and SK5 are sampling keyboards. They are also very small units with mini keys.
I don't see them as coveted as they were a while ago, so the price should have come down. And the sampling is very poor quality. I made literally dozens of recordings with these and really like the way the cheesy presets sound through guitar pedals. It's also great fun to record percussive sounds and loop them. Then you can make poly-rhythms by holding down different intervals.
Beatnigs used to use the SK1, and I've seen a few other acts do pretty well with them.