My AVR-fu is slowly improving.
I'm now using the AVR-PG1B programming cable I ordered from SparkFun, along with a DB9 cable and a breakout board that adapts the 10-pin AVR ISP header to a single-row 6 pin header, which then plugs into my breadboard with a cable and gender changer from Digilent. Very nice, works like a champ. (Note: avrdude calls this programmer "ponyser".)
I constructed a circuit interfacing the attiny2313 with an ICM7212 7-segment LED driver. So far, I only have 2 of the 4 supported digits wired, but it works nicely. Here's a photo (taken again with a very crappy camera) of the whole mess:
There's still plenty of room on the breadboard for another IC and a few switches, which are needed for my crazy top-secret project.