I have been using a serial-port AVR programmer (the Olimex one you can buy from Sparkfun) to do AVR development, but my laptop doesn't have a serial port.
There are a number of USB-based AVR programmers, and they're not particularly expensive (e.g., the Atmel AVR-ISP mkII is $34 from Digikey), but being the cheapskate that I am I was looking for something really inexpensive.
I'm happy to report that I've found one that works very well, and is only $10:
I'm using it with Sparkfun's AVR breakout board to bring the programming signals onto a breadboard using a single-row 6-pin cable from Digilent. Here's a picture of the whole setup:
The programmer is the small green board on the left. The breakout board is the even smaller red board connected to the programmer at a right angle.
In the host OS, the programmer appears as a serial port (/dev/ttyUSB0 on my Debian laptop). The programmer type for avrdude is "stk500v2".
As an additional bonus, the programmer supplies 5V to your circuit, using the host USB port as the power source. I don't think it will source a lot of current, but it's certainly fine for a few ICs and LEDs. The circuit on the breadboard in the picture has an atmega8 and an LED.