Here, I've drawn up a new picture explaining the fan thing better. I'm not an artist, so piss off if you don't like it
Ignore the yellow wire in the fan plug entirely. Solder the negative wire from the black slot on the fan plug to the negative lead on the battery. This can be either the battery itself (not recommended), or a 9v battery holder. The simplest ones are little flaps that connect to the top of the 9V with a red and black wire coming out. Black is negative, red is positive. Simple enough.
Solder the black from the battert to the black of the battery connector...then solder a wire from the red of the plug to the switch, then from the switch to the positive side (red) of the battery