Are you plugged into the front panel audio? If so, sometimes the cable that goes to the front runs across the video card and it picks up audible feedback. Repositioning the cable may fix it.
Other times, it's just a cheap cable that also shares cabling with the USB ports, and you will hear feedback whenever there is USB activity. Is that the case?
Do you hear this noise when you plug into the back port?
It's also possible the case is grounded imperfectly and there is a short somewhere. The next step would be to remove the motherboard from the case, set it up on some cardboard, plug in the video card\ram\PSU, and plug some headphones into the audio jack. If there is no noise, look inside the case and see what might be shorting it. Did you use standoffs on the motherboard tray? Is the tray warped?
You may want to post this on
Tom's Hardware instead of Dakka Dakka, Heresy Online, and Warseer. The advice you get here will range from well-meaning and benign to utter clownshoes, but it's a good place to ask about miniature stuff (go figure!). I linked the motherboard forum since I'm presuming you have a built in Realtek chip and not a standalone sound card.
You might want to just cut to
getting a standalone sound card and seeing if that works, by the way: they're pretty damn cheap and I'd rather spend $20 than infinite hours fiddling with things. They give better audio anyway. I have more money than free time, so this solution may not work for you.