So, the trick really is building a good dark mood without causing distractions. Any 'mood' type things you come up with are going to need to be carefully balanced. Music too loud makes it hard to hear. People jumping out of closets could be great for a 'horror peak', but overused it gets boring, not to mention that recovering from the shock of that happening would probably help actually draw people
out of the game. It reinforces that what you want people to actually concentrate on is only mental; you know, forces people to stop suspending their disbelief momentarily, and then readjust. Even mood lighting overdone just makes it hard to read dice/character sheets.
The blacklight idea could be cool. I've never tried that before.
I use music so much that I've decided that there is only one specific way to do it, and it requires some setup. First, you need to select your music; it should be something dark and ambient. No (well, little) vocals, leave your screaming metal music at the door. Pick up some Midnight Syndicate or some very dark piano music. Think Moonlight Sonata. For something a little more action packed, think Helloween. Gregorian chanting is also always good.
Once you've made your music selection, you need a subtle means of starting the music, and a proper sound system. Tinny music from laptop speakers on a game table that your players watch you click on isn't mood. Remember, presentation is important. Test the music at the proper volume for the room. Barely audible is preferable. You want people, about 10 minutes in, to just suddenly realize that there's music playing and be unsure of when it actually started.
That's creepy. When I use music, I set up my phone to connect to Winamp on my computer, and just queue the playlist before the game, and hit play on the remote control on my phone under the table when I'm ready to kick it off. Midnight Syndicate's "The 13th Hour" album has a fantastic track of nothing but crickets and somebody walking around on a dirt path. It's an excellent starter, and it just gets better from there.
Something else I like is props. If my guys find written clues, I like to actually make up the letters/documents they find, either hand written in my shakey cursive, or printed in a script style font from a computer. You can 'age' the paper by wiping the paper down with a wet teabag, and then crumping/unfolding the paper a few times. Maybe clip some of the edges to make it look a little torn. Fake blood stains perhaps for the extreme. Obviously, do this AFTER it's been through the printer. If you're willing to go to an absolute extreme, get some sealing wax and a cheap stamp and go to town. I made one of my players hold on to a sealed letter he was supposed to deliver for about 3 months, in real life. It was really getting to him by the end of it.
Voices help a lot too, if you can sustain them. Most of our stuff takes place in Eastern European countries, so we all use voices that are generically "Russianish". It's cheesy, but people get into it. It's fun.
You also have to remember that you need the right people for this kind of game. These kinds of things are VERY
RP intensive, and if you have people who can't/won't get into that, then it's not going to be as good.
I agree with a lot of what MDS says, but I'd be careful about trying to intentionally frustrate your players too much. You don't want to actually make them what to stop playing. That's been done to me in a few 'intrigue' games I've been invited to. There can only be so many red herrings and jumps in logic before people just don't want to play anymore. If you can pull it off though, it could be awesome!