If you're looking to build fantasy terrain on the cheap, then here's some supplies to get you started:
1) Foam Cutter - this thing is worth its weight in gold. It's basically just a big wire that gets heated up by a pair of batteries in the handle, and cuts through foam like butter. Practice with it and you'll figure out how to get some cool rock-like effects going on your hills and cliffs.
2) Carpenter's Glue - this stuff is worlds better than the white glue that kids use for crafts. It dries harder, waters more evenly, and is generally just better to use. Get a cheap paint brush to spread it on, and just remember to wash it really well before the glue dries and ruins it.
3) Sand - you can buy this in bulk from your hardware or garden store, which is really where you should be buying most of your scenery supplies if you're on the cheap. Hobby stores are great but everything there is very expensive. Just be aware that you'll probably have to buy WAY more sand than you'll ever need. You can also just head down to the beach and get sand for free

Glue the sand down with watered carpenter's glue, though be aware that it's super messy.
Alternatively, you can head to your local arts store and pick up some pummice gel. It's more expensive, but this stuff is fething amazing. It's a gel medium filled with little bits of pummice stone in whatever level of granularity you wish to buy. You can even mix paint into it to save you from having to base-coat. More expensive than the sand+glue method, but it's easier, cleaner, and more resilient. This stuff dries rock hard and is practically invincible.
4) Foam - You CAN reuse packing foam that you have lying around. Frankly though I wouldn't recommend it. It's really fragile, and terrain generally gets handled pretty roughly, so it won't last long. If you are using cheap styrofoam then make sure to seal the surface with carpenter's glue after you shape it to protect the surface. Otherwise just buy some foam from the hardware store. You're looking for something thick and sturdy (don't worry, your foam cutter will still go through it like a hot knife through butter). They're usually inch-thick sheets, and the stuff we always buy is blue. Sorry that I can't be of better help.
To build hills just glue sheets together to get roughly the shape you want, then go at it with the foam cutter.
6) Wire - wire is good for everything, but most importantly as a base to sculpt or anchor other things on. You can make trees by making a frame out of wire then wrapping the wire in masking tape, then covering it in glue or putty.
7) Balsa wood - it's fragile, but great. You can make all sorts of things out of balsa wood.
8) Bark - painted grey, bark looks surprisingly like rock. Especially if you shave it down just a little bit.
9) Spray Paint - specifically an acrylic spray paint like Krylon H20. NORMAL SPRAY PAINT MELTS FOAM. You don't want to spend all that time shaping your foam scenery only to destroy it with spray paint. Krylon h20 is safe for foam, and is probably the best undercoat you're going to find. Otherwise you can use regular hardware store paint and just apply it with a brush, though this is a pain in the ass and doesn't always adhere well to foam.
10) Plasticard, or styrene sheets - this stuff is a godsend for modellers. You can texture it, cut it to whatever shape you want, and the world is your oyster. It's kind of expensive though so use it only for smaller stuff.