So say the creators of Amnesia: The Dark Descent. "Monsters? Our level designs will scare you enough to make you cry for mommy. Our monsters? They're the back up. You know, in case you're playing the game in a brightly lit room with Disney music playing in the background. You wimp."
When will my life begiiiiin?
(P.S. Enjoy this, it's the happiest picture in the post.)
Don't get me wrong, I love monsters. So, I think, do the makers of Amnesia - they certainly poured enough effort into them to make them terrifying.
Everyone with me now: AAAAAHHH!
But that doesn't mean they rely on them. They do the same thing writers should do: they set the scene. You don't have to be subtle about it, either.
Fear is an interesting emotion. It's involuntary, but it originates in the brain, not the heart, the stomach, or the gut. So the reader has to realize that the scenario is frightening mentally, and you, as the writer, have to pull this off in a way that doesn't break immersion. Sound hard?
Amnesia is not subtle about the fact that you're supposed to be frightened. For starters, it takes place in an abandoned castle. It's still the scariest game I've ever played.
Subtle, guys. Reeeeal subtle.
But notice how there is no villain in the above picture? Not a single one. So why is it scary? How do you add the terror without the bad guy?
A) Your hero notices that something is... off. Look at the picture above - what's wrong with that picture? It's an abandoned castle, falling to ruin. Maybe a little creepy. But what's with the bridges? Hastily made, falling apart. Lit candles? Does someone still live here? Why would someone still be here? There's no visible ceiling, caverns under the narrow strips of floor. Who designed this building?
Something is wrong here.
B) The environment itself is dangerous. I mean, really. Look at that bridge.
C) Add a little light. Darkness is darkness, it's fright is in the mystery, but we're all used to that mystery. You can hear things in the dark, and the MC can dismiss it as imagination. We dismiss things we hear and smell easier than we dismiss things we see - especially things we think we see. Give just enough light to remove all doubt that something is wrong.
Those three things help, but if you really want to racket the tension, add a safe place. Yes, a security blanket. Anything that would give your MC comfort - it could be a warm room, a comforting friend, even a gun. Something that, when it's around (or your MC is around it), the reader know he stands a fighting chance. A Gandalf, if you will.
Then mess with it. Just a little.
About a quarter of the way through Amnesia, you enter a sanctuary. It's light, it has a fountain, there's sunshine through the windows. There are three doors leading to three different sections of the game, so this is a sort of gathering place. It's safe.
The player creeps into the darkness of the first room, is in danger, terrified, but completes the puzzle. Runs like hell for the sanctuary.
Everything is safe. *phew*
Next room. Repeat. Back in Sanctuary. But wait - what's that? Is that... a body in the fountain?
And BAM, safe place is not safe anymore. Someone's been here. It's tainted. And your character can't use that self-reliance and desperate confidence he needs to survive, because he never got it. He had safe room. Gandalf-friend. A gun.
The adventure begins again.
And that, my friends, is how Amnesia scares your socks off.