It is often said that details are what make a fictional world feel real. I heartily agree - but it's not just any type of detail, nor is it every type of detail. Details are especially easy to get lost in. How do you get both breadth and depth in your world? How do you pick the details that really give it flavor? How do you avoid drowning a reader in details and losing track of a story?
Let me take this a bit at a time, because details should typically be approached a bit at a time, in layers (it should soon become evident exactly why).
I'll start with worldbuilding. Often, when building a world, you want to know as much as you can - about the setting, ecology, architecture, social structure, manners - really everything you can possibly think of. So start with the basic principles upon which the story premise will rest. Then from there, try to work those basic principles down onto the smaller, more intimate scale. Work from institutions down to family structure, to manners, to individual definitions of identity and personal crisis.
That's just the start, however. The details required for worldbuilding are more general and more nebulous by definition; they are restricted more to the level of general principle. It's hard to push down as deep as you need to when you're working with worldbuilding detail alone.
To go further, you need to start writing the story.
The story will help you understand which details you need, because it gives you intimacy in the form of characters, their homes and relationships. The identity of a point of view character will have a huge influence on what he or she notices - as a thief will notice the placement of exits and the worth of valuables, while a snobby fashion-conscious lady will notice the details of clothing and of objects in a room.
Details also contribute to the emotional impact of a scene. If the character feels frightened, you can certainly say so - but you can also include details like sharp edges or the shape of shadows, details that deliberately contribute to the emotional mood of the scene even if they aren't part of the character's conscious knowledge.
Furthermore, details can work for you in establishing a theme for your story. Aligning details so they are described in terms that fit the worldview that underlies your project is the way I often like to think about it. I like to line up objects my Varin characters notice, and give them similes that directly relate to the fact that they live underground (because I can't draw attention to it otherwise, since they consider it so normal). Another technique is to place a certain detail, say the scent of oranges - in spots where a certain emotional mood is being achieved, and thereby link those spots together even if they aren't obviously linked in other ways.
There's a special value in details that go unnoticed. Very often the details of manners and speech will be partly subconscious, so that readers won't track exactly what it is that was said, but they'll get a "feeling" about it. I recently wrote a pair of lines as follows:
Character 1: "The music - I don't know how you do it," he stammered. "Is it the blessing of Heile on the Kartunnen[caste]? What's your secret?"
Character 2: "I - well in fact, sir, I don't know why I can do it."
I don't know if you can tell, but perhaps you can, that Character 1 is of higher social status but feeling quite awkward about asking questions. Character 2 is of lower social status, but the fact that he doesn't precisely echo the phrasing "how you do it" is a hint that something else is going on (to be divulged later).
I personally feel that it's worth being very attentive to social cues in dialogue! This will not surprise those of you who know me.
Finally, there's great value in the occasional unexpected detail. When we think "room" it's easy to think "door" and "furniture," but more fun if you throw in "cast bronze door" or "brass furniture." If you establish "market," many people's minds will leap instantly to the aisles of their nearest Safeway, or to a medieval market with booths, carts and the occasional horse - but not necessarily to the overpowering smell of cheese, or to camels and baskets of dates. Have fun; confound the easy expectations and your world will start to pop.
As you go through life, keep your eye out for places where details surprise you. Those can provide you with good opportunities to surprise people in your story. It's probably not surprising to have a person, when moving from one country to another, notice a change in the looks of the people, or the type of houses, or the food. But look for the unexpected.
I noticed something in France while I was there - it hit me all of a sudden, though the details of it had been collecting in my head for some time. It was about strawberries, when you buy them in a box (as I have done) in different countries. A box of strawberries in France isn't quite like a box of strawberries in Japan, or in the US. It's strange, but true, even when the plastic boxes are the same shape. A box of French strawberries will typically be full of relatively small fruit of varying sizes, all of which are very sweet. A box of Japanese strawberries will contain all strawberries of precisely the same size, and usually the same level of sweetness as well. Strawberries in an American box will vary in size, with at least two or three being very large and shiny red, but not necessarily sweet. This is one of the oddest patterns I have ever noticed, I admit, and I'm not sure if I'll ever be able to use it in a story as such. However, I could easily imagine myself putting a similar thought into the head of a traveler, such as, "I miss the fruit of my homeland; it was smaller, but always sweet."
Details like this don't need to be piled on a reader. The most critical factor to consider when writing, as opposed to worldbuilding (where profusion can be good), is the relevance of any particular detail to the story. If you pick your details carefully, keeping them well-suited to the character, to the mood and to the story's overall purpose, you don't need many for a large impact, and your world will feel much more "real."