Actually, you are - there is no world on a blank page/file until you put words there - and you can't possibly avoid it. Each word comes with a piece of world built into its connotations, its ins and outs. But I have met people who told me they were not worldbuilding for their stories.
So what happens when you think you are not worldbuilding, but you are anyway?
You get a glimpse into the subconscious structure of the fictional world inside your brain. Even if you are putting a single alien/fantasy made-up word on a page, it will still come with associations due to sound and due to similarity to existing words. Even then, if you take your awareness and make it conscious, you can start to play with people's expectations. What if you give your hero a horrible name?
Technology sets are another key issue. When we see a cell phone, we expect a computer. When we see one element that makes us think of a particular historical time period, our subconscious expectations fill in all kinds of other aspects of that time period. If you see the word King, you will tend to think medieval.
If you don't want readers to go in the expected direction, you have to block them, redirect them, and do it early. You have very little time to depart from the schemata - scenes/sets - that readers bring up before they will start feeling like you have betrayed them and their expectations.
This is why you often see people picking words carefully. Would you use King? If the situation is a little unexpected, might you use Majesty instead? Or Eminence? Brian mentioned Prince, which has an expected meaning of "king's son," but in fact is a ruling title in its own right in many real-world places. Picking words is really critical, because they bring important implications with them.
We talked about Anne McCaffrey's Dragonriders of Pern, which starts out looking quite medieval and then later turns out to be a world colonized by spacefaring humans. Many of us thought this gradual change was well done, but it doesn't work for everyone. People do drop out over a series.
In terms of climate, Che mentioned that it's very easy to write exactly what you experience.
Brian mentioned the problem Star Wars has with single-biome worlds (very unrealistic!).
There are also cultural expectations that sneak in, such as the idea that the "north" will be advanced and the tropics less advanced. These are colonial assumptions that we bring with us from our history.
I mentioned that I'd like to see an Inca fantasy. Brian said yes, that they would weave their way out of problems. Ropes and knots were their specialty, and super-sophisticated. He also mentioned floating gardens, and villages in the lakes of Bolivia. They had no plows, no animals they could ride, no enormous beasts of burden, but they were very technologically advanced.
We also talked about Etruscans, Romans, and Vikings, just to acknowledge that they were pretty advanced in their own ways. Roman concrete remains a mystery to this day because it was a family trade secret. Local materials in a place can vary widely. People indigenous to an area will optimize those materials, but colonists will typically try to import their own ways, even from a very different climate, and can run into problems. What if your home methods don't work?
One of the things that can enter into worldbuilding when we're not looking is our own history of reading science fiction or fantasy. Our expectations of fictional worlds are set by that which we have previously read. I mentioned how Aliette de Bodard described writing her sf/f in English, and said that it was easier for her because the sf/f she had read had been in English. The reading we do sets patterns in our minds that become easier to tread again.
Kimberly noted that sometimes no matter how hard we try to keep our worldbuilding conscious and specific, readers can overwrite it with their own preconceptions. We asked, "How do you put in enough?" The answer isn't clear, though, because different people will have a harder or easier time departing from their own schemata.
You can put evidence for difference into your story in many different ways. You can have outsider characters who are explicit guides to the world and can explain its rules (as with Breq in Ancillary Justice by Ann Leckie).
Another bias that often hides in fiction is racial bias. This topic could be an hour all by itself, but definitely watch out for it.
The best approach for an author is to try to make deliberate decisions to create expectations, defeat expectations, or re-derive a more standard expectation in a different way. Re-deriving is an interesting thing. Say you'd like to have a seven-day week in your world, with a two-day weekend, but you don't necessarily want to refer back to our world and the Norse (etc.) derivations of the weekday names. You have to find an alternate basis for the same system. Think through food production and when it occurs, and how that would affect the flow of life in your fictional region.
Brian pointed out that there are times when you can't work on food, such as during the far northern winters. Often you have a lot of intense time on followed by a long time off (hoping the food will last!). Summer vacations in the US are long because children had to be allowed to participate in the agricultural harvest. We joked about the pyramids of Egypt, but Brian pointed out that there have been big, fortified granaries before.
Brian told us about the origins of veal crates. They may seem cramped, but they originated in climates where there was snow, and for long periods, cows couldn't forage. In Switzerland, he explained, you could only keep the animals by bringing them inside on the ground floor of your house and feeding them with as much hay as you've managed to grow during the last season. The cows would give birth over winter before forage is available, so the calves live in a cramped, dark place until the weather allows them to go out.
Pigs are easier to grow than cows under the same conditions, which is why some populations rely more heavily on pig meat.
This topic is one that tends to lead us into a discussion of ways to depart from the subconscious default, and there are a ton of ways to do that! Thank you to everyone who attended.
Remember to join us tomorrow (Wednesday, 4/20/16) to talk with author Randy Henderson! I hope to see you there.