Person A: "But really good writers get to break the rules all the time."
Person B: "You're just saying that because you don't know how to write."
This saddens me every time I see it. Usually person A isn't actually saying that because they're trying to cut corners, but because they've heard this refrain about breaking rules. If you've been writing for any significant length of time, you've heard this - just as you've heard the rules about adverbs and "show don't tell," etc. At the same time, person B is often not trying to attack person A, but to defend the idea behind the rules.
So why are the rules there? Can they be broken?
When these questions come up, I'm extra glad of my experience in Pragmatics - because this is be a perfect time to talk about H. P. Grice's Cooperative Principle. This principle says, "make your contribution to the conversation optimally relevant and appropriate." As I've remarked before, this may seem terribly obvious. However, it is quite powerful, because the assumption of cooperativeness allows us to draw conclusions about other speakers (and in our case, other writers).
When you assume that the writer is being maximally cooperative, you're essentially relying on two things (both of which we assume about people in speech all the time):
1. The writer knows the rules
2. When the writer breaks the rules, he/she breaks them intentionally.
This is to say that following grammatical rules establishes a default value for language - a way of using language in which everything flows and nothing stands out or gets noticed. Consequently, breaking those rules causes a marked state, a state in which things stand out as having possible hidden/additional meaning.
Compare the workings of grammar, point of view and story structure to a room with which you're very familiar. You leave the room in a particular (default) state, and have a basic memory of that state when you return to the room. When you return, if something has been moved in the room, a. you notice it and b. you draw the conclusion that someone or something has moved that thing. Depending on what has been moved and how, you can easily conclude that someone opened the window and the wind blew some papers, that your husband picked up the envelope you left for him, that your cat knocked over the plant, or that you've been robbed.
To quote from my 2006 IROSF article on Point of View:
"The voice of a narrator is usually so transparent that we feel it without needing to analyze it. [...] But then, every so often, we run across a sentence like the opening of Alfred Bester's "Fondly Fahrenheit" (Science Fiction 101, p.100, emphasis added):
He doesn't know which one of us I am these days, but they know one truth.
Suddenly, the words used for point of view are not only visible but provocative. "What is he doing?" we ask. "Four different pronouns in the same sentence?" Then quickly we move on from our initial surprise to ask, "What special situation or alternate reality can this signify?""Readers - and I'd think, particularly readers of science fiction and fantasy - will be willing to do a lot of deduction to figure out precisely what special situation is being described. Personal taste is involved, of course. Some readers find certain departures from the norm bothersome, while some don't notice those much and object to others. However, to keep reading, readers must feel a level of trust for the writer. If a departure from the rules violates that trust, perhaps through making some change that the reader finds particularly irksome, or by using language loosely so that some grammatical changes don't paint a consistent picture of an alternate situation, then the reader will stop reading.
All right, so what about those famous folks we all know about who violate all kinds of rules (grammar, point of view, structure, etc.) in inconsistent ways - yet people read their books voraciously anyway?
I'd say there are a few core elements that have to be in place grammatically - like, say, that it's really convenient to be able to identify who your characters are and keep track of who is doing what, etc. Beyond those really fundamental confusion-inducing principles, though, the kinds of rule violations that are a matter of taste can potentially be outweighed by the content of the story. Usually that has something to do with the situation, the characters in the situation, who they are and what is at stake. When the story content is exceptional, a lot of readers will ignore faults of the writing. But not all of them - take my husband, who is not a writer but is a voracious reader and was absolutely furious by the time he finished The DaVinci Code. Okay, yes, he finished it, but he's unlikely to pick up another book by Dan Brown. I personally picked up Twilight to see what it was all about (because something about the story had to be exceptional) and couldn't stand it long enough to get past page 3.
The fact is, I love a good story. But I love a good story well written even more.
If you're a writer out there wondering about rules and whether to follow them, you're asking the wrong question. The question should instead be what it means to follow them, and what it means when you break them for specific effect.
Language is a marvel. Its patterns are complex, and multilayered. I discover things constantly that put me in awe of its beauty and complexity, as well as its flexibility as a tool. If writing is your passion, and you want to make it your life, it's worth the time and effort to explore what it can really do.
P.S. Since I wrote this post, Janice Hardy has put up a companion piece dealing with story, that other critical aspect of writing - and the part that tends to make people ignore problems of grammar etc. Go here to check it out!