Surprise! or Surprise?

Surprise is the emotional response to an unplanned event. It may equally well be the reaction to an event that was planned in advance but did not play out as anticipated.  In either case the resident emotion balances on the fulcrum of a business-as-usual stasis which may accordingly tip either to disappointment and dismay or elation and enjoyment.

There is no middle ground in surprise; a person, thing, or event is either more than you'd hoped for or less.  Getting what you want may come as a surprise, particularly if you'd despaired of achieving the goal, in which case the result is better than you'd expected (because you'd either written it off or put it on hold.)

As long as you are dealing with absolutes, you might as well make the point again:  a narrative without some direct or implied goal is not yet a story; every character of significance has some relevant goal to justify her/his inclusion in the first place.  Dorothy Gale's goal in The Wizard of Oz is to get back to Kansas.  Think how frequently she was frustrated from this goal during the arc of the story.  Think also that the moment she was back in Kansas, the story was over.  Thus the use of surprise in story assumes accelerating importance; when a character believes she has the stated objective in sight, a diversionary event (surprise) adds frustration, disappointment, and longevity to the story.

Surprise may have nuance, even irony, witness the iconic Somerset Maugham story from the early 1930s:"The Appointment in Samarra"

The speaker is Death

There was a merchant in Bagdad who sent his servant to market to buy provisions and in a little while the servant came back, white and trembling, and said, Master, just now when I was in the marketplace I was jostled by a woman in the crowd and when I turned I saw it was Death that jostled me.  She looked at me and made a threatening gesture,  now, lend me your horse, and I will ride away from this city and avoid my fate.  I will go to Samarra and there Death will not find me.  The merchant lent him his horse, and the servant mounted it, and he dug his spurs in its flanks and as fast as the horse could gallop he went.  Then the merchant went down to the marketplace and he saw me standing in the crowd and he came to me and said, Why did you make a threating getsture to my servant when you saw him this morning?  That was not a threatening gesture, I said, it was only a start of surprise.  I was astonished to see him in Bagdad, for I had an appointment with him tonight in Samarra.

Think about the emotional effect not only of the characters in the story but on yourself, having read it. Some readers will have articulated at least to themselves the role surprise plays in their approach to fiction, reckoning somewhere in their reading process the role surprise plays in a given story.  Writers understand that reversals and disappointments experienced by their characters help bridge the often impenetrable gap between character and reader.  How many persons do you know who have never experienced a reversal or setback?  Even though it is a part of the human genome to plan for such events, how many of us has gone through any significant span of time in which there were no disasters, disappointments, or need for Plan B?  It is not a moment too soon to recall Robert Burns' poem about the poor, unfortunate mouse whose nest he inadvertently uprooted while plowing a field.  "The best laid schemes o' mice and men/Gang aft agley..." or as some of the remarkable late nineteenth- and early twentieth-century Yiddish writers have noted, "Tsouris beliden,"  Trouble aboundeth.

Happy surprises are best watched with caution lest they seem a deliberate convenience; they should not be allowed to lead the reader to the conclusion or outcome without some careful fore-planning to make the reader accept the plausibility.  Thus it is always better to conflate happy surprises with, say, a discovered open zipper on a man or a visible bra strap for a woman, some embarrassment however tiny, recognition that even good things come with hooks.