MELANCHOLY404 TECHNOLOGY

It is 6.06am and I’m typing this in my pyjamas. I awoke at 6.04am, walked from the bedroom to the study, switched on my computer and got to work immediately. This is unusual behaviour for me. However, it’s a tried and tested technique for enhancing creativity, long used by writers, poets and others, including the inventor Benjamin Franklin. And psychology research appears to back this up, providing an explanation for why we might be at our most creative when our minds are still emerging from the realm of sleep.

The best evidence we have of our mental state when we're asleep is that strange phenomenon called dreaming. Much remains unknown about dreams, but one thing that is certain is that they are weird. Also listening to other people's dreams can be deadly boring. They go on and on about how they were on a train, but it wasn't a train, it was a dinner party, and their brother was there, as well as a girl they haven't spoken to since they were nine, and... yawn. To the dreamer this all seems very important and somehow connected. To the rest of us it sounds like nonsense, and tedious nonsense at that.

Yet these bizarre monologues do highlight an interesting aspect of the dream world: the creation of connections between things that didn't seem connected before. When you think about it, this isn't too unlike a description of what creative people do in their work – connecting ideas and concepts that nobody thought to connect before in a way that appears to make sense.

No wonder some people value the immediate, post-sleep, dreamlike mental state – known as sleep inertia or the hypnopompic state – so highly. It allows them to infuse their waking, directed thoughts with a dusting of dreamworld magic. Later in the day, waking consciousness assumes complete control, which is a good thing as it allows us to go about our day evaluating situations, making plans, pursuing goals and dealing rationally with the world. Life would be challenging indeed if we were constantly hallucinating, believing the impossible or losing sense of what we were doing like we do when we're dreaming. But perhaps the rational grip of daytime consciousness can at times be too strong, especially if your work could benefit from the feckless, distractible, inconsistent, manic, but sometimes inspired nature of its rebellious sleepy twin.

Scientific methods – by necessity methodical and precise – might not seem the best of tools for investigating sleep consciousness. Yet in 2007 Matthew Walker, now of the University of California at Berkeley, and colleagues carried out a study that helps illustrate the power of sleep to foster unusual connections, or “remote associates” as psychologists call them.

Under the inference

Subjects were presented with pairs of six abstract patterns A, B, C, D, E and F. Through trial and error they were taught the basics of a hierarchy, which dictated they should select A over B, B over C, C over D, D over E, and E over F. The researchers called these the “premise pairs”. While participants learnt these during their training period, they were not explicitly taught that because A was better than B, and B better than C, that they should infer A to be better than C, for example. This hidden order implied relationships, described by Walker as “inference pairs”, were designed to mimic the remote associates that drive creativity.

Participants who were tested 20 minutes after training got 90% of premise pairs but only around 50% of inference pairs right – the same fraction you or I would get if we went into the task without any training and just guessed.

Those tested 12 hours after training again got 90% for the premise pairs, but 75% of inference pairs, showing the extra time had allowed the nature of the connections and hidden order to become clearer in their minds.

But the real success of the experiment was a contrast in the performances of one group trained in the morning and then re-tested 12 hours later in the evening, and another group trained in the evening and brought back for testing the following morning after having slept. Both did equally well in tests of the premise pairs. The researchers defined inferences that required understanding of two premise relationships as easy, and those that required three or more as hard. So, for example, A being better than C, was labelled as easy because it required participants to remember that A was better than B and B was better than C. However understanding that A was better than D meant recalling A was better than B, B better than C, and C better than D, and so was defined as hard.

When it came to the harder inferences, people who had a night's sleep between training and testing got a startling 93% correct, whereas those who'd been busy all day only got 70%.

The experiment illustrates that combining what we know to generate new insights requires time, something that many might have guessed. Perhaps more revealingly it also shows the power of sleep in building remote associations. Making the links between pieces of information that our daytime rational minds see as separate seems to be easiest when we're offline, drifting through the dreamworld.

It is this function of sleep that might also explain why those first moments upon waking can be among our most creative. Dreams may seem weird, but just because they don’t make sense to your rational waking consciousness doesn’t make them purposeless. I was at my keyboard two minutes after waking up in an effort to harness some dreamworld creativity and help me write this column – memories of dreams involving trying to rob a bank with my old chemistry teacher, and playing tennis with a racket made of spaghetti, still tinging the edges of my consciousness.