Enhancing the Moment vs. Enhancing the Image Bank

When I used to go to the Caption Comics Festival, before the committee decided not to hold it in Oxford any more, one of the artists I enjoyed chatting with was Bib Edwards. Bib has passed away, but I still value his advice. He told me that to help me draw from imagination, I should sketch whenever I could, drawing everything I had the chance to. After four years, he said, I would have built up a reasonably sized image bank. And when I drew from imagination, I would subconsciously find myself using bits and pieces of the images from it.

This is a different use of memory from trying to remember a specific person (say) so that you can sketch them before they move away. I'm writing this partly to make that distinction. But I hope that psychology and neuroscience can help artists both enhance the moment and enhance their image banks.

While drawing from imagination is fun, it's more fun when you don't have to keep stopping to look up reference pictures for difficult poses, pieces of machinery, costumes. And, it's more fun when you don't have to stop and spend all day working it out. To see what I mean, consider people's arms. When you're drawing a cartoon, you can get away with drawing arms as tubes standing distinct from the body. I did that below, and the result is perfectly intelligible:

But now look at the real-life arms in this Wikipedia photo of a jersey:

The arm on the right is very slightly obscured by a fold of fabric from the jersey. Because of the bagging of the jersey, the one on the left has a characteristic double-curved dark line bounding the upper arm. When drawing more realistically, and indeed, even when drawing cartoons, it's good to be able to add such details. They inform about shape and pose, and they satisfy the brain, preventing the drawing from looking too schematic.

You can force such details into your brain through practice, practice, and more practice. And if that's the only way, that's what the artist must do. But it would be silly not to ask whether neuroscience can make the uptake of such information easier. For example, there's the phenomenon of flashbulb memory, whereby everyone remembers where they were when Princess Diana died. Surprise, importance, and emotion, it seems, make memories more vivid and less fragile. So could we hook into the flashbulb-memory mechanism in order to strengthen the memories to be added to the image bank? Perhaps by viewing the images to be recorded through smart glasses, and changing their colour or making them very bright. Perhaps chemically, by stimulating the "downstream" production of whichever neurotransmitters strengthen memory storage, if indeed that's how flashbulb memory works.

Or perhaps most of us already have such memories, and the problem is not storage but retrieval. That's hinted at by Allan Snyder's experiments on trans-cranial magnetic stimulation. A related question, though one which may apply to enhancing the moment only: does the brain contain an analogue map of the visual field? If so, is there any way to make it accessible so that one can draw directly from it? This would eliminate a lot of tedious angle- and proportion-judging.

One subgoal of such research should be to help artists consciously "see" the components of their image banks, so that they can mentally compose them, in full three-dimensional Technicolor glory.

Training Visualisation via Afterimages

There are several discussions on the Web about improving visual imagination and memory. I wrote about one of them in an earlier posting, "From 'I cannot mentally visualise' to 'god what a taste of fire'". The second part of that title comes from a comment by user "fritillary" who, explaining how she had revolutionised her own visualisation abilities, remarked:

It was months before I saw any results, but god what a taste of fire.

Another discussion that I've just found on lesswrong.com suggests a way to do this — namely, using afterimages. Actually, it's concerned with auditory "afterimages", but would it also work for visual memory?

The discussion is in a thread called "generalizing From One Example". (In case the site should disappear, I've archived this in the Wayback Machine.) The title refers to the "Typical Mind Fallacy", the belief that one's own mind is similar to everyone else's. According to user Yvain who started the thread, Francis Galton carried out a study on mental imagery, presumably the one written up in "Statistics of Mental Imagery", Mind, 5 (1880). Galton gave people detailed questionnaires about how well they could visualise. Some subjects couldn't form mental images at all, others had almost perfect imagery. But the ones who did assumed everyone did, and the ones who didn't assumed no-one did, to the point of supposing that anybody claiming they could image was lying. Thus Galton experienced the Typical Mind Fallacy. Most of the rest of the thread concerns applications of the Fallacy, including visualisation. The contributions that interest me today are by user "cousin_it", in this comment and this (archived here and here).

In the first comment, cousin_it wrote:

Regarding differences in mental imagery: only this winter did I really understand that good musicians have vivid aural imagination, while I couldn't hear any sounds in my head, period. Immediately after this realization I started exercising. By now I can hear complete monophonic melodies, and (on good days) imagine two notes sounding at the same time. Classically trained conductors can imagine a complete orchestral sound while reading sheet music. I don't see any reason why visual imagination can't be similarly trained.

Another user then asked how cousin_it practised this skill. Here's the reply:

The hardest part for me was the beginning, getting a toehold at any inner sound. Pick a note on the guitar - I started with D on the second string. Play it at a steady rhythm with rests, slowly fading away into nothing. (Might not be possible on the piano or other instruments.) At some moment the brain will start to "complete" the sound, even though by that point you're playing too softly to hear. Catch that feeling, expand on it. When you can "do" several different notes, try playing a simple melody and hearing it afterwards. After you're comfortable with that, try to hear a simple major scale without playing it immediately beforehand. Then work from unfamiliar sheet music without playing it - solfege-sing in your mind - by now I can do this quite easily. And so on.

One thing to investigate, I think, should be what happens in the brains of people who have learned to "complete". Does some part of the brain's state when hearing the sound persist once that sound is gone? Is there some kind of feedback that is keeping this going? Does the brain state of completers differ from that of people who have not learned to complete?

As far as applying the idea to vision goes, we need an analogue to fading notes. The obvious thing to try is fading simple shapes — perhaps lines or circles — on a computer screen. It would be a fairly simple research project to write suitable software and try it out on a range of subjects.

Not having such software, I looked around for an object I could try "fading". The pen pictured below (*) seemed a good choice:

It's in a translucent violently-magenta case, vivid enough to grab the attention, especially if direct light is shining through. It's actually quite a lot brighter than in the photo, as if softly glowing. I tried holding it horizontal in the middle of my visual field, then snatching it rapidly away along its own length while concentrating on retaining the image, doing this twenty times. I seemed to be able to maintain an afterimage for a short time, provided that I viewed the pen against a white background with no distractions, namely my ceiling. Is this worth practising? Of course, one has to beware of the N-ray effect.

By the way, I came across one advocate of afterimages for visualisation. It's Marko Martelli on his self-help site unchainmybrain.com . In his "How to Develop the Ability to Visualize Mental Imagery – From Scratch" (archived here), Marko suggests looking at a candle flame for ten to twenty seconds, then closing your eyes and trying to keep the afterimage for as long as possible. Describe the flame's colour, shape and other properties to yourself in words as fully as possible. Then open your eyes and repeat the exercise, for no more than five minutes. Practise regularly, Marko says, and you will see progress. Marko says that he developed this method after long practice, frustrated at not being able to visualise.

(*) Coincidentally, the pen was given to me at a discussion with Brookes University about applying sports science to drawing. But that's another topic.

The Semantics of Line Drawings I

[ The Semantics of Line Drawings II, Information Deformation Theory ]

I've just given a talk in Athens. Only virtually, unfortunately, because I do love the city. I spoke on The Semantics of Line Drawings. What are the different uses to which artists put lines? What kinds of information do they convey? And how can we express this information mathematically? We'll need to do that if we are to make computers draw and understand drawings.

I'd been invited by Project Thales: a group researching into the algebraic modelling of topological and computational structures. I've previously worked in this field in a very different context, that of making spreadsheets modular. But, these mathematical methods can be applied to a huge variety of different problems. One of these could be described, very generally, as the problem of how to integrate different points of view. For example, in urban design, how can you reconcile the very different vocabularies and modelling methods used by, for example, a transport planner, an expert on making streets safe from crime, and a designer who wants to minimise energy loss in houses? This is something that I and a colleague will be writing up later. I mention it here because it is, like the topic of this posting, an "art" problem. Algebraic modelling is very definitely applicable to art.

So let's get back to semantics: the way that languages convey meaning. For programming languages, and other artificial languages such as those used to describe integrated-circuit designs, this is vital for ensuring that your programs and machines do what they were intended to. Rather important for nuclear reactors and Google's driverless car, but not unimportant even for mobile phones, supermarket U-scans, and banks. (One might argue that the recession of 2008 could have been avoided had we had a decent semantics of bankers.)

The semantics of natural languages is more complicated, but needed for speech understanding, automatic translation, and searching. My interest in this is that there's a sense in which drawing is a language too. In some drawings, this may seem obvious. Some of the lines in a line drawing have a clear geometric interpretation — they denote the edges and "rims" of objects, namely the places where the line of sight grazes the object's surface. When projected onto the image, they form a silhouette. We now understand very well how to convert these lines back into a 3-D model: see the references I gave in this slide. So the meaning of these lines is clear.

But lines are used in many other ways. What are they, and can we formalise them?

Most people understand line drawings easily, and realise, for example, that the cross-hatched lines in the school-story picture below are not to be interpreted as lines, but as shadow:

Similarly, note the horizontal lines on the back of the leftmost animal below, and the vertical ones on the nose of the rightmost. These don't represent scars or other marks, but tell us about the orientation of their surfaces:

And in this drawing by Wilhelm Busch, the lines rising from the bathtub represent steam, even though steam does not have a definite boundary:

Here's another Busch drawing, from the well-known Max und Moritz strip in which two naughty boys torment the bourgeois townsfolk and eventually meet their doom in a baker's milling machine. In the picture, they have fallen into his dough:

I find those lines characterful, conveying a droopy, gloopy, effect. Not drippy: drooping is slower than dripping, with bigger drops. Which brings us on to cross-sensory association, the bouba/kiki effect, and asssociations such as those Paul Klee writes about in his Pedagogical Sketchbook. As you can see, I'm moving from the physical attributes that lines denote (object boundaries, light and shade, orientation, texture) to character, emotion, and mood.

I gave some examples of conveying these through line quality near the end of my talk, using excerpts from Leonard Doust's book A Manual on Caricature and Cartoon Drawing. The point is that artists do use such effects, so a semantics of line drawing must be able to handle them. One example is the drawing below:

Of the righthand figure, Doust says,

Please remember that it is not enough just to make a pattern or design. It must be in harmony in line and (or) tone with the character. This is, I think, patently obvious in Fig. 5. Surely a pun is not out of place when I say that here is a blockhead made of blocks. No imaginative sweeps nor complicated whirls as in Fig. 4, just plain solid chunks; and the whole, also a plain stolid design — heavy, with a sphinx-like stillness resulting perchance from extreme emptiness instead of, as in the Sphinx, extreme understanding.

A more extreme example is this one:

In a sense, this is as extreme as you can get, because it contains no recognisable objects whatsoever. The first design, Doust says, represents Speed. The second is Peace, and the third Intelligence. Whether or not you agree with him (I do), such drawings exist, so a formal semantics of drawing must be able to describe their meaning. How? I've hinted at that in my conclusion, and I'll write it up in a future posting.

(The first image is from The Greyfriars Holiday Annual, 1926. The second is from Lupo Alberto Collezione N. 7, 1993. The third and fourth are from Wilhelm Busch's Die schönsten Bildergeschichten für die Jugend, a 1960 edition presumably reprinted from Busch's late 19th century original. The fifth and sixth are from L. A. Doust's A Manual on Caricature and Cartoon Drawing, 1936.)