The Artist's Spex

I've been thinking about what could be done with mediated-reality glasses, i.e. ones that can edit your view of the world before you see it. As it happens, there are researchers developing what sound like some very suitable glasses very near me: in the Nuffield Department of Clinical Neurosciences, where Dr Stephen Hicks and his team have set up the Oxford Smart Specs Research Group.

Smart Specs use clever image processing to help people with very bad sight see better, as shown in the group of four images near the top of the Smart Specs page. The first image shows a lecture room in natural colour, with a view of some chairs and two men in the foreground, and a cupboard and door in the background. The following three images have been processed into monochrome, and show only the chairs and two men. Details have been thrown away, and contrast enhanced, making chairs and men much easier to see.

From an artistic point of view, the third image is the most interesting. It shows a very clear edge-detected image of one of the men, simplified so that only his hair, glasses, eyes, nostrils, mouth, chin-tip, T-shirt and trousers, and arms and hands (half in pockets) are visible. If you had a pair of Smart Specs and you wanted to cheat at urban sketching, you could do so very effectively by using the same algorithm to capture and render your subject in line.

But let's assume that you want to do the drawing yourself, rather than letting the computer do it for you. (Though perhaps, with kit such as this, as well as programs like the one Michael Haller, Florian Landerl, and Mark Billinghurst describe in "A Loose and Sketchy Approach in a Mediated Reality Environment", sketching will one day be as dead as John Henry's steel-driving.) How could Smart Specs help you learn to draw better? Here are some possibilities.

• Highlight regions of negative space. This is the space between objects. As I've mentioned in this blog already, paying attention to this space, rather than to the objects themselves, fools the brain into drawing more accurately. See, for example, Brad Kasten's "Once you master negative space drawing it will forever change the way you see the world" page. But you need to learn to "see" negative space, and a tool to point out instances in real scenes would be very helpful.

• Superimpose basic shapes. Here, you break objects down into simple two-dimensional shapes such as triangles and rectangles, and use these to create a framework for your drawing. There are some examples here at Bob and Phil's "Learn to Draw Whatever You Want" page. Once again, a tool that demonstrates instances in real scenes would be useful.

  The image processing would be harder than for negative space, because the former is an objective notion, whereas decomposition into basic shapes is subjective. You can argue about how much the body of a bird, say, must deviate from a triangle before it stops being useful to see it as one. But by inducing from example decompositions drawn by artists, it should be possible to define a best-match criterion.

Both negative space and basic shapes are techniques for seeing more accurately. So are the suggestions below, but in a rather different way. I don't think I'd call them "techniques" now: they're more like tools that could be applied either to the scene you're currently drawing, or to lots and lots of scenes that you can learn relationships from.

• Superimpose a rectangular grid, to help you judge angles and lengths. This can also be useful in composing a drawing, as explained in Russell Stutler's "Page 11. How to Sketch" page.

• Superimpose a horizontal rectangular grid, receding into the background, onto surfaces such as pavements and floors. This would provide a baseline for people's feet, and also show how distances parallel to the picture plane relate to distances perpendicular to it.

• Superimpse ellipses onto clockfaces, wheels, the tops of cups, to show how to draw circles in perspective.

• Superimpose plausible skull, skeleton, and muscles onto people and animals, to help understand their anatomy.

  This would require much much much more processing than the suggestions above, and I don't know whether it's feasible yet. But there certainly is work going on, as I found from Michael J. Black's presentation "Inferring 3D People from 2D Images".

• Label different types of crease in clothes. Creases are important in drawing clothes accurately. But also, even in an abbreviated way in cartoons, they're useful for giving information about people's poses. For example, is part of the sleeve round an arm stretched or not, and what direction do the creases show the arm to be pointing in. There are several different kinds of crease — see 'Types of Folds An Illustrated Tutorial from "Drawing People: How to Portray the Clothed Figure"' by Barbara Bradley — and it's important to use the right type for the situation. So it would be useful to have a tool that classifies creases in real-life scenes.

Now here's a different type of suggestion again. Not for seeing more accurately, but for "seeing as".

• Render reality in a particular style, in order to improve the translation from scene to style. Most lines in a line drawing don't exist in the scene, but have been introduced by the artist according to some set of conventions. In an analogue to sight-reading music, can you learn to look at a scene and "see" it in a particular artistic style? Russell Stutler suggests in "Page 11. How to Sketch" (under "Visualize the sketch on paper") that you can, and I've suggested a possible method here. You could do this on an ordinary computer, but running the rendering program on Smart Specs would enable users to immerse themselves in the results, perhaps making them much more effective.

And finally, here's one that would help with practising particular objects. When I'm in the street and I've just drawn a hand in a particular position, say, I often wish I could try drawing a slight variant. Not an hour or two later, when I can get back to reference books or computer, but immediately in order to help consolidate the original. So it would be great to have a program that can:

• See what I've just drawn, search Google's image bank for photos of variants, and display them in front of my eyes for me to practise on.

The Body as Regions in a Memory Palace

I've been trying a new memory method for fast sketching of people. I mentally divide their image into regions, and superimpose a number on each, trying to note the outline of the region and the most important lines in it. Then I walk through the regions in order, recalling the shape and contents of each.

Three things inspired this. The first was some passages in Joshua Foer's book Moonwalking with Einstein, in which he looks at the techniques that Ben Pridmore and other memorisation champions use to memorise such remarkable quantities of information. In Chapter Five, "The Memory Palace", Foer notes that the point of memory techniques is to transform the information we want to remember into a form that our brains were built for. And apparently, one of these is navigation. Foer quotes the Memory Grand Master Ed Cooke:

"The thing to understand, Josh, is that humans are very, very good at remembering spaces," Ed remarked from his perch on the boulder. "Just to give an example, if you are left alone for five minutes in someone else's house you've never visited before, and you're feeling energetic and nosy, think about how much of that house could be fixed in your memory in that brief period. You'd be able to learn not just where all the different rooms are and how they connect with each other, but their dimensions and decoration, the arrangement of their contents, and where the windows are. Without really noticing it, you'd remember the whereabouts of hundreds of objects and all sorts of dimensions that you wouldn't even notice yourself noticing. If you actually add up all that information, it's like the equivalent of a short novel. But we don't ever register that as being a memory achievement. Humans just gobble up spatial information."

This made me wonder whether I could improve my memory for the things I was drawing by thinking of them not as single and relatively small objects, but as landscapes that I wanted to navigate.

My second source of inspiration was the Oxford Centre for Human Brain Activity research on attention and memory that I blogged in "Attention restores forgotten items to visual short-term memory". As I mentioned in that posting, the research suggested to me that when sketching someone who I've only seen briefly, then about a second after seeing them, I should deliberately scan my remembered visual field. And I should pay attention to those places in it where I think the most salient details for my drawing are. The division into numbered regions that I'm writing about today prepares for that scan, and the numbers make sure that I scan everything.

The final inspiration was the idea of a memory palace. It's the main memorisation technique that Foer writes about, but was known long before him, dating at least as far back as the ancient Greeks and Romans. The idea is to first build a mental image of a palace or other building. Then become so thoroughly familiar with it that you can reliably walk through it, guaranteeing to visit all its rooms or landmarks in a fixed sequence without missing any. Once you've done this, your memory palace is ready for use. To memorise a list of items, mentally place one item in each room. There are lots of techniques for converting hard-to-visualise items such as abstract concepts into images that are easier to visualise, and for associating them with the room they're in. To recall a list, just walk through the palace and see what's in each room.

Here's a sketch made using the method described in my first paragraph. I think it worked well, given that I drew it entirely from memory, something I don't think I could have done otherwise.

It's a man who I was watching from inside Combibos, standing near a Saturday market stall being set up in Gloucester Green. I haven't kept a note of how many numbers I superimposed, but it would have been around eight. Two went on his head: one for the quiff, and one for the rest of his hair. One was for his face and neck, including prominent eyebrows and the different length sides of the V below the neck, plus the change in direction of the collar further up. One was for his further arm, and one for the torso and nearer arm: the amount of visible back to the right of his arm was important, and I may have used negative space to judge this. And probably two regions were for his legs, with special attention paid to position of feet and to the stretch creases along his nearer leg. To make the perspective right, the bottom of the nearer trouser leg should probably have been lower.

Granted, the drawing looks stiff. This is partly because I was being quite deliberate when I drew the lines, not thinking about how to change their character to reflect what I was drawing. So the man's hair was drawn with the same simple straight line as his sleeves and trouser legs, even though one would have had a fine texture, one would have been gently undulating, and one would (if denim, which statistically speaking, it probably was) have been full of inelegant wrinkles and bulges. My memory didn't extend to that level of detail.

In drawing this and other sketches, I was thinking almost entirely about numbered regions. Even though the idea was inspired partly by Foer's passage about landmarks and navigation, I wasn't changing mental contexts to think of this as navigation. Would that have helped?

I haven't objectively tested this memory technique, so although it seems to work, that could be self-delusion. If it does work, it needs practice. Part of that, I've found, is being able to superimpose a constellation of numbers in parallel, because there isn't always time to work through them one by one. I think it also needs some prior experience in drawing, so that you know which features of the subject will be most useful in helping the viewer "read" your sketch, and hence should be paid attention to.