The Semantics of Line Drawings I

[ The Semantics of Line Drawings II, Information Deformation Theory ]

I've just given a talk in Athens. Only virtually, unfortunately, because I do love the city. I spoke on The Semantics of Line Drawings. What are the different uses to which artists put lines? What kinds of information do they convey? And how can we express this information mathematically? We'll need to do that if we are to make computers draw and understand drawings.

I'd been invited by Project Thales: a group researching into the algebraic modelling of topological and computational structures. I've previously worked in this field in a very different context, that of making spreadsheets modular. But, these mathematical methods can be applied to a huge variety of different problems. One of these could be described, very generally, as the problem of how to integrate different points of view. For example, in urban design, how can you reconcile the very different vocabularies and modelling methods used by, for example, a transport planner, an expert on making streets safe from crime, and a designer who wants to minimise energy loss in houses? This is something that I and a colleague will be writing up later. I mention it here because it is, like the topic of this posting, an "art" problem. Algebraic modelling is very definitely applicable to art.

So let's get back to semantics: the way that languages convey meaning. For programming languages, and other artificial languages such as those used to describe integrated-circuit designs, this is vital for ensuring that your programs and machines do what they were intended to. Rather important for nuclear reactors and Google's driverless car, but not unimportant even for mobile phones, supermarket U-scans, and banks. (One might argue that the recession of 2008 could have been avoided had we had a decent semantics of bankers.)

The semantics of natural languages is more complicated, but needed for speech understanding, automatic translation, and searching. My interest in this is that there's a sense in which drawing is a language too. In some drawings, this may seem obvious. Some of the lines in a line drawing have a clear geometric interpretation — they denote the edges and "rims" of objects, namely the places where the line of sight grazes the object's surface. When projected onto the image, they form a silhouette. We now understand very well how to convert these lines back into a 3-D model: see the references I gave in this slide. So the meaning of these lines is clear.

But lines are used in many other ways. What are they, and can we formalise them?

Most people understand line drawings easily, and realise, for example, that the cross-hatched lines in the school-story picture below are not to be interpreted as lines, but as shadow:

Similarly, note the horizontal lines on the back of the leftmost animal below, and the vertical ones on the nose of the rightmost. These don't represent scars or other marks, but tell us about the orientation of their surfaces:

And in this drawing by Wilhelm Busch, the lines rising from the bathtub represent steam, even though steam does not have a definite boundary:

Here's another Busch drawing, from the well-known Max und Moritz strip in which two naughty boys torment the bourgeois townsfolk and eventually meet their doom in a baker's milling machine. In the picture, they have fallen into his dough:

I find those lines characterful, conveying a droopy, gloopy, effect. Not drippy: drooping is slower than dripping, with bigger drops. Which brings us on to cross-sensory association, the bouba/kiki effect, and asssociations such as those Paul Klee writes about in his Pedagogical Sketchbook. As you can see, I'm moving from the physical attributes that lines denote (object boundaries, light and shade, orientation, texture) to character, emotion, and mood.

I gave some examples of conveying these through line quality near the end of my talk, using excerpts from Leonard Doust's book A Manual on Caricature and Cartoon Drawing. The point is that artists do use such effects, so a semantics of line drawing must be able to handle them. One example is the drawing below:

Of the righthand figure, Doust says,

Please remember that it is not enough just to make a pattern or design. It must be in harmony in line and (or) tone with the character. This is, I think, patently obvious in Fig. 5. Surely a pun is not out of place when I say that here is a blockhead made of blocks. No imaginative sweeps nor complicated whirls as in Fig. 4, just plain solid chunks; and the whole, also a plain stolid design — heavy, with a sphinx-like stillness resulting perchance from extreme emptiness instead of, as in the Sphinx, extreme understanding.

A more extreme example is this one:

In a sense, this is as extreme as you can get, because it contains no recognisable objects whatsoever. The first design, Doust says, represents Speed. The second is Peace, and the third Intelligence. Whether or not you agree with him (I do), such drawings exist, so a formal semantics of drawing must be able to describe their meaning. How? I've hinted at that in my conclusion, and I'll write it up in a future posting.

(The first image is from The Greyfriars Holiday Annual, 1926. The second is from Lupo Alberto Collezione N. 7, 1993. The third and fourth are from Wilhelm Busch's Die schönsten Bildergeschichten für die Jugend, a 1960 edition presumably reprinted from Busch's late 19th century original. The fifth and sixth are from L. A. Doust's A Manual on Caricature and Cartoon Drawing, 1936.)