How is the eye's vision of a model transformed into the hand's work to produce a portrait? Film-maker and scientist Dr John Tchalenko led 'The Painter's Eye', a sciart project involving artist Humphrey Ocean and scientists from Oxford and the USA, to explore this question, and he looks back at what he learnt.

I first thought that making a portrait consisted of looking at the model and drawing the portrait, and that this entailed artistic creativity and was a quite mysterious process. I wanted to know more about how it was done, so I 'sat' for Humphrey Ocean as he painted Double-Portrait. In fact, 'sitting' as a model involved standing up for 12 days, so I learned the hard way that I had been off the mark by about 50 000 - the approximate number of times that Humphrey had looked at me, his model, and back at the canvas as he was painting.

But how was it done? How was the eye's vision of the model transformed into the hand's work? Finding nothing in the literature I set out to read around the subject. I discovered eye trackers, devices that fit on the head like a bicycle helmet which record very precisely where the eyes are looking, and motion sensors which can be used to record the hand's movements in space. And I found a sympathetic scientific unit - the University Laboratory of Physiology in Oxford - ready to help me. When the head of the laboratory, Professor John Stein, arrived at the Wellcome Trust where we were presenting our project for the sciart competition, he kept whispering to me: "Surely we are asking for £170 000, not £17 000?". 'The Painter's Eye' project was off the ground.

A roving eye

Together with Chris Miall from Oxford, we used the eye tracker intermittently on Humphrey as he was drawing Portrait of Nick. Later we did more eye-tracker tests with him, and even, thanks to Professor Robert Solso, an unscheduled fMRI [functional magnetic resonance imaging] brain scan in Stanford USA as Humphrey was sketching from photographs. From all these data we could conclude that Humphrey's eye movements at work were very different from his normal eye movements: while drawing, he made a sequence of regular single fixations, lasting about one second each, on selected details of the model's face, at a frequency of about 12 fixations per minute. Normally, fixations lasted about a third of a second, at a frequency of about 130 per minute.

From eye to hand

We could then combine these data with a frame-to-frame analysis of the eye tracker's scene camera recordings to see how the eye was coordinating with the hand. For example, the first line drawn for Portrait of Nick depicted the back edge of the upper eyelid of the right eye. It was about 5 cm long, concave downwards and with a changing curvature. It was drawn left-to-right with the pencil stopping twice on the paper's surface while the eye went back to look at the model.

This, I concluded, was the simplest case of the phenomenon of fading visual memory. The painter looks at the line while it is being drawn, and at the model while he is updating his visual memory. He can draw about 1.7 cm in about 2 seconds before needing to refresh his memory. This suggests the presence of a visual mental image acquired by the brain, held there for a short while and gradually fading away.

So I was beginning to build an answer to my initial question of how a visual perception of the model becomes an action, producing the picture on the canvas. The painter was using his eyes twice: first, to capture a visual detail, and second, to copy the mental image of the detail onto the paper. It is this process of acquisition and reproduction which seemed to be at the centre of artistic creativity in drawing.

However, as the Portrait of Nick progressed and acquired an ever-growing number of lines, the analysis became less clear. For example, the last-but-one line drawn for the portrait depicted the outer contour of the hair on the model's right side. It was 20 cm long, of gentle uniform curvature, and it was drawn in two strokes. For the second stroke, the eye first looked at the starting point, but then instead of following the pencil it looked at other parts of the drawing and at the model. The line was then reinforced very precisely, with, as before, the eye only coinciding with the pencil at the start. So the painter could draw up to 16 cm before referring back to the model, and he needed only to see the starting point. More importantly, about 10 seconds after having drawn such a line, and still without ever having seen it, he could retrace it perfectly without looking at it.

These and similar observations made during the latter part of the portrait suggested that a motor memory component now seemed to be involved in the drawing process, something we had not observed in the earlier stages. In both cases, the painter was using his eyes to capture the image he required, but now instructions were passed to his hand and executed on the paper without the use of the eyes. The eyes were focused elsewhere and were perceiving other details of the drawing or of the model.

So if a mental image existed, what was it of? The detail being drawn, or other parts of the drawing, or other parts of the model? Was there more than one mental image, or none at all, in which case the notion itself had to be revised? Was I back to invoking artistic creativity as a mysterious process?

There was a way out of the dilemma, which also made sense with several other observations made during the late stages of the portrait. Looking away from the drawing hand allowed for perception of the 'picture so far' to become an additional input into the drawing process. The more the picture progressed, the more the simple draw-pause to refresh-draw cycles was being perturbed by the influence of the emerging picture. Portrait drawing, at least for this painter, was a complex combination of a fading memory image and an increasing presence of the emerging picture. In the jargon of science papers, 'more research was required to elucidate this point'.

May be this is how science progresses: an initial mystery based on ignorance, then a discovery and learning stage, and a final mystery based on knowledge. Sounds a bit like art.

A version of this article appeared in the Wellcome News Science and Art supplement, 2002.

Dr John Tchalenko is at the Camberwell College of Arts, London.