Paintings really can be heard, scientist says

Sept. 7, 2006
Courtesy University College London
and World Science staff

The Rus­sian art­ist Was­si­ly Kan­din­sky was­n’t talk­ing non­sense when he claimed his paint­ings could be heard, a sci­ent­ist says. In fact, he adds, we all link sound and col­or men­tal­ly at some le­vel—and tend to do so in con­sist­ent ways, which art­ists can exploit.

Kandin­sky's "Composition VIII, 1923," in the Sol­o­mon R. Gug­gen­heim Mu­se­um, New York.

Some peo­ple con­scious­ly rea­lize the cross­o­ver of sen­ses in their brains, said the re­search­er, Ja­mie Ward of Uni­ver­si­ty Col­lege Lon­don. These in­di­vi­du­als have a rare con­di­tion in which the senses min­gle, called syn­aes­the­sia.

They may real­ly feel they hear paint­ings, for in­s­tance. But the con­di­tion does­n’t oc­cur on­ly with sound and sight: the most bi­zarre forms of it have been re­ported.

A stu­dy in the Aug. 22 is­sue of the re­search jour­nal Con­scious­ness and Cog­ni­tion, for ex­am­ple, found that some peo­ple link time and space. One de­s­cribed De­cem­ber as a red ar­e­a lo­cat­ed at arm’s length to the left of their bod­y.

Ward detailed find­ings of his own new stu­dies on syn­aes­the­sia at a talk at the Bri­t­ish As­so­ci­a­tion for the Ad­vance­ment of Sci­ence Fes­ti­val of Sci­ence in Nor­wich, U.K. this week. His work fo­cused on the sound-col­or-link­age of which Kan­din­sky spoke.

His found that most of us tend to agree with syn­aes­thetes on which im­ages match which sounds, and that we pre­fer them com­bined rath­er than alone.

Kan­din­sky, who lived from 1866 to 1944, “wanted to make vis­u­al art more like mu­sic—more ab­strac­t,” Ward said. “He al­so hoped that his paint­ings would be ‘heard’ by his au­di­ences. This seems more achiev­a­ble now that we have found such a strong link be­tween vi­sion and hear­ing.”

“Although in­for­ma­tion from the world en­ters our heads vi­a dif­fer­ent sen­so­ry or­gans—the eyes and ears in this in­stance—once they are in the brain they are in­ti­mate­ly con­nect­ed with each oth­er. Im­pres­sive­ly, they are con­nect­ed in non-random ways, so that some com­bi­na­tions of sound and vi­sion go to­geth­er bet­ter than oth­ers.”

In ex­per­i­ments, Ward said he asked six synaes­thetes to draw and de­scribe their vis­u­al ex­pe­ri­ences of mu­sic from the New Lon­don Or­ches­tra. Six non-syn­aes­thetes were asked to do the same. Also, an an­i­ma­tor crea­ted films com­bin­ing the mu­sic and drawn im­ages. These were shown to vi­si­tors at Lon­don’s Sci­ence Mu­se­um.

Furthermore, 100 im­ages were presented to over 200 peo­ple. They were asked to choose the im­age that best fit the mu­sic. They con­sist­ently chose im­ages drawn by synaes­thetes over others, Ward re­ported.

This shows, he said, that while non-syn­aes­thetes can’t hear a paint­ing or see mu­sic lit­er­ally, they do sense the cross­o­ver and tend to choose the “cor­rect” im­age. “All of us have links be­tween our hear­ing and vi­sion—even if we don’t really realise it,” Ward said.

That’s not to say that synaes­thetes hear pre­cise­ly the same sounds in “lis­ten­ing” to a Kan­din­sky paint­ing.

De­scrib­ing the art­ist’s “Com­po­si­tion VIII, 1923,” Ward re­ported, one synaes­thete said: “The jum­bled mass of lines gave var­i­ous tones, which changed as my eyes trav­elled round the pic­ture. When look­ing at the large mul­ti­coloured pow­er­ful cir­cle at up­per left, I get a pure tone which can be too much, so to re­lieve my mind of this I trav­el back to the ca­coph­o­ny of jum­bled lines and shapes. This paint­ing there­fore is a good bal­ance of con­trast­ing noise—pure tones and ca­coph­o­ny—which was a de­light to see.”

Anoth­er de­scribed it as fol­lows: “There is a huge splurge of sound left-hand top—boom­ing and vul­gar! Be­low it is a mousy lit­tle meee sound which then trans­lates in­to ‘o­h’s and ‘ah’s and pops at the var­i­ous cir­cles. The lines are sharp and are mov­ing to the right with the sound of steel—like blades scrap­ing against one anoth­er. The tri­an­gle and boom­er­ang shape are sur­prised and pop up laugh­ing with a ‘whooo’.”

The next stage of the re­search will use brain scans to mon­i­tor the brains of synaes­thetes when Kan­din­sky trig­gers sound or when sound trig­gers a vi­sion, Ward said.