Morning coffee science

When art meets neuroscience - the story of Synesthesia

It is the Autumn of 1911. Around this time of the year, Munich gets breezy, cold, and wet. During the night-time, you most certainly want to stay by the furnace with a hot drink in your hands. As the night comes, the streets of the downtown are light up. The nightlife of pre-war Munich starts. At the center of it lies the Munich concerto hall.

A crowd of gentlemen and ladies is slowly moving to the concerto house. Tonight is a big night because an Austrian composer by the name of Arnold Shoenberg is playing. Somewhere in the crowd is a 45-year old Russian artist Wassily Kandinsky. Dressed more modestly than the rest of the intelligentsia that attend such cultural events, he nevertheless does not feel out of place and is excited to see the performance. As he enters the concerto hall he is taken aback. It is packed. The atmosphere is humming with chatter and the smoke of tobacco drifting upwards towards the ceiling. As he settles into his seat he is ecstatic - he will finally see the famous leader of the Vienesse school that he has heard so much about.

The concerto blows Kandinsky away. Wasilly feels it all - the romantic motifs, the harmonical atonalities, and melodic progressions. More importantly than anything else, his vision becomes a little more blurred and patchy, a little more obscured by a lingering essence of yellow... Wassily feels like his eyes have a lively yellow filter, an occasional yellow mist that covers his eyes. During the performance it sometimes intensifies, then becomes more patchy, then again lingers at the edges of his field of vision - but it never leaves the periphery of his view. After the performance, Kandinsky rushes home and feverishly puts the image from his mind on a canvas. Tirelessly he works to produce something that will come to change his artistic trajectory. For the first time in his career, he no longer cares about representing real life. He wants to turn the abstract idea in his head into a physical representation. After 16-hours of artistic labor Kandinsky is content. He titles his new piece "Impression III (Concert)".

This account is partly a figment of my own imagination. I wrote this in 2021, at my working desk at Singapore. None really knows what Kandinsky felt when he listened to Shoenberg. However, he did attend Shoenberg's concert in 1911 and he did feel inspired by it. The yellow visual evoked by Shoenberg's musical performances lead to him painting a piece called "Impression III (Concert)" (see below).

The moral of the story is not about Kandinsky's art, Shoenberg's music or the socio-economic stratification of pre-war Munich. Rather, it is about the reason why Kandinsky saw color when he listened to music. In neuroscience, we call this phenomenon synesthesia. This condition can be understood as a "union of the senses". In the case of Kandinsky, his perception of sound was accompanied by visuals. This specific type of synesthesia is called chromesthesia.

Lots of great artists had synesthesia: Vincent Van Gogh, Mikalojus Konstantinas Čiurlionis, Duke Ellington, Vladimir Nabokov, Franz Liszt, Jean Sibelius to name a few. Even currently known artists like Lady Gaga, Pharell Williams, Kanye West, Frank Ocean - all hint of having certain "unions of senses". Undeniably, a lot of them would proclaim that the "union of senses" shaped their art but also allowed them to be "better" artists. One of the reasons why Edward Kennedy "Duke" Ellington was so good at music is because he had a pitch-perfect hearing. He would often say that seeing the sounds in colors allowed him to easily tell apart different tones.

Not everyone who has synesthesia has chromesthesia (sound → color). For example, Vladimir Nabokov, the author of the infamous novel "Lolita", would see certain letters as being colored. Different combinations of words and sentences would give different color patterns that would inform him how to structure his thoughts better. This type of synesthesia is called Grapheme-color synesthesia. Interestingly, there are about 80 different types of synesthesia. Examples include synesthesia where flavor gives rise to color, where vision evokes sensations of touch, and where hearing feels like being touched at specific parts of your body.

The vast variety of different synesthesia types means that a common neural mechanism is very difficult to pinpoint. In this blog, I will be covering the neural explanation of the most common type of synesthesia, Grapheme-color (letters/digits → color). As with the other blogs, a true explanation is always more difficult than it seems and the explanations that my undergraduate brain can comprehend are usually crude simplifications. Regardless, let's delve into the neuroscience of synesthesia.

By far the most popular theory to explain synesthesia is the "cross-activation" theory. The main idea behind this hypothesis is that synesthetic sensations come about as a result of increased cross-talk between brain regions specialized for different functions. For example, the additive experience of grapheme-color synesthesia (letters/digits → color) would come about as a result of cross-activation of brain regions responsible for letter/digit recognition and the areas specialized in color recognition (called V4) (see picture below). Researchers hypothesize that the dual activation of different sensory regions could be due to a neurodevelopmental deficiency in neuronal pruning.

Neuronal pruning is a phenomenon of neuronal elimination - our brains have to become more selective about the world that surrounds us so that it can "work" better for us. Why do we need to eliminate neurons? Isn't that going to decrease our brain?

To put it simply - when we are born, our brains are "very very packed". Infant's brains have about 100 billion neurons while the adult brain has only about 15% of that. As we learn and grow, circuits and synapses that prove to be most relevant are strengthened while others are weakened or eliminated. Selective removal of "not useful" neuronal components is key to having a healthy and adaptive brain. As the human brain matures, the V4 region (color) and the grapheme region (digits/letters) are supposed to be "separated" by neuronal pruning as that would have been desirable for survival in ancient times. In synesthesia, "separation" is hindered and sensory areas that should not be connected are still connected even as the infant's brain develops.

Evidence for the "cross-talking" hypothesis mainly comes from functional neural imaging studies. Functional neuroimaging refers to the method of analyzing the movement of oxygenated blood in human brains when they are performing certain tasks. The exact physics of neural imaging are beyond the scope of this blog post, but the main point is that when a specific area of the brain becomes more active, it also starts to require more blood to "fuel" its activity. The flow of blood can be captured with machines called magnetic resonance imaging (MRI) machines and can then be analyzed with computers. Functional neural imaging studies have been investigating synesthesia. Here is such a study would look like: a patient would be put in an imaging machine and be shown certain stimuli. In this case, it was a paragraph that the participant in the machine has to read. Two groups were made - those with grapheme→color synesthesia and those without. When people with synesthesia read a paragraph, the color region (V4) was more active than when the same task was performed by people without synesthesia (Hubbard et al. 2005a). This finding clearly shows that there is a correlation between a synesthetic experience and the dual activation of two different sensory brain regions.

*Regions thought to be cross-activated in grapheme-color synesthesia (green=grapheme recognition area, red=V4 color area).

It is important, to note that even if certain types of synesthesia can be explained by the "cross-talking" hypothesis, others are more difficult to explain. Some sensory regions are far away from each other and the dual activation is difficult to explain by neuronal pruning of adjacent brain regions. The neural basis of synesthesia is still somewhat of a mystery for neuroscientists and is an active area of research.

A common thing people recently diagnosed with synesthesia says is that they did not know that it is not "normal" to have "unified senses". The truth is that about 4% of the global population experience some form of synesthesia. Some scientists even believe that there is a spectrum of synesthesia. For example, when most people hear nails on a chalkboard they involuntarily cringe. Cases like these can potentially be interpreted as cases of "weak synesthesia". Moreover, there is quite a bit of evidence that one can also induce synesthesia to an adult brain. It has been commonly observed in the medical literature that synesthesia can be induced by head injuries like temporal lobe epilepsy, head trauma, stroke, or brain tumors.

An interesting idea is floating around that synesthesia can be induced by chemical agents such as LSD, mescaline, or even marijuana. Maybe ordinary non-synesthetes like you or me could also experience this "unification of senses"? Maybe chemically induced synesthesia could enter the art scene and create new art forms and concepts that would change the way we see the world? Or maybe it already has... it is just that we haven't been aware that these influences have a neuroscientific term for it - Synesthesia.


The story at the beginning of this week's blogpost is both fictional and slightly ahistorical. It would be wrong to say that Shoenberg was a famous artist back in 1911 and that crowds flocked to see him perform. At the time Shoenberg was a struggling artist and I used this story to add dramatism to the story.

It is also not entirely true that seeing the concert was the driving factor behind Kandinsky's change to abstract art. The discussion as to what inspired Kandinsky to make abstract art is both complex and too long for this piece to delve into. Watch a video on synesthesia here.

The article was prepared by Matas Vitkauskas on behalf of INA

Sources: neurological phenomenon Kandinsky experienced,another sense%2C such as sight. details see Hubbard et,(2005a) Image sources:

Kandinsky's later art: Impression III (Concert):