Data for Emotion

Angela Ferraiolo

Data is often seen as a static entity, a body of information that waits to be sorted, classified, visualized and presented. Along these lines, our digital environments, the ‘data spaces’ we inhabit, are largely built on computational structures that process information according to predetermined, emotionally neutral rules. But the human experience of data and data systems is often deeply emotional, creating a tension between the affective experience of data and the procedural nature of computation. In an increasingly ‘datafied’ world, a more systemic view of data becomes interesting to artists. The dynamic relationship between data and emotion is an area that offers the potential for artistic and theoretical explorations.

My recent artistic research looked at political speech as a means of bridging the gap between static and dynamic views of data. Rather than presenting the meaning of the text through the well-known expressive lens (word frequency, tag clouds, etc) of data visualization, I explored a recent presidential address to congress, by tying the sentiment analysis of a text to the computational rules of an emergent system, allowing data to act as a transformative force that alters computational behavior.

This research draws on concepts from cellular automata theory, affective computing, and generative art. The idea was to research a system that uses sentiment analysis to score a political text for emotional content over multiple dimensions, then translates these emotional scores into specific computational parameters. The system continues by applying those parameters to modify both the rule structure and visual properties of a cellular automaton. Finally, this system creates a feedback loop where the automaton’s behavior influences subsequent emotional processing.

This research asks how data can function beyond the quantitative to use data not as a descriptive body of facts, but as a generative force that produces new computational realities.

Affective computing was first formalized by Rosalind Picard (1997) and explores how computers can recognize, interpret, and process human emotions. The field evolved rapidly with advances in natural language processing and machine learning which enables increasingly sophisticated sentiment analysis of text.

An important aspect of affective computing, sentiment analysis involves computational schemes for analyzing the emotional tone or feeling behind written text, usually asking whether someone’s speech is positive, negative, or neutral in affect. Sentiment analysis is not unlike teaching a computer to read between the lines the same way humans do. When you read a movie review that says “This film was absolutely amazing!” you can tell the person loved it. Or if someone writes “Worst customer service ever,” you know they’re upset. Sentiment analysis helps computers recognize emotional cues automatically. In analyzing a text, sentiment technologies look for clues like specific words (great, terrible, love, hate), or punctuation (lots of exclamation points!, etc). Sentiment analysis can also read context to make a best guess about how someone is feeling when they wrote or said something.

Contemporary sentiment analysis systems extend beyond simple positive/negative polarity to identify specific emotional dimensions. Two of the tools used in this research were VADER (Hutto & Gilbert, 2014) and NRC Emotion Lexicon (Mohammad & Turney, 2013). These tools allow a multi-dimensional emotional analysis of text and establish frameworks for quantifying complex affective states.

More recently, continuous cellular automata such as Smooth Life (Rafler, 2011) have extended these concepts into systems with gradual state transitions rather than binary (onn/off, alive/dead) conditions. These continuous models provide more visual possibilities and more nuanced rule systems and can produce organic, flowing patterns reminiscent of biological systems. Artists like Sage Jenson have utilized these continuous CAs to create evolving digital landscapes that blur the line between computational and natural systems.

However, much of this work treats data as content to be visualized rather than as a structural element that reshapes computational systems themselves. More relevant to this approach is the work of artists like Memo Akten, whose project ‘Learning to See’ uses machine learning models to transform visual input, effectively altering the computational ‘seeing’ process itself. The artist Casey Reas has also explored how simple rule changes in computational systems can yield dramatically different visual outcomes, though typically without incorporating external emotional data. Sound artists like Robert Henke have developed systems where the emotional content of music influences visual patterns, creating affective feedback loops between sound, image, and audience. The approach used in this research intends to go further by suggesting how emotional states might reshape computational processes rather than simply representing them.

• Emotional Computation System: Translates emotional scores into specific computational parameters governing the cellular automaton.
• Sentiment Analysis Module: Processes text input using the VADER sentiment analyzer and NRCLex emotion lexicon to generate multi-dimensional emotional scores.
• Rule Mutation System: Applies different rule structures based on dominant emotional states.
• Visualization Layer: Renders the cellular automaton with visual properties that reflect both the emotional states and the emergent computational behavior.
• Feedback Loop: Captures the cellular automaton’s activity level and feeds it back into the emotional processing system.

• Neutrality (measured via trust scores)
• Cruelty (measured via anger scores)
• Suspicion (measured via fear scores)
• Deception (measured via disgust scores)

Each emotional dimension influenced these parameters differently. Neutrality stabilized parameters near their baseline values, creating balanced, structured patterns. Cruelty widened the birth-death threshold gap and increased neighborhood size, producing aggressive expansion. Suspicion narrowed thresholds and created oscillatory behavior, leading to unstable, fluctuating patterns. Deception introduced asymmetric influences and hidden ‘fault lines’. The dominant emotional state also triggered different computational rule structures which are a continued area of exploration in the ongoing research of this idea.

• Neutral State: Created balanced, structured patterns with stable propagation and regular spacing. Structures tended toward symmetry and predictable evolution, forming stable “gliders” and “pulsars” reminiscent of Conway’s Game of Life but with smooth transitions.
• Cruelty State: Generated aggressive expansion patterns with rapid colonization of space. The wider birth window and erosion of isolated cells created competing fronts that collided and generated complex boundary formations. The visual effect was one of territorial contestation and dynamic struggle.
• Cruelty State: Generated aggressive expansion patterns with rapid colonization of space. The wider birth window and erosion of isolated cells created competing fronts that collided and generated complex boundary formations. The visual effect was one of territorial contestation and dynamic struggle.
• Deception State: Created apparently stable formations that unexpectedly fractured along hidden fault lines. The micro-faults inserted by the rule system remained dormant until data conditions triggered their activation, resulting in sudden pattern collapses that were not predictable from the surface appearance.

• Emotional Resonance: Certain text inputs created emotional profiles that produce highly active cellular patterns, which in turn reinforced those emotional states through feedback.
• Emotional Damping: Other texts produced profiles that gradually diminished cellular activity, creating a negative feedback loop that eventually stabilized at a lower energy state.
• Cyclical Behaviors: Some combinations of text and initial conditions produced oscillating systems that cycled between different emotional states and activity levels.