Projects / Sonifying Shakespeare

While instructing at the NKS summer schoolwww.wolframscience.com, I helped students from diverse backgrounds to implement their ideas in Mathematica. One of the my favorite projects was working with the artist Elizabeth Latta to visualize and computationally explore the famous Shakespeare play The Tempestweb.archive.org.

Among the many promising ideas we investigated, two in particular were interesting enough that I’d like to show them off. The beautiful drawings are Elizabeth’s work, and I planned and wrote the code.

The first was a technique that used a network of the major characters in the play to indicate the relative importance of their interactions. In this visualization, each character is a node in the network and each interaction is an edge. To be precise, the thickness of the gray bond between two characters represents how often they speak lines in the presence of one another. The size of each portrait indicates the total number of lines each player has. So main characters should appear large, and have strong connections to their frequent stagefellows.

The second technique isn’t quite a visualization, its an “auditization”. The idea is that each character is assigned a note, and the play is simply, um, played — every time that character speaks, a note is played. A note is sustained for an amount of time proportional to the number of lines he or she is saying. One last tweak is that I use Mathematica’s pattern matching to ensure that long streams of repetitive, boring note patterns are elided somewhat. The general effect is passable music, or at least quite different from most algorithmic music. Take a listen! Just remember, you’re listening to the play at about 30x speed!

One of the things that I think makes this idea work is that a play must already conform to a particular grammatic, scene, and dialogue structure — a structure that leaves traces on the way the players share the stage. The rhythm of back and forth between two sparring characters cannot be too lop-sided. Multiple characters have to be orchestrated carefully to move the plot forward. Scene changes must permute the characters if there is to be any extended tension. Characters and motifs recur throughout the play is plans are hatched and executed. All of these forces, and others, make the non-local structure of the dialogue interesting — simultaneously familiar, and unpredictable — the very same properties a score must have if it is to be interesting to the ear.

I’m going to try this idea out on other Shakespeare plays, and other plays in general, to see if I can discern major differences in the music they can generate.