Sounding the Net: Interview with Jason Freeman

Peter Traub: In N.A.G., the sounds that come off the Gnutella network are not altered or affected much except for looping and time/pitch shifting (i.e., no comb filters or other electro-acousticky type effects). In having the application create this chaotic collage, what were the salient features of the found sounds that you wanted to come through?

Jason Freeman: Part of my motivation in writing N.A.G. was in exploring the serendipity of the search process -- the pleasure of accidental discovery and juxtaposition. You search for "madonna" and end up not only with "Borderline," but also with "Lady Madonna." You search for "beethoven" and end up not only with Beethoven's 5th, but also with a techno remix of it. You search for "solitude" and get a strange mix of duke ellington and the soothing sounds of the ocean.

PT: The serendipity thing seems to be important in a number of net-based works. Amy Alexander's Multicultural Recycler, N.A.G., and some of my net-based pieces all rely on the serendipity of chance crossings of sounds or images as an important factor in the work. I'm curious if your interest in the serendipitous moment (a.k.a. happy accident), applies to your other non-network compositions? Do you think the network pushes artists in the direction of using serendipity/happy accidents just by semi-chaotic nature?

JF: That's an interesting question. I think that serendipity plays a role in just about everything I write, but it manifests itself in a very different way in pieces like N.A.G. than in my instrumental music. When I'm "through-composing" instrumental music, chance relationships between sounds play themselves out during the compositional process, not during the performance. For instance, in _Prior Art_, there is this strange relationship between the piece and the first three notes of the overture to Verdi's La Forza Del Destino. It's not a connection that I recognized as I wrote the first draft of the music, and it's not a connection I'd expect most listeners to pick up on. But it's something that I eventually noticed while writing the piece, and it helped me see what I had written from a new perspective and make some major revisions. I'm convinced that the piece would have turned out quite differently if I hadn't made this chance association. On the other hand, serendipity in N.A.G. plays a role not so much in the development of the software (i.e. "composition") as in the moment of its use (i.e. "performance"). (I think I've explained this enough in my previous e-mail so I won't give any new examples here.)

As for the second part of your question, I don't think it pushes artists in this direction, but it does perhaps encourage artists who are already predisposed to this direction. One way of thinking about the chance crossings / serendipity / whatever is as constructing a network which connects disparate content together by mere virtue of juxtaposition. That network of associations doesn't need to exist on a computer network, for sure, but when computer networks offer huge databases of pre-existing content, it provides a great environment in which to do such things.

PT: Making the process transparent to the user seems to be something that is meaningful to you and important to the work. I noticed a similar approach in your discussion of 'Glimmer'. Were there processes that you would have liked to initiate in N.A.G. or particular features you would have liked to make transparent to the user but were unable to do so due to technical/technological limitations? I realize this is kind of a broadly scoped question as it can be applied to almost any piece of art or technology-based art.

JF: I would have loved to do a visualization of the network to complement the auralization, showing what music was coming from which remote machines, how much data was being transmitted from each, and where those machines were geographically located. There are visualization projects somewhat akin to this (e.g. minitasking), but it was something I didn't have the CPU cycles (not to mention the development time) to implement.

In terms of the audio, I would have loved to add an option to phase vocode the music instead of just changing the playback speed, but again, CPU usage was a problem, and this was also tricky to do in the technical environment I'd chosen (Quicktime for Java). The phase vocoding might have helped keep source material more recognizable by keeping its pitch constant while stretching or compressing it in time.


PT: In terms of the limitations that you mention above and those that may be imposed by the environment (slow CPUs, limited bandwidth, etc) in creating a piece of networked software art that will work on a large number of platforms and systems, how do you think those limitations affect your art? It would seem that because of these limitations, certain ideas and processes gain primacy, and because they gain primacy within the world of online art (because most people have to consider them), they become closely associated with the aesthetic of online art. I'm curious if there are aspects to your work or the work of others that you would fit into that description, and if so, how do you think about them when considering the work?

JF: Hmmm...the examples I gave before with N.A.G. are really more about pushing against the limits of CPUs than of networks, and as such may be less relevant to your article.

But the first thing that comes to mind with your question is latency. It's certainly something we had to deal with in Auracle, and most real-time online works have to deal with it in some way or another. I'm hesitant to say that it contributes to a certain aesthetic, because there are so many different ways to deal with latency. In Auracle, we try to spread out user gestures to minimize their overlap, since it's impossible for users to synchronize their gestures with each other. In WebDrum (Phil Burk) and DaisyPhone (Bryan-Kinns), a loop proceeds according to an internal clock, with user modifications processed as soon as they are received -- so latency does not "disrupt" musical time. Or in many live multi-side performance works like The Technophobe and the Madman, there is deliberately no sense of common meter, so that latency will not cause tremendous problems.

What I'm trying to say is that what relates these works to each other is a common problem, not a common solution. If you are writing for orchestra these days, you must account for extremely limited rehearsal time in your writing. If you are writing for a brass quintet performing in a hall with a 27-second reverberation time, you must account for that in your writing. And if you are writing a networked piece, you must account for latency in your writing. But there are an endless variety of creative ways to do so.

PT: How did you think about the various intersections of control between you as the composer/programmer and the end user manipulating the choices that you provided them? What do you consider the most important compositional choices you made in developing the piece?

JF: With N.A.G., I wanted to let each user decide how active to be in controlling the software. At one extreme, a user can check a box to have the program create searches by pulling keywords from network traffic, leave all other settings at their defaults, and let the software run unassisted indefinitely. On the other hand, a user can manually select which query results are downloaded and constantly tweak all the parameters available in the GUI.

The most important compositional choices were deciding where and to what extent to allow users to modify algorithm parameters. In essence, these decisions defined the boundaries of an interactive space for the piece. For example, I decided that I would retain algorithmic control over playback position within each file and prioritization of files for playback -- these were too essential to the concept of the work for me to allow them to be changed. On the other hand, it didn't really matter to me how many files were playing at once, so I left this up to the user to decide.

There's a good example of this whole issue with another (non-network) piece of mine, MetaMix. Someone once wrote to me asking me about changing a parameter to allow for a wider range of values. The modification was trivial, but I felt that it would allow things to happen which were outside of my conception of the work, so I didn't want to change it. As it turns out, this man was able to do some Java programming himself, so he downloaded the source code (available under GPL) and made the change himself; I helped him out a little bit, pointing him to the section of the code he needed to tweak. He got to do what he wanted, and I got to keep the work the way I wanted, so everyone was happy. But the whole thing still feels a little strange -- both that I was so controlling about this and that this user was so committed to pushing beyond the limits of the space.

PT: You mentioned your 'control-freak' nature in your discussion when you were here, and I thought it funny because I think very similarly about myself in relation to my work. Even when control freaks try to give up control, we seem to be very interested in controlling exactly what control is given up and how. Perhaps this can be said of contemporary composers in general. It sounds as though, with MetaMix, you had created a program that was a composition in itself, and that the user wanted to use it as a tool for his own compositions. So there is this very fine line between a program that is a work and program that is a tool, and that the crossing of the line is dependent in some way on how much control the composer is willing to give up. Would you agree that networks seem to be an almost ideal space for playing with that line of control in a way that was previously unavailable to artists? How do you think about that line the broader context of your work?

JF: I think software art is an ideal space for playing with that line of control. The Internet comes into play in this respect mostly as a distribution medium, particularly as one which can recontextualize the content it distributes. Some people download my software art from my web site, where it is introduced to them in a manner which I control. But far more people discover this software through shareware/freeware web sites (e.g. versiontracker, download.com, SMM), blogs, articles, etc., most of which link directly to the download and not to my page. Someone who finds my stuff on download.com is probably going to have very different expectations about what it is and what they can do with it than someone who finds it through my web site.

In a broader context, I guess it gets back to what I was saying last week at UVa -- I play with this kind of control in my relationship with performers in my instrumental music as well. I suppose you could think of an ensemble of musicians as a kind of network, if you want -- each "node" is a player that receives data in their ears, processes it, and sends data back out through their instrument. In a "client-server" approach (i.e. fully-notated score and conductor), the importance of the network is de-emphasized, the nodes just execute the instructions they are given. In a "peer to peer" approach (i.e. free improvisation), the network itself becomes much more important. I usually opt for something in between, so I can take advantage of a little bit of what both paradigms have to offer. (I hope I didn't push this metaphor too far...)


PT: no you didn't. i actually think that's a very intersting way of looking at performers and various types of performance.

JF: I already talked about the importance of serendipity which the network makes possible with N.A.G. I really wanted to tap into the fascination with network searches that I (and just about everyone else) had upon first discovering the world wide web. But since the web is dominated by commercial outfits these days, and I don't ever spend time "surfing" so that I can marvel over the fact that some guy made a whole site dedicated to chicken parmesan, I turned to a network (Gnutella) which is not dominated by commercial interests, and which doesn't have as effective search tools, as a way to revisit the excitement of serendipity in network searching -- where the process is a goal in and of itself.

With Auracle, I think the network's allure is different. In one sense, it is simply a practical way to bring people together into a (virtual) space who are in geographically disparate locations. But I think the anonymous nature of the network is also critical to this project. We wanted people to be comfortable in being creative even if they had no musical training. When users log in -- with only their username and geographical location known to other players -- they have the privacy they need to be confident in what they do.

PT: With Auracle, I like this idea of a network of developers/artists working on a network-based music application. Did you and the other developers have a lot of input into the aesthetic/compositional choices made in the application, and if so, how did the collaborative effort work.

JF: Yes, we absolutely did have input. There were certainly contentious issues, but we were usually able to talk through them and reach a consensus.

A lot of the challenge with the collaboration was bridging the geographical (and time zone) divides: we had people in Germany, Italy, Arizona, and California working on the project. So we used a variety of tools to facilitate ongoing discussions -- mailing lists and a Wiki (both for aesthetic discussions and large-scale plans) and project management software, a bug-tracking database, cvs, etc. for technical collaborations and day-to-day details.

But our most important collaborative tool was actually Auracle itself. Once the bare-bones architecture was complete (including text chat), we started having twice-weekly "developer jams" on the current build followed by Internet phone conferences. Some weeks we were inevitably focused on bugs and technical issues, but many of these meetings were devoted to purely aesthetic discussions. It was an important way for us to routinely step back and evaluate things and decide where to head next.

PT: No real question here, just an observation that in the process of developing a networked application for collaboration, the application eventaully became the conduit for the collaboration in its own creation. There's quite a nice relational feedback loop that happens there. I wonder if this commonly happens in the development of network-based collaborative applications?

JF: I imagine it must, but I don't know first-hand of other examples.

PT: There are a lot of people out there right now working in the 'network-x' domain artistically, creating music, animation, interactive pieces, etc. that all don't just use the network as a medium but seem to be about the network itself. This is asking for a bit of prediction on your part, but I'm wondering what you see as the future of networks as a subject? Are they dense and complex enough that they will continue to be a subject of works for a long time to come, or do you think this fascination with the idea of the network itself will eventually just give way to works in which the network is the essential medium but not really a subject of the work (much like speakers aren't really the subject of a tape piece, but yet are necessary for its existence)?

JF: I'm having trouble answering this question as is, so I'm going to answer a slightly different one. I am fascinated by networks because they are "dense and complex" (which many of them are), but speakers and amplifiers are complex systems too. What differentiates networks for me is not their complexity but rather their fallibility. I am attracted to networks which fail to accomplish the goals they have set out: they are often unreliable or inefficient or inaccurate in the tasks they perform. And it's the chaotic, unpredictable process by which they sometimes utterly fail, sometimes triumphantly succeed, and often fall somewhere in between, that makes them interesting to me. (Think of Tom Johnson's Failing as a kind of instrumental parallel.)

PT: What do you consider interesting failures of the network in N.A.G.? When you search for 'madonna' and get 'lady madonna', do you consider that a failure on the network's end or the user's (for being too unspecific)? Or is is the fallibility thing further below the surface, in the way the traffic stops and starts, stalls, goes too slow, etc.? I'm wondering if you can give me some specific examples of how the fallibility aspects that interest you play out in N.A.G. or Auracle.

JF: It would be hard for me to fault the end user on this, because the query interface doesn't really allow the user to get too specific -- there's no advanced searching based on tag type (e.g. artist, album, year, etc.); I suppose that's my interest in pushing the "serendipity" aspect and taking some control away from the user. But even if that level of control existed, there are tons of mislabeled things out there (e.g. Fur Elise by Bach is something I've found several times).

For the second part of your question, I'll give you two examples. One of the curious things about Gnutella is that as democratizing a technology as it is, it actually is tremendously biased towards content which is already very popular (and therefore hosted by many different clients on the network). N.A.G. reflects this; since more popular content is more likely to show up on a search results list and more likely to be downloaded successfully, it is more prominent in the mix. But eventually, something strange happens, which I'm indebted to Thomas Goldstrasz for pointing out to me. (If your German is good, check out http://netzspannung.org/cat/servlet/CatServlet/$files/273049/goldstrasz_I.pdf.) Basically, the most popular songs download _quickly_, and then they're done. That leaves us with just the "background" to focus on, the music which is less popular, coming through less reliably, more fragmentary, etc. For Thomas, that's where the software starts to actually get interesting.


PT: This is a really interesting phenomenon, as it points to this somewhat hybrid mixes of a audio filter and an informational filter. Much like particular frequencies are reinforced in a comb filter, or removed in a notch filter, particular popular pieces of data effectively cancel themselves out sooner in your piece, emphasizing the less popular pieces of data (and this is represented sonically). Is this a fair metaphor? Is it a metaphor you could possibly draw out even farther?

JF: Yes, I think your metaphor is apt, though I'm not sure what I could do to extend it further. Instead, I'll just note that this filtering, to an extent, mirrors the way we actually hear music in our lives. Most "Top 40" hits saturate the radio airwaves for a short amount of time and then sink into oblivion. Other music takes a longer time to spread, but it tends to stick around longer.


Example 2: What would it have meant for us to have "succeeded" at the analysis / resynthesis task in Auracle? In one sense, success would have been recreating the original sound from the analysis data we send over the network. But that's absurd; if we did that, then we'd have just created yet another audio conferencing app. For it to become something more, we needed to fail at the task of transmission, sending an insanely small amount of data across the network so that only the most essential features of the audio remained in tact in their resynthesis.

PT: What happens when the technologies are no longer emerging but have become standard, and the level of fallibility drops (like landline phone networks for example - they're still fallible, but not nearly to the same degree as computer networks and most people know how to use them). Would the network become a less interesting medium and resource for you in that case?

JF: As a focus of the work itself (rather than just a practical means of dissemination), yes, they'd become less interesting. But there's always something new and not quite ready for prime time on the horizon...

PT: I'll leave it at just these questions for now and let your responses dictate where we go from here. However, for this last question, I'm simply asking if there is anything particularly important or relevant about your work with networks or the projects I mentioned that I have not asked about (with respect to the topic of my paper of course). If so, please feel free to chat about it or suggest a direction for me to go in with future questions.

JF: Yeah, one more thing. Metcalfe's Law: the usefulness of a network equals the square of its number of users. Does this apply to net art / sound? I think about this particularly in relation to Auracle, where we've as of yet failed to attain a critical mass such that users are likely to encounter other players to "jam" with when they go online.

PT: What do you think is preventing Auracle from acheiving that critical mass? Lack of publicity or that it just hasn't been around long enough? It would be nice to get a group of people together for a jam session. Have you thought at all about linking Auracle into an already existing social networking site like Friendster or Orkut. I'm trying to think of ways that people could easily ping their already existing social networks to start jam sessions that are perhaps simpler than going through the scheduling page on the Auracle site. Are there any plans underway to publicize Auracle or attempt to jumpstart the userbase?

JF: We've done a lot of publicity, especially in Germany, of Auracle, though we could always do more. I think that integrating Auracle in with some standard IM software might give us a little boost. But we actually do get a reasonable number of people to visit the site, the problem is that only a tiny fraction of them ever launch and use Auracle. So here are the big issues on my mind:

1) The majority of PCs (including almost all desktop machines) lack built-in microphones, and only a small percentage of users have external mics or headsets.

2) The attention span for net art is amazingly short. Very few users are willing to download a plugin -- no matter how easy and quick a process it might be -- just in order to use a site. So the technical limitations of in-browser technology, which led us to rely on the JSyn plugin, create a real accessibility problem.

3) While many people routinely plan their time so they can catch a favorite sitcom or sports team on television, very few are willing to plan their schedule around computer-based activities.

#1 is already starting to change, as laptops begin to outnumber desktops and as computer-based telephony starts to take off. #2 will eventually change, as computers get faster and faster and projects like Auracle can be implemented in environments like pure Java. But I hesitate to make a prediction on #3. As Internet services gradually replace television, will people be more willing to schedule their computer-based activities? Or is part of the appeal of those activities the fact that they needn't take place at particular planned times? My own plans for future projects are hedging my bets: I'm focusing on creating collaborative environments where the participants need not be on the site simultaneously. (Think of a mailing list or Wiki rather than a chat room.)


PT: I'm well into the paper writing process right now, and its a big one. I'm currently working on the section about N.A.G., and realized that there was one important question I forgot to ask you. N.A.G. seems to really play with this border of legality regarding copyright and fair use. The fact that the work is downloading what the RIAA would consider illegal copies of music files, but then appropriating them for art puts the legality of the transaction in much more of a gray area, and this is very interesting. I'm just wondering what you think about this now, and how you thought about it in the creation of the work and in choosing to use Gnutella.

JF: This is actually the easiest question you've asked me, since I've been asked it a lot, so here's the quick answer: I didn't create N.A.G. with the intention of commenting on the legality of file-sharing, or Gnutella, or the RIAA. I was simply interesting in exploring things about Gnutella which I found fascinating. I do believe that N.A.G. falls under fair use, while many other uses of Gnutella do not, though I realize that others may disagree with my opinions. Between the time I originally conceived N.A.G. and the time it was launched, file sharing on the Internet changed dramatically: the iTunes Music Store had been launched, the first wave of the RIAA's lawsuits against individual file sharers had been filed, and the media was pronouncing peer-to-peer networks such as Gnutella to be irrelevant and in trouble. In my mind, N.A.G. contributes two interesting points in this ongoing debate: 1) DRM'd content may prevent some people from using copyrighted material illegally, but it also prevents some people from using it in creative ways which fall under fair use. (N.A.G. could not exist if the only way to obtain music online were through services like the iTunes Music Store.) 2) Peer-to-peer file-sharing networks put a significant percentage of all recorded music into a digitized, searchable database, and all we can think of to do is to swap files? How boring! There must be more interesting things to be done here! (N.A.G. is one simple possibility.)