« Suspicious Packages | Main | [Talking-Points] Wizard of Oz study »

February 04, 2008

Noteworks - The Visual Metaphor

Quick Update:
We’re currently bringing the UI up to speed, users can now zoom in and out with the mouse wheel, and move the view within the window! Ease of use will increase exponentially from here on out as composing moves away from the code level.

Not so Quick Update:
Our design review stirred up a number of thought provoking questions, and I believe everyone on the team was amazed by the sheer variety and quality of comments / suggestions. It’s all too easy to become short sighted and single-track minded with any project of this magnitude. If there’s one word that would best describe the piece of software we hope to create (and one word I’ve overused in my blogging to date) it’s “dynamic.” The environment from which this project emerges could not be any less so, if it’s to be successful.

The discussion tended toward the subject of what sort of visual metaphor’s we’d be invoking. Will the distance between two nodes represent the delay between them? Will louder nodes be larger? Will melodies and instrument timbres be represented by different colors (a common synaesthetic phenomenon)? Could waves propagate outwards from firing nodes and amass in swirling vortexes? These are all valid questions, however, thus far we’ve been focused on the musical aspects of Noteworks, and haven’t devoted as much time to design aesthetics.

When I began working with the computer-music software package “Reason” I was so sold on the studio rack metaphor that I took it for granted when I pushed tab and “took a look at the back of the synths,” the chords jiggled and swung as I re-routed the wiring. Even though this was a new piece of software, I was immediately able to understand how routing worked because I applied my knowledge of how to route physical devices.

When I began working with Logic Pro I was immediately drawn to the synthesis and effect modules not only by their sound, but also by their look (which I will admit, though it may reduce my credibility with the audiophiles). The synths have a polished, racecar like finish. Switching audio filter modes between parallel and series (on the ES2 synth module ) causes a central disk to rotate around in a way that conjures up images of the star-trek enterprise. It’s a pleasure to watch, and the interface begs to be interacted with. Bottom line: the interface can make or break a piece of software.

This is either a screenshot of: A. Sculpture, one of the many synth modules in Logic. Or B. A time machine, sent back from the future.

Our current situation is addressed in one of my research papers for Prof. Sheila Murphy’s course on Digital Media Theory:

“While advances in technology have allowed for increasingly precise levels of control, ease of use has always been a top priority for software engineers. This becomes especially apparent in applications that generally call for a high level of creative freedom… The ability to digitally sculpt sounds has allowed composers a higher level of control over sonic spaces than was ever possible through common music notation methods. However, as rhythmically driven genres of computer-music have been assimilated into mainstream listening environments, the production tools have undergone a similar commercialization. This commercialization is seen as a forfeit of precise control of programming languages for the functionality and accessibility of graphical user interfaces...

…While analyzing a computer-music production interface such as Reason, it is easy to lose a sense of creative potential that has been rendered possible through computer technology. The user interface is seamless to the point that the user potentially realigns their expectations of what is possible in a digital system with that of a physical system.”

To address the questions I raised initially, I’d love to see a tight visual correlation between what’s seen and what’s heard, but not at the expense of stifling the creativity of the user. The million-dollar word here… options. A relevant example: spell check and auto formatting in Microsoft Word. It’s great when Word indents numerical lists exactly the way one would like it to, or when it automatically corrects spelling mistakes that would have gone un-notice… It can be a huge timesaver. The flipside of the coin: it’s equally frustrating when word auto-formats when the user has no intention of doing so. If I didn’t have the option of turning off auto formatting, I’d have pulled out half my hair by now.

It was mentioned during the design review that it’s tricky to design software easy enough for a child to use, with advanced functionality for the expert (die-hard) user. Children all over the world can use Word, but it’s also the go-to app for many corporations and higher learning institutions. So is word a success story then? Well, Word conjures up the “piece of paper” metaphor. I used to draw all over the paper when I was young, where’s the “doodle” tool? I used to love making paper airplanes, where’s the “fold this document” command? These functions must not have been deemed important enough to be included within the scope of the visual metaphor.

The reason many people haven’t noticed that Word doesn’t have a doodling tool is because users quickly begin to think within the scope of the application they’re sitting in front of. Instead of “what’s possible?” the question becomes “what’s possible with the tools I’m currently using?” I’m not ready to put a cap on noteworks capabilities; similarly, I’m not ready to commit to any visual metaphors. Although, I am wondering exactly how a “fold this composition” tool would work…

Noteworks isn’t just a different application for creating computer music; it invokes a different way of thinking that will need new metaphors. I’m not looking to re-invent the wheel by designing a sleeker, more user friendly MAX/MSP, or a more flexible Reason. I’m hoping months after noteworks is released, I’ll be able to look at the compositions of others and say, “wow… I never thought about that possibility.” I can’t say exactly which direction Noteworks is heading, but I can say I’m working with the most talented programmers on the planet. So long as we create something flexible and dynamic (bingo!) enough for the mind of a child, I think the adults will warm up to it.

-Rob

Posted by rlalexan at 03:31 PM | Comments (0)