PDF Slide Deck: Audio Sketchbook

Problem Description
Game jams are sessions of designing, prototyping, and building games constrained by a particular theme and time limit.  Game jams can last between 6 and 48 hours, and attract participants with a variety of skills and backgrounds.  From empirical evidence, most of these participants are either game designers or programmers.  Aside from design and programming, games often employ art and music.  But while most games produced through game jams include some form of art, significantly less include music or sound effects.  This lack of audio support could be from a lack of sound design participants.  If so, how can we encourage more audio designer participation?  The lack of music could also be a result of poor audio prototyping tools.  If so, how do we create an easy-to-use prototyping tool for iterating sound effects and music?

Proposed Solution
To answer each of these questions, I propose a product that allows for better, simpler communication of audio effects.  From empirical evidence, most game jam participants are comfortable with sketching designs on paper, whiteboards, blackboards, and computers.  During these sketches, they are also making gestures and providing sound effects.  If we capture these sound effects during the sketching process, we can utilize these effects in prototyping the game.  I propose an “audio sketchbook” tablet app that allows users to sketch images and record audio over these images.

This sketchbook would have three modes, or “layers,” of sketching.  See Figure 1.  The first layer is the drawing layer.  It allows users to draw freely.  Ideally, users could utilize standard digital sketchbook options such as setting the color, thickness, and opacity of their brush.  The second layer is the audio layer.  While a user performs brush strokes, audio is captured.  This interaction takes inspiration from the “Singing Fingers” iOS app (http://singingfingers.com/).  The final layer is the editing layer.  This layer allows users to select sections of the audio layer for editing.  Editing options include copying, “slicing”, “stretching,” and deleting audio sketches.  Through these three layers, users can create, read, update, and delete their line or audio sketches.  By providing a sketching tool for prototyping audio, I hope to give game jam participants a simple way to iterate and produce audio effects with a medium they are already comfortable using.

Figure 1: Layers of Audio Sketchbook