SoundFORMS is a sound synthesizer (generator) and sequencer (organizer) that allows users to interact with a physical representation of a sound wave in real time.

Concept

• Program sound sequences
• Visualize sequence pulses
• Modify synthesized sounds by touching the wave

3D models

The CooperFORM row pins closest to the user act as buttons that trigger sound waves and drum beats. The user hears the sounds and sees the wave forms in real time.

User can create waveforms and manipulate them with the predefined hand gesture vocabulary.

Technical Overview

Our system is built on top of the existing inFORM CooperFORM, a square dynamic shape display. We use the Javascript interface developed by Xiao Xiao, et al, to write code for the synthesizer. We used the web audio API to generate the different synth sounds and drumbeats.
3 Waveform options: A user can manipulate the waves on a shape display using predefined gestures to switch among a sine wave, sawtooth wave, and square wave.

GIF showing different wave patterns
Preset drumbeats: soundFORM comes pre-configured with 9 different drumbeat patterns from which a user can choose.

GIF showing waves and drumbeats

Application

• Use in live concerts
• Demonstrate synthesizer for music education
• Use as an instrument in a recording studio

Future Work

• Expanded gesture vocabulary for wave sculpting
• Program drums with touch detection
• A protoboard for audio mapping
• Sequence drums on table in real time

Related Work

• PocoPoco
• Tenori-On
• Teenage Engineering Synth

Team

Donald Derek Haddad, Pat Davivongsa, Halla Moore, Brian Tice, Aubrey Colter

Final Presentation Slides

Final Paper

A tangible platform for music creation and playback in three dimensions.

One way we propose to make music is by using a music sequencer: a way to place notes in time. We are interested in being able to touch sound and manipulate sound by touching it in real time.

The basic idea is to arrange musical notes in time on a grid. A cursor sweeps left to right.

Our system would have 2 modes: Compose and Playback modes.

Compose mode: the shape display becomes a canvas to create music. In the upper half, the surface becomes a touchable, sculptable music creation section. Vertically the shape display turns into a musical staff. Think of the grooves on a music box. In the bottom section, the shape display turns into a beat creation section.

Playback mode: the patterns will move across the display. There could also be a waveform that allows the user to tangibly manipulate pitch.

Idea 2: Tactile Education

An adaptable play surface that mimics traditional toddler block games. Using the shape display to teach toddlers and help them develop their hand/eye coordination. “Adaptable” is the key word, it allows for multiple games without taking up more space. More compact than having several separate games, easy clean up. Vision of eventually being able to download games, like a tactile tablet

Potential games:

Simon Game – Instead of lights flashing, pins bounce, and toddler has to push down pins in same order

2-player – getting parents or siblings/friends to interact with the toddler using the table.

Shapes – teaching basic shapes, like squares, circles, triangles. It could be expanded to showing letters, or with projection, could teach color.

Sorting – like the shape in holes game, except outlines instead of holes

Recognition – the pins could form several shapes and an audio file could announce the name of one, then the toddler should push down all the pins in that shape. We could also combine projection with this to add color recognition

Matching – Create a shape that matches a computer formed shape, either from scratch, or from a shape that’s almost there, like the “Find the Differences” game

Our slides can be found here.
—
 

Sequencer can be an interesting exploration. Considering using the actual sound shape displays make would be interesting. It could be interesting if you can make a music which can be made only with your approach. Here is a cool related work POCOPOCO.

Not sure about the education tools. Just having several different games is weak. I will recommend to focus on one compelling scenario.

The rectangular keys are programmed to be white and black keys. The user can press down on the key, and the correct note sounds. When the user lifts their finger, the key returns to the original position.

Modes of playing:

• Play for fun/practicing (normal piano mode)
• Play to learn — the piano stiffens the keys you are not supposed to play, and lowers the key you are supposed to play, to teach the user where to put their fingers to play a certain melody
• Head-to-Head: the piano will likely only take up 1/2 of the shape display. The other half of the display can be another piano that can be played by a user opposite of the first user.

2. Shape Display Whack-A-Mole

Played similar to the classic arcade game. The display is programmed to raise blocks of 3×3 pegs at random intervals and in random locations. The user tries to hit the blocks that are upraised. If the block is hit, it could ripple back into the display.  Could be used as a game for children, in an arcade, etc. Or it could be used for physical therapy and rehabilitation.

I also think it would be really cool to integrate with some other piece of technology, like a LeapMotion.

Aubrey Colter

ajcolter@mit.edu

The Shoe Lovers:

Aubrey Colter, Manisha Mohan, Andrew Ringler, Udayan Umapathi

Project Overview

Introduction

How many pairs of shoes do you own?

There are many reasons why we own many pairs of shoes. We have shoes for exercising, working, playing, dressing up, feeling comfortable, managing different seasons, etc. We change shoes a lot.

We encounter many different floor and ground textures throughout our day-to-day. But most days, we don’t carry extra pairs of shoes for those different textures because shoes are bulky. Furthermore, it usually is not practical to change our shoes when encountering a new surface, such as a slippery wet floor in the office or an icy sidewalk in the winter.

Problem: Our shoes do not change traction based on the surface they are touching.

Problem Discussion

There are many common scenarios where a shoe that changed its traction based on the surface it is touching would be practical.

1. Winter Safety: Increased shoe traction can prevent slipping on icy surfaces. This is particularly a concern for pregnant women and elderly people who are at a high risk for serious injury if they slip and fall.
2. Outdoor Activities: Additional shoe traction can improve grip on varying terrains. For example, hikers encounter paved and dirt paths, and need varying levels of traction depending on the surface and the incline of the path. Rock climbers need more traction in places where they do not have a large or solid hold on the rock surface.
3. Indoor Floor Surfaces: Indoor flooring varies from wood to tile to carpet. Shoes that have good friction on carpet may be too slick to walk safely on polished wood. Floors can be wet or at an incline which increases the need for additional traction to avoid slipping.

Solution: Design pneumatic shoe soles that dynamically change traction based on environment sensing.

Design

Sketches

1) Cleat-style pneumatically actuated spikes come out of shoe.

2) Pneumatically actuated coils

3) Different traction patterns

Preliminary Designs and Fabrication Iterations (GIFs)

We created our molds using DragonSkin 10. Initially, we experimented with using balloons to create air chambers in the silicon.

We also experimented with layering different traction patterns.

Final Fabrication

Finally, we settled on using mylar sheets, laser cut in different patterns, to create the air chambers.

GIF of mylar fabrication process

Prototype GIFs

Basic sole inflation:

Inflation on shoe:

Demonstration of increased traction on shoe:

Use case: Environment sensing and dynamic traction

Future Work

We know our work with changing traction can be applied to other fields and is not limited to shoes. We see direct application for the following:

1. Rubber gloves: Glass dishes are very slippery when wet and soapy — and they will break if you drop them. Other dishes are less slippery (and less likely to break) and require less grip. We envision gloves that could adapt their grip based on the type of dish the wearer is holding.
2. Tires for cars and bicycles: Road and trail conditions can change suddenly, especially in the case of inclement weather. We envision designing tires for cars and bicycles that change their traction dynamically based on environmental factors (such as weather, road condition, and speed). Furthermore, we think combining different traction patterns on the same tire would give more or less traction when used together or separately.

Currently, it is unlikely that a computer can predict changes in surface texture. We envision a smartphone app for Pneu Shoe wearers that allows them to control the traction of the shoe, based on what they can observe about their environment. Tapping one of these four basic images would adjust the traction of the shoes to provide better grip on these surfaces.

App prototype homescreen:

Related Work

Jifei Ou and New Balance: Jamming Shoe

GeckSkin, Crosby et al. UMass Amherst

Artificial Setae with Carbon Nanotubes Zhong Lin Wan et al. Georgia Tech

The Reebok Pump

Heelys

Retractable Skates

Sports cleats

In our research, we did not encounter previous work relating to changing the outer sole of the shoe.

Final Presentation Slides PDF

For example, it could be used as a baby monitor for a deaf person. Typical baby monitors relay the sounds of a baby, and often use a light to indicate that sound is coming out of the monitor. But this sound could be the baby cooing, talking, softly crying, or crying loudly, and a deaf parent would not know. The bracelet could connect to the person’s smartphone via Bluetooth, which runs an app that monitors the decibel level of sounds in the phone’s microphone. The bracelet would react, inflating somewhat for a soft cry, and gradually growing tighter as the baby’s cries turn to screams. The parent would then know to go check on the child. This could be extended to alert a deaf person of any loud noise in their home, such as children fighting, something falling and crashing, etc.

Sketches:

As another example, this could be used by diabetic patients to monitor their glucose and insulin levels (only the app contents would need to change). As the insulin in their blood decreases, the bracelet could gradually inflate. When it dips below certain level (set by the user), the bracelet could reach its maximum inflatable capacity, which would be tight and uncomfortable on the user’s wrist, alerting them to test their blood and inject insulin.

Aubrey Colter

ajcolter@mit.edu

My specialty is coding! I work in Java, Python, C++, JavaScript, jQuery, HTML, CSS, and Android. I have some familiarity with MATLAB and iOS: Swift.

I’m really looking forward to this class. Shoot me an email to say hello: ajcolter@mit.edu

Fun fact: I’m conversationally fluent in Spanish and Japanese. これから宜しくお願いします。