The aim of this project is not to ‘prescribe’ hearing impaired people a spatial instrument so that they can start interacting with music. There are already many cures to deafness and the currently incurable kind (sensorineural hearing loss) is not to be treated with the techniques being used in this project as it is a rather a neurological deficiency than a mechanical one. This project rather aims to create a setting where hearing impaired people and people with optimal hearing can generate and listen to music together without having the impaired individuals feeling like they need an “aid” or “fix” to fully be a part of a musical context. This is eventually an attempt to revert the marginalizing effects of biological obstacles on communally shared spaces and activities

Sci-fi inspiration: J G Ballard, Sound Sweep
My departure point was J G Ballard’s science fiction story Soundsweep where, due to the noise pollution in a distant future, contemporary musicians start shifting the sound range within which they compose into an ultrasonic level. Therefore, since there are not any audible music left; sound, once created, does not leave any residues behind. In this future society, where everybody seemingly listen to this “ultrasonic” music and enjoy it, nobody practically hear anything. This scenery inspired me to design a musical setting where the actors are seen as practically listening to nothing ,with their open ears, yet do react to music that is being played literally inside their heads through bone conduction!

Above, you can find some excerpts from J G Ballard’s science fiction story “Sound Sweep” with my illustrations. And here, you can view a more interactive narration of the story.

On the other hand, in an era where huge efforts are in the making for the democratization of musical creation and accessibility, it seems unfair to see hearing impaired individuals being a part of the communal aspect of music at a very restricted level. With today’s technology, we are able to cure most hearing deficincies, and the ones that are currently incurable (most kinds of sensorineural hearing loss) are being profoundly studied. Therefore, it is not the medical effort but the designer’s will what seems what is missing in enlarging hearing impaired’s social comfort zone today. The question I want to ask with this statement is: what can we do to create more inclusive spaces not only for those who are experiencing biological hardnesses (this is still a type of “pozitive” marginalization in fact) but to arrive to a level where these “impariment” implications are no longer relevant. In the search for an answer to this question, I decided to create a spatial music instrument to be played collectively by both hearing impaired individuals and individuals with optimal hearing.

To provide a bit background on the anatomic underlays of the hearing impariment, we can broadly state that there are mainly 2 kinds of hearing impairement: sensorineural hearing loss and conductive hearing loss. Today, if you are a hearing impaired individual and if your doctor is telling you that there is nothing the medical world can do, then you are most likely to have a sensorineural hearing loss which stems from deficiencies regarding the neural pathway of the audial perception. Contrarily, the conductive hearing loss occurs due to mechanical problems in the structure of the ear itself; before the soundwave captured by pinea reach cochlea and get transformed into a neural signal through the hair cells covering cochlea. In this type of hearing loss, as the problem is between cochlea and outer ear, bone conduction based hearing implants can be of solution since they directly excite the cochlea by by-passing the outer and the middle ear. If problem is in cochlea itself generally the deficiency can be resolved by cochlear implants.

In my project, I use a simple oscilator creating a magnetic field between two cones around which a string is tangled. As the string pass across cones, the change in magnetic field vibrates the string. This vibration is them amplified via a small metal piece connecting the string and to a large canopy covering cones. This metal canopy vibrates as the string vibrates. When one press this surface againts his skull, the bones constituting the skull starts vibrating, so do cochlea as an extention of these bones. This is how a piece of sound can reach to one’s cochlea without using one’s outer and middle ear.

Overall, since the project altogether was more ambitious than what is required by How to Make Almost Anything’s final project which is a demonstration of the combination of different skills learned during the course (hopefully in an interesting way), I decided to break the project into sub goals:

• 1.design and build the dome
• 2.design and build a bone conduction circuit that is embedded inside the dome
• 3.design and build a LED circuit that flashes according to the beat of the audio
• 4.design and build a vibration motor circuit that responds to the beat of the audio
• 5.eventually, instead of using an audio file, design a gesture-based instrument so that the hearing impaired users can also produce the music itself other than just listening to it.

I aimed to complete the first 4 steps in the framework of this course, yet despite my efforts on trying to understand the way H-bridge works, I managed to do only the first 3. More on the process is below:

EXTENSIVE DOCUMENTATION OF THE FABRICATION PROCESS:

Part1_Fabricating the elements for the geodesic dome structure

A.Dome’s base

step1: Buying the metal tubes
I decided to use metal tubes for the base of the dome because this is where the vibration motors were going to be placed and steel is great to intensify vibration! I bought 10 of this EMT conduit from home depot; total cost was around $35.

step2: Cutting the metal tubes
I used the metal chop saw for cutting the tubes into the dimensions I needed in N-51

step3: Grinding the edges of the metal tubes
After cutting the metal tubes, the ends needed grinding.

For half of the tubes, I used metal grinder at N51, and for the rest, used the drilling machine with a grinder end.

step4: Bringing the tubes together to create the sitting elements of the dome

I had previously 3D printed the nodes:

Assembling the tubes with nodes:

et voila!

B. Dome’s struts

Struts are the main structural elements forming the dome. Their press-fit nature enables them to work as 3D conduits made of 2D sheets in a reversible manner which is essential for the modular and sectional nature of the project. besides,  having press-fit joints enables struts to be “openable” when needed, given the electric underlay of the project,  without disturbing their structural functions. My dream is for this structure to travel from one city to another by connecting to hearing impaired communities around the globe. In such a scenario, havin a 3D structure disassembling into 2D sheets offers an unbeatable logistic freedom!

step1: Buying the materials for struts
I decided to use 2 differet kind of material for the struts. One is a black museum board from Blick and the other one is a polycarbonate sheet from McMasterCarr. I bought approximately 10 black museum board (cost around $100) and 4 polycarbonate sheet (2 of 24×24″ + 2 of 24×48″; total cost of 4 boards were around $70).

step2: Making the cardboard press-fit struts
The press fit struts were to be made by cutting & scoring (with a laser cutter) then folding a black museum board. Therefore I started experimenting with different notch designs and scoring frequencies:

After deciding on the optimal design, since the frequency of my geodesic dome was 2, I only had to fabricate two different type of struts with different lenghts.

step3: Making the polycarbonate press-fit struts
the main reason why I wanted to make some of the struts with polycarbonate (which is clear yet not brittle as acrylic and not non-structural like polyprophelyn or acetat) is to stick some LED circuitry into its surface with vinyl cutter. However with polycarbonate, since it’s not possible to use the laser cutter due to the material’s release of toxic smoke, the process was more tricky than it was with cardboard. I was only able to use waterjet cutter for this material with which the concept of scoring doesn’t exist.

step4: Waterjet cutting the polycarbonate sheets

step5: Scoring the pieces manually by using a mask, cutter and hot air gun

So, I designed and lasercut a mask which helped me score the pieces neatly by hand.

after placing the mask on top of the waterjet cut polycarbonate piece, I clamped them to the desk to prevent layers from sliding.

then scored the polycarbonate carefully with a cutter. The mask helped me make sure that the lines were perfectly straight!

then, by applying heat to the fold lines, I increased the material’s elasticity which helped me fold it easily.

et voila!

Part2_Producing the electronics of the dome

A. Bone conduction

step 1: Deciding on the amplifier’s gain
I made 3 different iterations (gain:20, gain:50, gain:200) of my amplification board for bone conduction circuit. Used the datasheet for LM386:

gain 200
I connected a capacitor (20uf) between the pins 1 and 8 of the chip (LM386)

gain 50
I connected a capacitor (20uf) and a resistor(1.2 k ohm) between the pins 1 and 8 of the chip (LM386)

gain 20
pins 1 and 8 of the chip are disconnected

Finally, I decided on using the gain:20 one since the sound was the most clear in this circuit.

step 2: Fabricating 4 amplifier circuits

B. Beat reflective lights embedded on a clear strut

after vibrating the skull to create the sound within ones cochlea, I wanted to have a visual feedback as well as a tactile one. For the latter, I would need a vibration motor circuit and I decided to postpone it for future updates of this project since simply there was not enough time. However I decided to try the former.

step 1: Design & fabricate a small section of a LED strip for the proof of concept
homemade LED strip trace:

tracing copper sheets with vinyl cutter, trying different force intensities:

sticking them into a polycarbonate strut’s surface and populating the circuit with LEDs and resistors:

step 2: Connect the circuit to the beat detection system (arduino + processing)
Having made my own circuit for the core electronic part that was bone conduction, I decided to use an arduino duemilanove that I made during input devices week for the light addition. The programming environment was Arduino IDE and Processing (through running the standard firmata). I basically used the adapted version of my code from the interface programming week for beat detection on processing.

step 3: Actually fabricating 3 homemade LED strip embedded clear polycarbonate struts
After successing in the “proof of concept”, I decided to fabricate LED homemade LED strips for the lenght of at least 3 polycarbonate struts since I was separating the beats into 3 channels thru Processing. The main reason why I wanted to do my own LED strip by using a ton of LEDs and copper stickers is because I find the commercial LED strips super kitch. So, for the sake of being able to end up with exactly what I wanti I spent painstaking hours for the vinyl cutting of the copper, sticking the copper bands into the polycarbonate surface and soldering the resistors + LEDs into it. Ugh!!!

after vinyl cutting more copper bands, I started placing them onto the struts’ surfaces

then I soldered the resistors and LEDs

all looked pretty good, so I decided to try each strut with the actual beam detection current. Video below is a timelapse of my trouble shooting session

all done! now it was time to snap close the strut and articulate it to the structure!

C. Forehead band: interface between the bone conduction devices and the skull bones

step 1: Sewing velcro into elastic band
I bought 2 rolls of elastic band and a one roll of velcro. After cutting them into desired lenghts by making sure that they will be adjustable to different head diameters, I needed to sew the velcro into the elastic strap.

I had never used a sewing machine before but I learned how to use it pretty quickly. I was way faster compare to sewing by hand.Before using velcro and elastic band, I made some trials on a scrap textile.

Of, course I got it jammed couple of times but this problem can easily be solved by pulling the mingled thread out of the bobin repository (make sure to higher the needle before to not to break it!)

et voila!

step 2: Sewing bone conduction devices into the forehead band

finally all the velcro bands were sewed into the elastic band. now it was time to sew the bone conduction devices into the head band. I did that by hand since it required lots of attention.

now only thing I had to do was to solder the ground and VCC cables to the bone conduction device during the final assembly of the dome.

et voila!

Part3_Final integration and the set up of the dome

so far, it had been very challenging to come to this final phase. I appreciate to the truth in the illustration below:

The metal bases of the dome: check!

The black carton and clear polycarbonate struts: check!

Bone conduction amplifiers (4 of them!): check!

Adjustable forehead bands to interface the bone conduction devices to the skull: ckeck!

I have started with marking the decagon projection marking the edges of the horizonral struts. This reference is important to be able to know where to put the metal bases.

After placing the metal bases, I started articulating the black carton struts to them.

Then articulated the black carton struts to black carton struts.

even though the assebly theoretically looks fairly easy, it actually requires quite an effort since making holes in every branch of the connecting nodes is a painstaking process!

When all the struts were articulated, it was time to stretch the electric cables across them. The aim was to distribute sound (via an amplifier) from the sound source in the middle (for now, it’s a computer; at the end of the Spring semester, it will be a gesture controlled instrument!) to the each bone conduction devide embedded in a forehead band worn by an individual sitting on the metal base.

The reason why there are thousand cables is because I need 2 currents (ground abd VCC) for each LED strip embedded polycarbonate strut (there are 3 in total) and 2 currents (ground and VCC) for each bone conduction device (4 in total). Below, how the source is looking like:

and it was time to demo!

I told the users to plug their ears wittheir fingers to enhance the bone conduction effect!

Please stay tuned for the further iterations of the project, that is it for now!

Inspiration

This project was inspired by a concept of automated matter which has taken form in many sci-fi films (X-Men, Watchmen, Matrix), and recently at the Media Lab. The idea is a hidden bridge that appears or materializes only when needed, enabling the user to confidently drive off a cliff or into the water, to be met by rising supports just in time. Ultimately the bridge supports shoudl support a car when present, and otherwise go away / retract / disentigrate.

 

 

 

 

 

Preliminary Sketches

Inputs

Light – Photoresistor
Photoresistors are light-dependent resistor (LDR), also referred to as a “photocell”. Resistance across them decreases with exposure to light. Wired from an analog read pin to ground.Read from an analog read pin.

Pros
+ Easy to detect – The kind of light detected is the same wavelength we perceive naturally. Being able to easily see the input signal makes debugging significantly easier.
+ No additional hardware – In most demonstration conditions, light – in the form of daylight, synthetic fluorescents, LED, and halogens – is present

Cons
+ Inconsistent – Visible light levels vary A LOT between spaces. This makes for a bad output for a linear actuator.
+ Reacts to ambient light – In the context of a full-scale implementation, a photocell input would result in the bridge delaying for clouds, or any other opaque object that could move over the bridge path.
+ Reaction not limited to the car – Part of the appeal of a bridge that materializes out of nowhere is the fact that it only works under certain conditions, or for certain people.

 

Magnetic Field – Hall Effect
Hall effect vary the voltage drop from a power source according to the surrounding magnetic field- including the Earth’s resting magnetic force. THis allows it to sense up, down, and the presence of other electro-magnetic fields.

Pros
+ Reaction is limited to the presence of a magnet
+ Magnetic field detection would serve as a basic identifying factor that would enable only individuals cognizant of the requirements of the bridge to activate it, and do so discreetly

Cons
+ Magnets + electronics = ?x??1?$?
Other unknown side effects

Video Demo (clickthrough)

Output

Pneumatic Actuation
Air powered movements. Using one air compressors, gates are turned on and off to inrease/decrease pressure in tubes and chambers to create movement.

Pros
+ Air is pretty friendly
+ Compatible with water
+ System could scale relatively well

Cons
+ Requires a large air compressor
+ Requires weights for submersion
+ Air compressor makes noise
+ Actuation makes loud pops with each pressure release
+ Lots of hosing requires
+ No small form factors for cool models
+ $$$

Hydrolic Actuation
Basically the same as pneumatic, but using liquids instead of air.

Pros
+ Pressure isn’t an issue.
+ Scales well
+ Very compatible with submersion

Cons
+ Requires bulky/large reinforced/structural material
+ $$$$
Magnetic Actuation
Magnetic field detection would serve as a basic identifying factor that would enable only individuals cognizant of the requirements of the bridge to activate it, and do so discreetly.

Pros
+ Reaction is limited to the presence of a magnet
+ Very little background noise
+ $

Cons
+ Magnets + electronics = ?x??1?$?
+ Side effects

Stepper Motor / Lead Screw Actuation
Oringally I didn’t find stepper motors appealing, but then I discovered these steppers with lead screws that act as linear actuators. These are used for many 3-axis machines and other various uses. They are also used in CD/DVD drives to move both the CD chassis, and the laser scanner. Because CD/DVD drives are now essentially obsolete, China and Russia end up buying lots of them as e-waste, recycle them for parts, and then sell them for SUPER cheap. So these stepper motors are plentiful, and cheap.

Pros
+ 1.7″ actuation distance + Lots of torque
+ Tiny
+ Cheap. Between $1-2

Cons
+ No datasheet
+ Tricky to wire to
+ Does not come with a slider

Sourcing
+ Amazon
+ eBay

FSM0815-KD95 Specifications
+ Bipolar (2-phase) stepper motor
+ DC 5V
+ 1000RPM = 16.7 revs/s = 3.35cm/s actuation speed
+ 22 degree step angle
+ 16 pulse (pulses required to make one ful revolution)
+ Dimensions: 6×7.5×13.5mm / 0.24″ x 0.29″ x 0.53″ (L*W*H)
+ Screw Diameter : 2.5mm / 0.1″ Download My Model
+ Screw Length : 43mm / 1.7″
+ Weight: 33g

Stepper Sliders

 

Electrical Design

I decided to make the array out of discreet baords with no communication, mostly for the sake of keeping things simple. Eventually I would like to make a version with centralized controlled, as this would allow me to create more wholistic movments and animations with the pylons.

My board design was based off of the hello.world.steppermotor example found on this page. I added a jumper to the board for a remote sensor daughter board, and moved te power jumper around to accomodate a series of baord being powered in series by one cable.

Autorouting in Eagle never gets old. Eventually I settled on this configuration.

I also designed chips for a power jack, and the remote sensor for each primary board.

Pylons

Final

This post presents the final iteration on my Mercer Box build. Please see my previous post for information on early design decisions and some details on box construction.

Inspiration

Inspiration for the project came from two sources: direct passages from Do Androids Dream of Electric Sheep? that speak to the empathy box’s design and previous projects from others who have created empathy box prototypes.

The following quote serves as the primary material from the novel, which serves to guide the design of my project:

“So, taking a deep breath to steady himself, he grasped the twin handles.

The visual image congealed; he saw at once a famous landscape, the old, brown, barren ascent, with tufts of dried-out bonelike weeds poking slantedly into a dim and sunless sky. One single figure, more or less human in form, toiled its way up the hillside: an elderly man wearing a dull, featureless robe, covering as meager as if it had been snatched from the hostile emptiness of the sky. The man, Wilbur Mercer, plodded ahead, and, as he clutched the handles, John Isidore gradually experienced a waning of the living room in which he stood; the dilapidated furniture and walls ebbed out and he ceased to experience them at all. He found himself, instead, as always before, entering into the landscape of drab hill, drab sky. And at the same time he no longer witnessed the climb of the elderly man. “His own feet now scraped, sought purchase, among the familiar loose stones; he felt the same old painful, irregular roughness beneath his feet and once again smelled the acrid haze of the sky not Earth’s sky but that of some place alien, distant, and yet, by means of the empathy box, instantly available.

He had crossed over in the usual perplexing fashion; physical merging – accompanied by mental and spiritual identification – with Wilbur Mercer had reoccurred.”

[…]

“Releasing the handles he examined his arm, then made his way unsteadily to the bathroom of his apartment to wash the cut off was not the first wound he had received while in fusion with Mercer and it probably would not be the last.”

Excerpt from: Dick, Philip K. Do Androids Dream of Electric Sheep?

The book dealt with the central questions of “What make something alive? Does consciousness require empathy? Does something that is conscious, but not empathetic have a right to live?” To provide the perspective from a spiritual standpoint, the religion of Mercerism is introduced. Mercerism’s main goal is to engender empathy through a collective painful and arduous experience. The religion was not completely described through the book, but one aspect we are familiar with is the method by which Deckard, Iran, John Isidore, and others participate in the religion.

As we see in the quoted passage, device called an empathy box permits the user to “fuse” into the shared experience of Wilbur Mercer’s trek up a mountainous staircase by tightly grasping the boxes’ handles. Mysterious figures (unseen to the user) throw rocks as Mercer and the user climb striking Mercer and the user of the empathy box. Importantly, the pain from the rocks directly affects the user as demonstrated by Deckard and Iran (and John Isidore in this presented passage) often emerging from the fusion box interaction with cuts and bruises. It is not made clear how mechanism this occurs, or even really what the device looks like for that matter, so here I used my imagination to fill in the holes.

Analyzing the experience described in the novel, an accurate representation of an empathy box would contain three main components: a visual component to convey the hallucination of Mercer climbing the mountain, a haptic component to deliver the impulse from the falling rocks, and a “fusion” component such that multiple people can undertake the experience together.

Prior Art

Given the empathy box’s importance in the book, it is no surprise that others have previously worked with the concept of an empathy box for devices of the their own. I have found two devices in particular. The first is from MIT Media Lab alum Sophia Brueckner.

Her empathy box serves as a gateway to her exploration of mental and physical well-being and the meaning of “connectedness” in the digital era. In her exploration, she channels Sherry Turkle’s Alone Together and George Lakoff’s theory of embodied mind. Ms. Bruckner designed her devices such that when two or more devices are used simultaneously, the handles would heat, and the users would share a common warmth.

For more information, please see her personal website: http://www.sophiabrueckner.com/empathybox.html

Another take on the empathy box comes from European art collective IOCOSE, comprised of Matteo Cremonesi, Filippo Cuttica, Davide Prati, and Paolo Ruffino, with electrical design by Mirko Mattioli. Again, they design a tabletop device to elicit an empathetic response through a collective experience. The collective experience here is more local – shared with the people in your immediate vicinity, rather than strangers in a remote location. A group of people form a line by holding hands. The first person rests a hand on one electrode, and the last person does the same with the other electrode, forming a ring. After adjusting the settings for number of people and pain level, the box sends an electric shock through the line of people which can be felt together simultaneously. Just as in the novel, the interaction is painful rather than pleasureable.

For more information, please see the artist’s website: http://www.iocose.org/works/empathy_box.html

Experience and Technical Design

Recalling our three empathy box components, we see that experiences form these previous works focused on both the haptic portion and the collective “fusion” portions. Generally, they are less focused on emulating the story’s description, instead playing off of the concept. Since this course is about tying the fabrication close to the functionality of the science fiction description (not just the abstract idea), I decided to tackle the visual component which is yet untouched by others. The haptic component would be integrated such that the visual and tactile feedback would be synchronous. Because of time and budget, I decided to build only one of these devices, however the solution implemented is scalable, and, if multiple devices were constructed, they can easily be linked such that multiple people could undergo the experience simultaneously.

What technologies could I use to emulate the experience described in the novel? The equipment used to convey the visual component within the novel is ambiguous, saying the image formed from a glow emanated by a single cathode-ray tube:

“When he turned it on the usual faint smell of negative ions surged from the power supply; he breathed in eagerly, already buoyed up. Then the cathode-ray tube glowed like an imitation, feeble TV image; a collage formed, made of apparently random colors, trails, and configurations which, until the handles were grasped, amounted to nothing.”

An important aspect of the visuals was how enveloping it is, so rather than simply using a screen, I decided to try my hand at developing a virtual reality experience. The visuals would remain true to the novel’s description.

For the haptic experience, I focused on the impact of the rocks. To be able to generate a similar feeling, a mechanical impulse is necessary, as opposed to an electric or heat shock employed elsewhere. The electromechanical actuator that is the best at generating high impulses is the solenoid.

Design Diagram

Construction

All project files can be found here: https://www.dropbox.com/sh/niomxiik2bulwau/AAB09YdrP1ZmB9dhQgaMwnyMa?dl=0

Since I covered construction a little bit in my last post, I’ll only briefly comment on it here.

If you would like to build a Mercer Box yourself, download SolidWorks files from link above. Laser cut 1/4” acrylic using provided .dwg files. Assemble using acrylic weld solvent according to the SolidWorks assembly file provided. For the handles, use two pieces of ¾” PVC tubing and epoxy to the acrylic. To hold it all together, bolt layers using ¼” nuts and washers on the 4-¼” threaded rod posts running through each layer.

Partially completed view of first prototype:

The solenoids used in this project are 12V, 350mA, 20 N p/n JF-0826. The solenoid uses M6 screw mounts that interfaces directly with the laser cut templates I generated. 0.1” brass rod was cut and grinded to form the shaft that pressed against the user finger when the solenoid was engaged. My solenoid plunger required a lock stop to prevent it from falling out, so a washer was press fit onto the brass plunger shaft to provide a hard stop.

The circuitry for the solenoid driver can be found in this article: http://electronicdesign.com/analog/what-s-all-solenoid-driver-stuff-anyhow. Another good alternative resource can be found here: http://playground.arduino.cc/Learning/SolenoidTutorial. As a mechanical interface to the DC power supply unit, a banana plug dual port was installed across the acrylic back plate. That item can be found here: https://www.virtuabotix.com/product/virtuabotix-heavy-duty-port-gold-plated-red-black-banana-plug-speakerpower-receptacle/?gclid=Cj0KEQiAtMSzBRDs7fvDosLZmpoBEiQADzG1vFA8d7BCIjeWiKrbM6TC–q8rZIPJJqNJaGVRe2cU18aAl5G8P8HAQ. This project used a Sparkfun 5V Pro Micro to control the power transistors, however any microcontroller can be used. Pictures of the circuit on the breadboard can be found in my earlier post. The USB cable should be threaded through the hole in the black plate. Voila, you have yourself a certified Mercer Box! Download the Arduino code from the link above and upload to your microcontroller using the Arduino software.

The development of the visuals was a particularly instructive experience for me. In addition to learning Unity from scratch, I had to learn to interface with a number of other hardware devices, namely the Arduino and the Oculus Rift. The first step involved creating the world using a terrain builder, a rocks asset pack I round on the Unity Asset Store, and other skybox and lighting assets. For terrain traversal, I baked a navmesh. Next, I purchased a monk asset, and rigged humanoid animations to him. An AI nav script took him from the bottom of the mountain to the top of the mountain in a reasonable fashion. A script spawned boulder assets above and in front of him every certain number of frames. Another script linked collisions between rocks and the monk avatar to messages sent over an opened serial port with the Arduino. A virtual camera movement script trains the user’s view on the the Wilbur Mercer character. Throughout the development, physics were modified such that the interaction looked as realistic as possible.

I would like to thank Ben Berman and Phil Cherner for their assistance in getting me up and running with the Unity software. Ben kindly lent me his hardware including the computer and the Oculus Rift. Phil provided some example code from an earlier project for spawning game objects and sending serial data from Unity.

An Oculus Rift DK1 running through Oculus 0.7 Windows runtime was calibrated with the configuration utility. Unity 5 handles the stereoscopic rendering and VR head motion control behind-the-scenes, and automatically relays the visuals to the head mounted display.

Conclusion and Future Directions

During the demo, I have the user pull the Oculus headset from its compartment within the top of the box, and don the headset. They are immersed in the visualization of Wilbur Mercer slowly ascending the mountain. They are free to look around – at the figure of Wilbur Mercer climbing or up ahead toward the mountaintop from where rocks come rolling down toward them. I instruct them to grasp the handles of the Mercer box, which guides their hands to within striking distance of the solenoids. Upon seeing the rock strike the figure of Wilbur Mercer, the solenoids fire, creating a quick sharp pain on the user’s knuckles. Just as in the story, our cloaked Wilbur Mercer is never able to ascend to the top of the mountain, instead getting continually reset.

The final form turned out to be one behemoth of a project spanning a number of different parts and disciplines — mechanical enclosures and actuation, electrical components and communication, and virtual scripting and modeling. In the end, I am glad they all came together in such synchrony. Through seeing the avatar continually bombarded and feeling physical pain upon impact, one does empathize with the virtual avatar, in addition to themselves feeling tormented. Testers have told me that the experience is mentally and physically fatiguing.

We need not look into the science fiction future to be able to realize the utlity of this device. In the world today, there are a number of situations where empathy, engendered through a device like this one, can help to make for social change. Given the haptics are designed for impulses, it is not hard to foresee this device being used to deliver visions of a battlefield, such as the perspective of a tormented solider caught in the thick of a firefight. A particularly timely example would be engendering empathy for refugees of war-torn areas, such as Syria. By visualizing and feeling the experience of a displaced civilian in an area of shelling, for instance, many would feel more inclined to help with the refugee’s situation.

Future improvements of this project could focus on creating richer haptic experiences by, for example, using more solenoids across the body or incorporating other types of actuators. Additionally, creating more of these devices would allow for multiple users to share in the experience, building out the third dimension of the device expressed in the novel.

I. INSPIRATION

The base SF story inspiring this project is DAREDEVIL. Other sources of inspiration are  Ironman,  & Batman in regards to how high-technology  (or super technology) can be dressed and the synergy between the body and the device. For that synergy there is a learning process involved, an adaptation to the usage of the device in order to effectively get advantage of using it.

    

Other source of inspiration was the personal impression I get from blind people, I admire how brave they are and their capability to overcome obstacles.  The way how our body can adapt our senses is simply amazing

II. THE FINAL PROTOTYPE

This device is composed of two parts. The goggles and the glove. LiDAR and sonar sensors (ULTRASONIC)  are located in the goggles in order to measure the spatial information (proximity)

The information of distance is computed to determine proximity levels en each sensing region. So the frontal vision is divided in three zones, one for the LiDAR and the other two for the sonar sensors.

The first three fingers receive a tactile vibration that is mapped in regards to zone and the proximity level.

The tactile vibration produced at the glove has two attributes:

+Intensity

+Frequency

Two configuration of SW can be set to three different haptic profiles:

1. All fingers receiving tactile vibration at the same time but each finger receives different frequency depending the distance of the objects surrounding the user

2. Only one finger receives the vibration depending on the zone where the closest object is.

3.  “Radar format” in which the actuation in the fingers follows a sequential pattern and then the information about the distance is differentiated by the difference in  the intensity  of the vibration.

The glove also has a sonar sensor located at the middle fingers, this sensor provides a hard ping in the fourth finger. The purpose is to use this sensor to explore the surrounding in the very short range. So the actuation of this fourth finger has different threshold levels for the vibration profile (the hard ping).

The main technical challenge involved with the implementation of the haptic profiles was in setting a variable PWM cycle on each actuator. To accomplish this, an object was created in Arduino and the structure of the program follows a “multitasking” format to allow multiple actuators to be driven with different PWM cycle. Using the dedicated PMW pins would constraint the behavior of the vibration to only have the oscillation period of the micro and thus we could only play with the intensity level.

  HIGH LEVEL CONCEPT

Technology for fast adaptation

Adaptability Is THE characteristic that has  allowed us to survive as specie and has shaped us to what we are today.

But when it comes to migrating to another planet the human race will have to adapt to live with other gravity level, other sources of energy, different radiation levels, etc

III. FUTURE PATHS

+ Design of a Product Design  

A consumer electronics firm (like Samsung) is a suitable company to take this prototype and concept to the “product” level. They would be able to provide all the robustness needed in the device. These are the products that I visualize:

♦Walking-navigation assist device for blind people. Similar to the functions of the current prototype but adding a matrix of actuators to deploy 2D figures and communicate information in Braille code

♦Devices for safety standards assurance in manufacturing facilities. This idea can be translated to provide feedback in a helmet or other parts of the body where the user can easily interpret warnings of safety hazards.

+ Research (how evolution is learned?)

By studying this type of devices that involve learning processes, designers of large-scale engineering systems could get insights on how to effectively implement evolutionary engineering in the development of such very complex systems

Developed by NECSI, evolutionary engineering  proposes the utilization of a well-known –and proved by nature- strategy to develop complex systems. Inspired by the evolution of species, this method suggests the design of complex systems can be achieved by the gradual aggregation of “tiny steps”. The baseline system is one already stable and robust to which an incremental function is added with several alternatives that are intensively evaluated to learn and then document all the needed requirements that guarantee robust deployment of the new (slightly enhanced) system. Best (strongest) alternative is selected for the next iteration. While in one hand rapid-prototyping for fast iteration becomes crucial to accelerate the learning, in the other hand the broader the space of alternatives the higher the possibilities that the selected alternative to be close to the optimal one

IV. SciFi DEVICE  (hand faster than eye)

Why should I spent my time watching a tablet while I can get all the info at hand while I still converse with my friends keeping eye contact?

The goggle-glove is the future version of the goggle chrome. This browser is in your hand. This is a highly sophisticated glove made of programmable materials that deflect in shape to show any graphic information (like a map or a web site). The paradigm is inverted, instead of the hand moving along the information, is the information the one that moves thru the hand while we browse.

It’s like “feeling the goggle maps” in your hand instead of watching it. With this glove you can browse anything and instead of seeing it you are feeling it. Enhancement of the cognitive capabilities enable us to understand the information as if we would be watching it but faster. The environment in which this device exists, is one where people has evolved its natural senses to such high level that, tactile sense is even faster than sight in many aspects, so, for speed purposes the people prefers to get the info thru their hands instead of being “attached” to watch something. We are talking about multitasking senses. People has the ability to keep a conversation naturally while at the same time is browsing the news thru their hands.

The enablers:  programmable materials, shape shifting clothes, wireless communication, sensing technologies of all type (motion, position, temperature, pressure, CO2 levels…etc)