Anirudh Paper-final

My submission for final assignment is attached

8 Cassandra

Intelligence transfer through habits

Since elementary school, I have gone to school with friends smarter than me. It has been a laborious battle to accumulate knowledge to impress them. At one point, on the weekends, I sat in the middle of the floor with a circle of open textbooks, shifting clockwise from one book to another, so that even as I came to be bored with one subject matter, I could maintain a constant rate of learning. I developed heuristics for studying such as minimize the amount of material you need to memorize in favor of deriving concepts from previously learned concepts and notice big ideas applicable to multiple domains. Whenever I discovered and adopted a new learning heuristic—it rapidly accelerated my rate of learning—much more so than picking up a one-off fact about something like the xylem in plants (the conduit for transporting water in plants, by the way).

As I moved onto even more difficult proving grounds, the people around me became even smarter and seemed to have better heuristics than me. One friend has amassed a network of dense and compact knowledge, in contrast to the superficial and thin veneer of knowledge that is easy to accumulate through rote memorization, by using analogies and even stretched analogies to link new material with previously acquired knowledge. Another friend is one of the best programmers in the world as ranked by international competitions—and from working with her on joint programming assignments, I noticed that she reflects continuously about the bigger picture of the problem (and change plans early if something better comes up) even in the midst of working on gritty implementation details.

From personal observation, some of their heuristics (let’s call them good learning habits, if they employ the heuristic over a long period of time) were major factors for how they were able to accumulate so much intelligence! As I got more involved in the field of intelligence augmentation, I asked my dear friend of ten years and involuntary subject of my one-way academic rivalry, if he had any habits responsible for making him smart. He offered this rule, always be thinking. He noted that even at MIT, there were instances he would hear people say, “I am so tired. I don’t want to think right now.” “No!” he says, “You always want to be thinking. Thinking is fun!” He said that he keeps a short list of problems on hand that he would think about for fun during all the little extra margins between events of his day. Apparently, my friend’s rule has been derived and employed by the great physicist, Richard Feynman, as well.

A recent heuristic that I have personally adopted and have found to accelerate the rate of learning or at least to give the illusion of control over it is, if you aren’t learning fast enough, learn another way. If what you are doing is not working, then you need to switch algorithms or twiddle the parameters. It makes no sense that if you are not achieving your desired output to wait, hope, and pray for variance and noise to push you over the edge. It is time to swap out your algorithm for another—any other—even if the potential algorithm does not appear to be better than the one you currently use. Your current algorithm has been empirically validated to not work, so science tells you that you should test another one. This heuristic has helped me a lot in grad school thus far, because whenever I get stuck because it does not afford me the opportunity to stay stuck. I always have the option of changing my routine (and as a computer science major, I like the idea of programming and optimizing my life.)

These word-of-mouth testimonials about heuristics begs the scientific question—can we measure the use of these heuristics in a controlled manner? Yes, we can!  Research in this area that I will name “intelligence transfer through habits” is not new and highly cross-disciplinary. Education researchers have for more than fifty years studied what they call “teaching thinking” in which they attempt to teach kids to think like a physicist, mathematician, or historian [1]. Another education researcher by the name of Costa has even drafted a list of sixteen good habits of mind [2]. Costa’s rules are more general but very much in the flavor of the heuristics that I presented earlier. Unfortunately, as noted by David Perkins of the Harvard Graduate School of Education [1], research in the area of teaching children how to think has fallen out of popularity with the emergence of rival camps of educational philosophy, in combination with the fact that short-term benefits of teaching students how to think are less observable than rote “back to the basics” repetition of the three R’s.

In another discipline entirely, behavioral biologists have greatly advanced the study of habits. They have noted that habits may account for much of unconscious behavior and that we only limited mental capacity to make conscious decisions  [3].  In addition, there has been work on how to form and break habits [4].

For particular domains such as computer science and mathematics, experts have worked to actively consolidate the heuristics that have worked for them and have used them to publish books such as The Pragmatic Programmer [5] for computer science, How To Solve It [6] for math, and Getting Things Done [7] for efficiency. None of these rule compilations have been systematically tested but they seem to work—again by testimonials from those I respect and their overwhelming sales volume . The act of learning from an expert offers hints as how to proceed for intelligence transfer via habits.

—————–

The end-goal of this research related to the problem of intelligence augmentation—how to amplify the natural born intelligence of humans—from the vantage point of interfaces for transferring “intelligence” from one person to another. One likely way to transfer “intelligence” is to transfer habits. Applications could be built upon wearable platforms like Glass or smartwatches to just-in-time prompt people to execute a particular habit at the appropriate situation.

For this class project, I hope to settle a few matters of scientific curiosity. While it is possible to ask people to adopt an arbitrary collection of habits, would we get a different result by asking people to adopt someone else’s habits? My hypothesis is “Yes!” due to nature of humans as social beings. In fact, I believe that asking people to adopt someone else’s habits would facilitate adoption and retention rates. The basis of this conjecture is evolutionary. Humans have evolved to mimic others in our social group through our body posture and facial muscles. We have higher cognitive functions that facilitate admiration and emulation of those we respect. In addition, there may an emotional or social component of using something originating from another human being. And lastly, there may be a factor of the tried-and-true success narrative that will boost results based on rationality or placebo effect.

I have designed two experiments to test this hypothesis. The first experiment attempts to measure if habits are more likely to stick coming from another person. I plan to:

• Give people super long list of vocab words to learn
• Measure the number of words learned (3 groups) and rule adoption rate / persistence (2 groups)
• Experimental groups:

–      Control: Ask them to maximize the number of words they learn that week

–      Ask them to follow some rule in order to maximize the objective

–      Tell them some rule is someone else’s habit: then ask them to follow some rule in order to maximize the objective

The second experiment aims to test if adopting someone’s habits change you as a person. I plan to:

• Ask people to self-evaluate their personality
• Help people to encode their daily habits
• Ask people to wear someone else’s habits for one week
• Ask them to re-evaluate their personality

References

1. Perkins, D.  40 Years of Teaching Thinking: Revolution, Evolution, and What Next? 
2. Costa, A. L., & Kallick, B. (2000). Describing 16 habits of mind.
3. Bargh, J. A., & Chartrand, T. L. (1999). The unbearable automaticity of being. American psychologist, 54(7), 462.
4. Duhigg, C. (2013). The Power of Habit: Why We Do what We Do, and how to Change. Century.

Assignment 8 – Sophia

Mindfulness Bracelet

Mindfulness is the state of being aware in the present moment as well as taking a non-evaluative and non-judgmental approach to one’s inner experience.  Adopting new habits and new ways of thinking is very difficult.  New books and studies are constantly released with methods for self-improvement and the secrets to happiness, but trying to remember these in the midst of the hecticness of life can seem impossible.  Practicing mindfulness is one way to improve awareness of habits and goals during those busy moments, but remembering to be mindful itself can be hard! Is it possible to build a device that can improve mindfulness?

To change habits and reach goals we need to remember our intentions at the right time.  Existing tools for remembering and achieving goals are evaluative and can feel judgemental, two qualities which go against the teachings of mindfulness. Notes, visualizations, inspirational images, logging tools, point systems…even tying a string to your finger…are all easy to ignore when your attention is elsewhere and not easy to take with you.  Calendar reminders and alarms feel judgemental and intrusive.  Devices like Fitbit and reflectOns are evaluative and may feel judgemental to some. In general, systems that use rewards or punishment to motivate are shown to not be as effective in the long run.  Once the system is gone, the person’s behavior tends to go back to normal.  In addition, happiness and satisfaction also lessened using punishment and reward systems because the accomplishment feels less authentic.  Developing intrinsic motivation through a mindfulness meditation practice is supposedly one of the best ways to train yourself for awareness, focused attention, and willpower, but it can be hard to integrate with your daily routine. 

Some people are already experimenting with devices to bring awareness to emotions.  The EmoBracelet (http://www.theage.com.au/technology/technology-news/gadget-warns-online-traders-of-dud-decisions-20091015-gy7d.html) was developed for stock traders to be mindful of stress and overexcitement.  The Noumic (http://www.noumic.com/) was developed for use as a meditation tool.  However, both of these use lights and/or sounds, which are disruptive and attract attention, and the Noumic is not wearable and must be held. 

I propose to build a subtle yet stylish bracelet that provides gentle reminders during your daily routine.  Embedded within the bracelet is a sensor to measure galvanic skin response.  Galvanic skin response is used to measure emotional arousal.  In response to strong emotions, the bracelet would become warm and give the user’s wrist a gentle hug to remind the user to be aware of her actions and mental state.  Galvanic skin response can show whether you are excited or stressed but it has trouble distinguishing between the two.  Though for other applications this can be a problem, mindfulness requires a non-judgemental approach and this would actually be a benefit in this application.  The user would be reminded to reflect during all kinds of heightened emotions, which is what mindfulness encourages.

I would find such a device useful within my own life.  For example, I am reading Flow by Mihály Csíkszentmihályi.  In this book, he describes methods for making our work feel more meaningful and increasing happiness through focused attention, challenging myself, etc. It is very hard to remember his methods when I am actually in the middle of my work and stressed out. What if the bracelet gave me a reminder through warmth and a gentle squeeze to think about these methods that I want to remember?  Also, affirmations (shown to be effective at increasing happiness, decreasing stress, and changing habits) and calming exercises are also hard to remember.  Could my bracelet detect my stress and then subtly remind me to relax? Could such a device be used to help during meditation? During meditation, it is extremely easy to let your mind wander. Could I use a sensor to detect my wandering attention and gently guide my thoughts back to my meditation?

To implement this this device, I would start simple, maybe just using heat. Flexible heat circuits are easy to get and could be sewn into fabric or cast into flexible silicon.  Galvanic skin response sensors are also available online, but I believe that this bracelet would be effective even if it only went off randomly.  Currently, I don’t know how I would implement the gentle squeeze functionality, but I think the bracelet would work with just heat.   For a device like this to be useful, it would have to have good battery life and recharging would have to be easy.  It would also have to be stylish and unobtrusive so the user wouldn’t mind wearing it.  Ideally, it would be nice to have a small screen on the bracelet so you can optionally use it to remember different things.  Also, it might be useful if the bracelet can be tied to other devices using bluetooth or some other type of wireless communication in order to be able to tie it in with different contexts and perhaps do some logging.

Through this project, I hope to demonstrate that I can increase the likelihood of changing my behavior, habits, and thought patterns. I believe that this will be effective even if the reminders are done randomly, reinforcing pathways and associations during all kinds of situations. I also hope that the bracelet can be so subtle that it can eventually trigger different thoughts and behavior even if the user isn’t fully conscious of the physical signal.

7 Cassandra

I was inspired by the Klopfer and Sheldon paper in which the authors transitioned students from playing a game about environmentalism to authoring their own environmentalism game. The act of having students develop their own game was valuable both for content creation specific to the students’ geographical location and as a learning process. I really admired this approach for its pragmatic value– fewer resources are required because students can be relied upon to generate content as part of the learning process!

This was the first time I have thought of user-based content generation from the context of learning, and I am excited about the scalability of the idea. In addition, to requiring fewer “expert” users and “teachers, I think that the idea may be powerful from a motivational perspective. Requiring users to generate content that will be used by other users, may motivate them to yield higher quality content. Beyond the environmentalism game example, I think the technique could be used more generally in the form of teaching. I frequently hear TAs or professors say that you really learn material when you have to teach it. What if we could apply the same concept to regular students? Imagine a student is learning a subject and given questions probing their understanding of the material in the traditional manner. Imagine that the student is told that with some X% probability, their answer will be given to another student learning the same material when the other student is stuck and, in keeping with the quiz show “Who Wants To Be a Millionaire”, phones a friend for help. In this scenario, the student may be motivated to learn the material more carefully for the sake of potentially helping a classmate down the road. The student may be more motivated in this group learning scenario than individualized learning. Humans are social beings after all, and user-based content generation during learning may be useful for more than content creation.

7 Anirudh

AR has tremendous potential- Games are themselves interesting because they demand critical thinking and involve the user emotionally due to games’ incentive or the mere thrill involved during the play. The idea of unfolding history/drama in the locations where it has actually occured is new and possible with AR. Though i believe joy of learning has got nothing to do with the complexity of technology componenets involved.

Though while designing such systems we’ve to be careful about over-bombarding the user with data/information more than the user can digest. Games esp. fall into such danger due to the short sensory time in making quick decisions.

The author’s ideas about using players as game characters is interesting. Though in a fast paced game it could be distracting, the same is more relavant when the user is involved in exploring an area and needs human contexts to it. e.g. overlaying ex-presidents while someone points at the white house. OR a game in which the player is contesting against his friends.

The overall work also reminds me of http://arisgames.org/demo/ done at U-Wisconsin

Seven – Dhairya

As opposed to full blown AR implementations like the upcoming glasses and the otherwise vision of full AR, it was a good break reading about something that was partial and location based AR games. Although AR was responsible for the overlay and display in the mobile games, I think the reason for their success is that the system is PLAYFUL! People actually go out and explore, emotions are evoked, people are engaged, in such a situation I feel even with limited AR and display in terms of mobile, the outdoor aspect of it more than compensates for it. There was also an interesting take in the other paper regarding ubiquitous learning – the benefits – especially the fact that the previous learning profile is always continued – the persistency and the situatedness of the aid. I am not fully convinced that a mobile is a good platform for learning, even for that matter the AR glasses – I mean, they can give you information – they can give you knowledge, facts – but so much of learning is doing, being, and more importantly SENSORY. Visual being just one of those senses. And the success of the apps mentioned in the reading is attributed to the multimodal nature of their use – although only the visuals were digital, rest was ANALOG.

My big takeaways from the reading was that PLAYFULNESS and MULTIMODAL (DIGITAL + ANALOG) play as much an important role in learning as JUST IN TIME and CONTEXTUAL.

Assignment 7 – Sophia

I thought the system that was developed for learning Japanese polite phrases was an extremely good example of just-in-time learning, and it made me think about what might be some other good applications.  Maybe a similar system could be used to help people give public presentations.  A wearable device that reminds the user of their notes on a particular slide might be a much less intrusive than the speaker reading their notes from a computer.  In addition, perhaps such a system could help pull up relevant notes and research during Q&A.  Both would help the user maintain their connection with the audience much better than having to look at things on a screen on the podium.  Also, maybe a system like this could help people in dating situations…helping them get through initial icebreaker questions, reminding them of the other person’s interests, etc.

The location-based AR games were also interesting.  I liked the idea of certain games being written for a specific geographical location, especially as tools to improve the landscape or to encourage people in those locations to get to know each other.   The article also mentioned games which get customized to whatever location you are in, and both of these types of AR games are discussed in great detail in the science fiction novel, Halting State, by Charles Stross.  In his novel, he wrote about an elaborate spy game that actually turned out to be a real branch of the  government secret service.  Players were unknowingly working for the government against actual terrorist threats.

Assignment 6 – Sophia

I very much enjoyed the reading on deschooling.  In my own experiences with school, I resented being told what I needed to learn and how fast to learn it.  I often wished I could move faster and have access to teachers that could teach me different subjects not available at my school.  I found myself ill-prepared for college (because I lacked certain knowledge that I needed) and even more ill-prepared for a role where I finally had to make choices independently (having had it drilled into me that my own intuition is inferior to the wishes of my teachers). I strongly believe that a system where students can request what they learn would be much better fostering genuine curiosity and autonomy. In the real world, someone will not be planning everything out for you, and being able to manage your own objectives is an invaluable skill.  On top of that, I think the world would just be more interesting with more diverse, interesting, and passionately curious people.

However, I am not sure how this system would prevent the “charlatans, demagogues, proselytizers, corrupt masters…” the author speaks out against.  Present in structured education, I believe they would still be present in a less structured way of learning.  Choosing teachers, students, or peers would be fraught with the same dangers as craigslist or online dating.  How do you know someone is not dangerous? How do you know if someone actually has the knowledge they claim to have? How do yo know if someone will be a valuable, responsible teacher and mentor? I don’t think the solution to this will be as simple as doing reviews, giving scores, or using some kind of point system.  While the internet makes information very easy to find, it is hard to know if it is reliable, and I think the same could happen with this more informal approach to teaching.

 
This leads me to think that maybe a hybrid approach might work best, a system that has a standardized core (that doesn’t take up all of the student’s time like it does now) but that allows much of a student’s education to be tailored to their own interests and goals using this less structured method.

Assignment 5 – Cassandra

Structuring Information for Peer-to-peer Learning

(aka. swapping out your brain for someone else’s)

Models

Google’s mission statement is to organize the world’s information, but rendering knowledge in searchable form is only part of that problem. The other part of the problem is to provide structure to information. As Ashby framed in the end of his 1956 book [1], intelligence amplification is mainly a “selection” problem. We are simply presented with much too information to utilize. Some of this content is wrong or low quality. In contrast, high quality content is both factually correct and reusable in the sense that one high quality idea can be used to beget many new secondary ideas. This type of high quality content with a large impact is a model. In fact, the objective of the entire scientific discipline attempts to take vast amounts of downstream observations and organize the observations in such a way as to be explained by or inferred from a minimal number of upstream models. The quality of a model is judged by both its ability to explain current observations as well as to make predictions for the future. The structure provided by these upstream higher order models (and not the downstream observations) comprise intelligence.

Some of Google’s information is indeed about these models, but it represents information about models equally and in the same fashion as other information. It may be of interest to instead treat models as a separate and more powerful class of knowledge when indexing world knowledge.

Life Models

There are many classes of models. Scientific models are the subject of academic research. There are also implicit life models that each individual assumes. These individual models can arise from (1) conscious deliberation, or they might simply be (2) implicit habits picked out through life. Indeed, a considerable part of variance in cognitive intelligence seems to arise from a person’s deliberate selection of a world model and their daily habit patterns. These two types of models, conscious models and habitual defaults, may serve as vessels for transferring intelligence models from person to person.

As a user scenario, imagine an application for a wearable like Glass that lets you “wear someone’s habits”. If you find a daily routine that works well for you, you can share it as an effective habit schedule for other people to try. Alternatively, you can try out the habit schedules of others by downloading them and having Glass prompt you with just-in-time situational alerts for executing a particular habit. This device would allow you to be more productive by emulating people you respect. Or it might let you “live the life of someone else” as an interesting costume change. Or it might serve as an therapeutic mechanism for breaking out of daily habit loops.

This type of device also has interesting implications for collective intelligence. By crowd-sourcing questions to such as “what are the most effective patterns for daily life?” and “which belief frameworks are powerful?”, we can curate a haul of answers from the world’s population. Each person on the planet has either explicitly or implicitly reached a personal answer to these questions. Our only task is to render their answers into forms easily adoptable by other persons.

The task can be phrased in a variety of ways. How can we share our individual brain structure with others? How can we share our world models with others? How can we share our patterns with others? Two different approaches are proposed in the next section.

Implementation

a) The Logical Approach

Perhaps the most obvious approach is to condense our individual philosophies into their purest most form and explicitly express them. In this case, the technical challenge is to design simple descriptive languages [__] for models of this type. Interested parties may wish to swap out their current model for a new one, consciously and deliberately reprogramming themselves.

Such a descriptive language for specifying world models will likely require both what and why components. The what-component can be modeled after rule-based logic systems [__] of the style “WHEN [EVENT A], DO [THING X]” The why-component that converts rules into reasons can be used for building hierarchical trees that can be further used converting reasons to higher-order value systems.

In a continuation of the habit-based example above, some may want to adopt the morning routine of Mr. Rogers, the beloved actor from the children’s show Mister Rogers’ Neighborhood. Mr. Rogers’ morning routine [4] can be summarized as “waking up at 5 a.m.; praying for a few hours for all of his friends and family; studying; writing, making calls and reaching out to every fan who took the time to write him; going for a morning swim; getting on a scale; then really starting his day”. The first few lines of a Mr. Rogers daily program could be written as:

(what) WHEN 5AM, DO wake up. % (why) “Because time is precious”

(what) WHEN DONE, DO pray for friends and family % (why) “Because I believe in God” AND “I love my friends and family”

Interchanging models for those of others will causes us to swap out our routinized defaults for a new set of defaults and will likely result in a cascade of effects. It may be particularly advantageous to replicate models of the people that you admire, for instance, an established researcher in your discipline. It may be fun to adopt the models of one of our ancestors, for instance, to live a day as your grandmother would and capture her behavioral and life lessons. It may be a literal way of experiencing “life in someone else’s shoes”, and feel more connected to either strangers or people that you know and care about.

b) The Chaos Approach

The alternative approach is to give up on rationalizing behavior and instead attempt to infer method from madness. Consider the following design of a file system designed for brainstorming. First note, that it is possible to infer the content of someone’s brain from her computer file system, particularly when the computer is used for storing and organizing ideas (as in the case of a researcher). It is possible to organize the file system in such a way that it can be used for brainstorming if specific structural rules are followed. My personal file system is organized in such a way to facilitate brainstorming by having two searchable folder ‘mentors’ and ‘me’. The ‘mentors’ folder contains plain text versions of the most inspiring research papers. The ‘me’ folder contains subfolders with plain text ideas of my own and plain text notes inspired by the content of presentations and meetings.

The logic of this organization is inspired by two research papers with powerful ideas: “What Would They Think” [2] and “Remembrance Agent” [3]. First, the idea of the Remembrance Agent to look at what you are reading and writing and propose past content based on related keywords. Second, the idea of What Would They Think is to show the affective reactions of respected mentors when reading new content. We can combine the two ideas to address Ashby’s “selection problem” identified in the first section of this paper– that a large component of intelligence involves the selection of the most useful information out of the large pool available. The proposed file system for brainstorming searches keywords against the ‘mentors’ and ‘me’ directories to bring up relevant material as well as near-miss material that are less relevant but serve to bring in tangential ideas.

The important idea here is not my file system itself but that if multiple users organize their file systems in the manner described, the file system becomes an external representation of an individual’s mind. File systems then become interchangeable in such a way that I can brainstorm with someone else’s personal ideas and mentor preferences. The file system brainstorm serves as an information dump of someone’s mind that has not been distilled into logical rules but instead persisted as the chaotic pile of quirks and nuances that embody ourselves.

Proposed Artifacts

To recap, the artifacts proposed within this paper were:

1. Wear someone else’s habits wearable for REPROGRAMMING SELF WITH SOMEONE ELSE’S HABITS
2. Remembrance Agent + What Would They Think file system for BRAINSTORMING WITH SOMEONE ELSE’S MIND

Conclusion

Both proposed systems propose the augmentation of intelligence facilitate the spread of powerful habits and ideas, giving us the framework to swap out our minds for the more preferred and intelligent minds of others. The proposal is only for systems to provide additional structure and transparency to knowledge that the world already has. Control remains firmly in the hands of each individual who has suddenly been endowed with the capability of self-intelligence augmentation.

References

1. Ashby, W. R. (1956). An introduction to cybernetics. Taylor & Francis.
2. Liu, H., & Maes, P. (2004, January). What would they think?: a computational model of attitudes. In Proceedings of the 9th international conference on Intelligent user interfaces (pp. 38-45).
3. Rhodes, B., & Starner, T. (1996, April). Remembrance Agent: A continuously running automated information retrieval system. In The Proceedings of The First International Conference on The Practical Application Of Intelligent Agents and Multi Agent Technology (pp. 487-495).
4. Hattikudur , Mangesh. 15 reasons Mr. Rogers was the best neighbor ever. http://edition.cnn.com/2008/LIVING/wayoflife/07/28/mf.mrrogers.neighbor/index.html

 [Slides from class]

Assignment 5 – Sophia

Good Eye

Teaching composition and design principles to artists and designers is difficult.  Some people have an inherent knack for composition, and, for others, it is learned.  In both cases, it develops only with much looking and practice, and it is not something which can be easily learned from a book. Traditionally, it is taught by showing slides of the work of other artists and designers and then by critiquing finished assignments. Creating a painting or making a design, however, is made up of a series of hundreds decisions where the artist/designer is working without guidance. Only after many hours of work, hundreds of steps later, might an artist receive any feedback on their work. Often the most paralyzing part of the process is staring at a blank or nearly blank page and not being able to determine what the next step should be. I propose to augment this process by building a tool which assists its user in developing a “good eye”.

 To do this, I suggest a system that works with the user during the process of developing a composition, aiding them in making all those small decisions as opposed to only receiving critique once a piece is finished.  I imagine that the user would place a shape on the canvas or screen, and the system would propose a series of possible next steps that the user could make on screen or projected directly onto the work.  Even just being able to see how a one choice might affect the rest of the composition without committing to it would very helpful because this is something that can be hard to do when using physical materials (cutting out shapes of paper, etc.). The possibilities would be varied in shape, value, texture, color, and line, which are the ways that an artist or design can influence the feeling of the final composition.  The user could cycle through the possibilities and choose what feels best.  The user could repeat this process for as long as it is helpful.  The way the system would make suggestions would be derived from an algorithmic analysis of a bank of images of the work of other artists and designers.

I see a few different use cases for such a system.  I imagine that this could be very useful for teachers of art and design.  Teachers commonly tell students to study the work of artists before or after giving an assignment, but this sort of system would help the student during the process of making a work.  A teacher could assign a set of images to feed to her students’ systems to hone certain skills.  In addition, a teacher could tailor the input to each student, diagnosing weaknesses in the student’s design sense and prescribe artists that would help to counteract those weaknesses.  I also see such a system helping artists and designers working alone.  The user could teach themselves by inputting the works of artists they admire, and using the system would help them understand what it is about those works that is so appealing and successful.  Furthermore, the images fed to the system need not even be works of art but any source of inspiration, any image, and the system could become a new way of making work in general.

A system like this could also be easily adapted to record how an artist arrives at a composition and then play back those steps to someone else.  This would be almost like a paint by number but with an emphasis on the process of arriving at a composition and not just reproducing the final result (in which it can be very difficult to understand how the artist/designer got there), and it could also be a very useful learning tool.

When working digitally, something like this could be built directly into the graphics software. For sketching and painting, a camera could record the work in progress and the suggestions could be projected onto the work itself or shown on a screen on top of an image of the work.  Ideally, the user would be able to see the suggestions directly on their work and not on a separate screen.  Using projection would be problematic because projected colors do not have the same quality as those made with real materials, which will affect the user’s ability to weigh out their options effectively.  Also, artists need to work in bright light, and projections would not work as well in those conditions.  Projecting onto already colored surfaces also would also be a problem because the colors would mix and shapes would overlay and not occlude each other.  Many of these problems might be avoided by using AR glasses.

Such a system would also be useful for learning to arrange 3d spaces, making sculptures, etc. but this use case would likely have a confusing UI and be hard to implement.  Starting out, the algorithm for making suggestions would likely be very primitive and might not always analyze the input images correctly.  Understanding that one shape occludes another and that they are not just two shapes side by side also seems like something that would be hard to implement in the algorithm.  In addition, often good compositions are nuanced ones, and it would be hard for a computer to understand these nuances.  However, despite these limitations, such a system could still be very useful.

That this system might train artists and designers to all work in a similar way is a valid concern, but I would argue that this is no different than using any tool or technology (paint, a pencil, Photoshop, etc.), which also imposes constraints on how to think and work. By being able to iterate through the design choices more easily and reversibly, users could develop a “good eye” more quickly.  This system would also help the user learn from someone else’s working process, which is hard to do now.  I imagine this could be used in class, but that it would be particularly useful as a way for people to teach themselves when they do not have access to expensive art and design schools.  Furthermore, I believe that such a tool will help its user find her unique voice as an artist much more quickly.  As the user trains the system, she will discover what she likes and define her own style. Finally, such a system might even grow to be more than a learning tool and become a new method for creating works in ways I cannot even predict.

Slides