MAS 962: Interaction Techniques for Virtual Environments

Tuesday, November 10, 1998

Sensing vs. Measuring: Any sense using computer vision?

INTRO

Let's go back to sensing, measuring, perceiving

Talked about sensing as being able to perceive with one measurement

Joe Paradiso - direct mapping of input (measurements) to output (music, etc.)

• measuring techniques
• electric fields
• gyros
  

If you can build these relatively complex systems for sensing and can map these to desired output WHY would you want to deal with vision?

Try to answer that today

USING COMPUTER VISION

What, How, When Might You Use Computer Vision?

• Recognition
  • absolute: who you are in the whole world; can be done unobtrusively so people don't know it's being done
    • drivers licensing
    • welfare fraud
    • FBI
    • immigration
  • relative: who you are in the current interaction; keeping track of where you've been or what you've seen
    • museum
    • brain opera
    • tutorials
  • relative easier than absolute because:
    • more reliable (set is sparser)
    • picture of person is current
    • no giant database
• People Detection (vs. detection of movement) - can reasonably tell if it's a person, not a dog, not something falling
• "Color" - easy to measure visual property; more accurate depiction of environment
• Facial Expressions - are they surprised, bored, etc.
• Direction of Attention - also perceive user's attention
• Gesture Recognition
  • American Sign Language
  • iconic gesture interface - define gestures; design non-confusing language
  • Wall Street trades
  • disabled people
• Motion

Why Would You Use Computer Vision?

• Don't want electronics on the person - desire distal measurement, remote sensing
  • intrusive/intimidating
  • unencumbered
  • awareness/magic/suspension of disbelief
• Variable resolution
• Non-complicity - user doesn't have to comply for system to work
• Common "vocabulary" of sensing between system and user

Warning - High Expectations

• system should just "see"
• system that knows I'm happy sometimes; expect it to know all the time
• brain opera not a familiar concept - no expectations

RECOGNITION OF HUMAN MOVEMENT - Prof. Bobick's slides

(Much of the information from Prof. Bobick's slides and presentation can be found on Jim Davis' and Aaron Bobick's web page, Appearance Based Motion Recognition of Human Actions, additional information on the projects can be found through links from Jim Davis' home page)

motion - video - compare one frame to next

Building Blocks

• movement - primitive motion
  • atomic, indivisible
  • example: batting (single swinging motion)
• activities
  • sequence of movements
  • example: pitching (wind-up, leg kick, forward motion, ball release)
• actions
  • semantic labels, higher level labels
  • example: tagging out a runner, a football play (requires knowledge of context and "meaning" of movements)

Motion Energy Images - MEI

Highlight regions in which any form of motion was present

Motion History Images - MHI

Show regions where movement has occurred in such a way that intensity is a function of recency of motion

STAR TREK DOORS

From previous weeks' discussion: how do we build Star Trek doors that open automatically just in time for someone to pass through

• Coarse floor sensor, pressure sensor
  • negative: if facing away from door and rock back, this will open falsely)
• Add direction via Joe Paradiso
  • com badges
  • shoes
  • negative: requires compliance, doesn't work for non-Star Fleet
• Pressure image
  • better floor sensing
  • negative: long cue time
• Face facing door via vision
  • negative: false alarms, "where is he"
• IR - heat proximity
• Looming detection via vision (method used by fly in landing)
  • calculate time to impact
  • when is as important as if

Why not just say "door open"; we're just talking about a fancy doorknob

What's the loss to an interactive system of NOT having sentient doors?

• there would be no sentient doors
• info is power (info from doors could be used for other things)
• no magic
• cognitive load

Invisible technology is cool

Perception is that Star Trek doors are just next step up from grocery store doors - easily made - not so

Back to Course Home Page