Thought: The Future of Mobile and Embedded Application Input

Information about Thought: The Future of Mobile and Embedded Application Input

Published on June 29, 2015

Author: TechWellPresentations

Source: slideshare.net

Content

1.   K4 Keynote   4/16/2015  9:45  AM         "Thought: The Future of Mobile and Embedded Application Input"   Presented by: Jim McKeeth Embarcadero Technologies           Brought  to  you  by:         340  Corporate  Way,  Suite  300,  Orange  Park,  FL  32073   888-­‐268-­‐8770  ·∙  904-­‐278-­‐0524  ·∙  [email protected]  ·∙  www.sqe.com

2. Jim McKeeth Embarcadero Technologies As lead developer evangelist at Embarcadero Technologies, Jim McKeeth is a key part of Embarcadero’s developer community outreach. With more than twenty years of programming experience, Jim travels the world speaking at conferences and sharing his excitement and knowledge. He holds a patent for the swipe to unlock and pattern unlocks used on both iPhone and Android phones, plus a number of other computer- and software-related patents. When not traveling, Jim is an improvisational performer with ComedySportz Boise and enjoys spending time with his family.  

3. 4/6/15   1   Thought: The Future of Mobile and Embedded Application Input JIM MCKEETH EMBARCADERO TECHNOLOGIES [email protected] @JIMMCKEETH ‹#› About Jim McKeeth •  Lead World Wide Developer Evangelist •  Host of Podcast at Delphi.org •  Longtime developer ◦  Object Pascal, Java, C#, JavaScript, Objective-C, etc. •  Invented and patented swipe to unlock in 2000 ◦  US Patent # 8352745 & 6766456, etc. •  Improvisational performer with ComedySportz Boise •  I am not a neuroscientist, but I find this all fascinating! 2  

4. 4/6/15   2   ‹#› 3   • Founded 1993 • Provides Architect, Database and Developer Tools • 3.2 Million Customers including 97% of Fortune 2000 • 35+ Product Awards – Consistent Innovation • 500+ Employees in 29 Countries • www.embarcadero.com Embarcadero Technologies ‹#› Agenda • The Science of Reading the Brain • The Science of Direct Brain Input • Consumer Brain-Computer Interfaces • Demonstration - Volunteer flying the Quadricopter • Summary & MindTunes 4  Images  from  Wikimedia  Commons  or  Product  websites.

5. 4/6/15   3   ‹#› “In five years, science fiction will be real!” 5   -Claus Torp Jensen (@ClausTorpJensen) Senior Technical Staff Member and Chief Architect API Economy at IBM (@ibmapimgt) What will the world be like in five years? ‹#› Brain-Computer Interface 6  ST:TNG  Episode  93  (4x19):  The  Nth  Degree  (1991)    

6. 4/6/15   4   Reading the Brain The Science of ‹#› Polygraph (aka Lie Detector) • Measures physical manifestations of stress that come with lying • Blood pressure, pulse, respiration, skin conductivity, etc. • Looks for changes. • Requires specialized training. • Invented in 1921 8  

7. 4/6/15   5   ‹#› Heart Rate Monitors • Bluetooth GATT Profiles supports all Bluetooth Smart Heart Rate Monitors. • Your app doesn’t need to support individual heart rate monitors. • Assigned Number: 0x180D 9   https://developer.bluetooth.org/TechnologyOverview/Pages/HRP.aspx https://developer.bluetooth.org/gatt/profiles/Pages/ProfilesHome.aspx ‹#› Magnetic Resonance Imaging (MRI) •  Uses oscillating magnetic fields at different resonant frequencies. •  Functional MRI (fMRI) uses software to analyze images. •  Shows activity (blood flow) in different parts of the brain. •  Combined with brain mapping can reveal thought information. 10  

8. 4/6/15   6   ‹#› Electroencephalography (EEG) • Measures electrical activity along the scalp. • Detects voltage fluctuations resulting from ionic current flows within neurons of brain. • Millisecond-range resolution (faster than CT or MRI) • Typical clinical usage has 19+ input electrodes as well as ground and reference. • Invented between 1875 and 1924 11   ‹#› Magnetoencephalography (MEG) 12   Uses Arrays of SQUIDs (Superconducting QUantum Interference Devices)

9. 4/6/15   7   Direct Brain Input The Science of ‹#› Change Perception of Reality 14   The  “headjack”  from  The  Matrix  movies  -­‐  1999  

10. 4/6/15   8   ‹#› Transcranial Direct-Current Stimulation (tDCS) • Constant, low current delivered via electrodes. • Results: ◦ Increase cognitive performance ◦ Enhance language & math ability ◦ Improve attention span ◦ Aid problem solving, memory & coordination • Recent DIY “Brain Hacking” Craze ◦ http://www.reddit.com/r/tDCS 15   Accelerated Learning tdcsplacements.com http://thebrainstimulator.net/ ‹#› tDCS with Mobile • foc.us just launched new tDCS headsets with Bluetooth API • Targeting athletes and gamers • Developer devices available • http://www.foc.us/index.php/v2-tdcs- developer-edition • http://www.foc.us/bluetooth-api 16  

11. 4/6/15   9   ‹#› Transcranial Magnetic Stimulation (TMS) • Similar strength to MRI • Magnetic induction of electrical currents • Used to treat neurological conditions • Produces: ◦ Flashes of lights in vision (phosphene) ◦ Muscular activity 17   ‹#› Ultrasonic Transcranial Stimulation Device • 2000 patent from Sony (US6584357 B1) ◦ Finer targeting than TMS ◦ Theoretically invoking sensory experiences ◦ Continued research and patents (US7350522 B2) 18   hHp://www.newscienLst.com/arLcle/mg18624944.600       Speculative work, “There were not any experiments done.”

12. 4/6/15   10   ‹#› Ultrasonic Neuromodulation for Spatial Resolution •  Virginia Tech Carilion Research Institute study published in Jan 2014 Nature Neuroscience •  Used “targeted ultrasound” to boost “sensory perception” •  Targeted an area of the brain as small as the size of an “M&M” 19  hHp://www.eurekalert.org/pub_releases/2014-­‐01/vt-­‐udt011014.php     ‹#› 20   Using More than 10% of Our Brain? LUCY2014 We all use 100% of our Brain Capacity

13. 4/6/15   11   Interesting Milestones Science Fiction Becoming Reality ‹#› Brain-to-Brain Input 22  The  “Hat”  from  Brainstorm  -­‐  1983  

14. 4/6/15   12   ‹#› Brain-to-Brain Input at University of Washington in 2013 • Input from Computer • EEG from brain • Transmitted over campus internet • Replayed through TMS to another brain • Output to computer 23  More information http://bit.ly/EEG2TMS & http://bit.ly/bci2cbi ‹#› See What You See - Hear What You Hear 24  The  “Monitor”  from  Ender’s  Game  –  2013  

15. 4/6/15   13   ‹#› See What You See •  In 2010 a team lead by Shinji Nishimoto partially reconstructed YouTube videos based on brain recordings. •  Used fMRI •  Overlaid similar videos to create composite video. •  http://youtu.be/nsjDnYxJ0bo & http://youtu.be/EdGjB6XCXOA 25  http://bit.ly/ReconstructingVisual ‹#› Hear What You Hear • 2012 - UC Berkeley & Brian Pasley decoded internal neural processing of auditory information. • Reconstructed sound on computer from electrical signals in observer’s brains. • Resulting sound was recognizable when compared to original. 26  hHp://bit.ly/ReconstrucLngSpeech    

16. 4/6/15   14   ‹#› Data Entry via BCI 27  ST:TNG  Episode  93  (4x19):  The  Nth  Degree  (1991)     ‹#› Typing via BCI • Brain researcher Jonathan Wolpaw at Wadsworth Center in Albany, NY • Wear EEG while looking at a flashing matrix of characters • Each character flashes with a different pattern • ~15 seconds to type a character • The Verge http://embt.co/typebci 28   .! .! .! .! .! .! .! .! .! A! B" C" D" E" F" .! .! G" H" I" J" K! L" .! .! M" N" O! P" Q" R" .! .! S" T" U" V" W! X" .! .! Y" Z! 1" 2" 3" 4" .! .! 5" 6" 7" 8" 9" 0! .! .! .! .! .! .! .! .! .!

17. 4/6/15   15   ‹#› Uploading Your Consciousness 29  Transcendence  -­‐  2014     ‹#› Human Connectome Project • Map of neural connection the brain • The neurological version of the genome • Unique to each person, changes over time • Representation of memories, thoughts, etc. • www.ted.com/talks/sebastian_seung • humanconnectomeproject.org • humanconnectome.org 30   Image  by  Xavier  Gigandet  et.  al.  

18. 4/6/15   16   ‹#› OpenWorm Project • Mapped the 300+ neurons and 7000+ connections of a worm (C. elegans) • Created a software model of connectome • Connected to LEGO robotic representation • Behaved like worm (without training or programming) • http://bit.ly/openwormlego & http://www.openworm.org/ 31   ‹#› As of July 2014" Wearable User Interfaces" Brain-Computer Interface" Internet of Things" Autonomous Vehicles" expectations! Speech-to-Speech Translation" 32  h:p://www.gartner.com/newsroom/id/2819918 time! Plateau of! Productivity!Slope of Enlightenment! Trough of! Disillusionment! Peak of! Inflated! Expectations! Innovation! Trigger! Plateau will be reached in:! more than 10 years"5 to 10 years"2 to 5 years"less than 2 years" obsolete" before plateau" Gartner's 2014 Hype Cycle for Emerging Technologies August 11, 2014 BCI   Wearable  User     Interface   Hype Cycle Autonomous   Vehicles  

19. 4/6/15   17   Brain-Computer Interfaces Read-Only Consumer ‹#› 34   •  EEG with 1 dry electrode •  Measures “attention,” “meditation” and eye-blinks •  Bluetooth communication •  SDK for iOS, Android, PC & Mac •  $129.99 to $79.95 retail neurosky.com •  Hacking: ◦  http://makezine.com/2011/05/10/behind-the-brain-blinker/ ◦  http://hackaday.com/2012/12/20/modifying-an-eeg-headset-for- lucid-dreaming/ NeuroSky’s MindWave (also in XWave headsets)

20. 4/6/15   18   ‹#› Emotiv EPOC •  16 wet electrodes ◦  14 EEG electrodes – read brain waves + 2 reference electrodes •  Gyroscope to read head movements •  4 mental states, 13 conscious thoughts & facial expressions – 4 Processing suites •  Wireless (encrypted) connection to Windows, Linux or Mac •  $399 desktop or $499 for Bluetooth Smart (coming soon) •  www.emotiv.com 35   ‹#› InteraXon’s Muse •  7 dry EEG sensors via Bluetooth •  Focuses on “brain fitness” •  4 hour battery •  Basic Mac & Windows SDK now, full SDK coming soon (incl. iOS & Android) •  Works with Windows, OS X, iOS, Android, etc. •  $299 available now •  choosemuse.com 36  

21. 4/6/15   19   ‹#› OpenBCI •  Open source hardware & software •  Bluetooth, Arduino, EEG •  8 wet Electrodes per board (can daisy chain boards) •  Full open EEG & BCI platform •  www.openbci.com starting at $449 •  No iOS or Android yet 37   ‹#› Emotiv Insight •  Expected eventually •  Starting at $299 •  5 dry sensors + 2 reference •  Bluetooth 4.0 LE (Smart) •  Battery Life: 4+ hours •  SDK: Android, iOS, Mac, Linux and Windows Platforms 38  www.emoLv.com  -­‐  Comparison  hHp://bit.ly/1cfliTn    

22. 4/6/15   20   ‹#› No More Woof (“work in progress”) • EEG for Dogs • Recognize brain patterns and match with mental states: • Hungry, Playful, Curious, etc. • Translates to human • By Nordic Society for Invention and Discovery (NSID) • Raised $22K on IndiGoGo 39  http://www.nordicinvention.com/nmw.html Emotiv EPOC More Details on the

23. 4/6/15   21   ‹#› Expressiv Suite •  Detect facial expressions ◦  Eyelid & eyebrow positions –  Wink, blink, furrow ◦  Horizontal eye movement ◦  Smile, laugh, clenching & smirking •  EEG sensors picking up signals to muscles (not brain waves) •  Very fast (10ms) 41   ‹#› Uses for Facial Detection • Realistic avatar expressions • Anything requiring fast input • Facial gesture detection • Basic eye tracking • Fast input & responses 42  

24. 4/6/15   22   ‹#› Affectiv Suite •  Detect emotions / mental states ◦  Excitement ◦  Engagement or Boredom ◦  Meditation ◦  Frustration •  Passive detection •  Variable levels of each •  Short & long term tracking 43   ‹#› Uses for Emotion Tracking • Realtime media evaluation • Stress analysis and management • Mood monitoring • For more accuracy combine with other data sources • Eye tracking, heart rate, etc. 44  

25. 4/6/15   23   ‹#› Cognitiv Suite •  Detect conscious thoughts •  Requires training •  Detect 13 thought patterns with intensity level •  Left, right, up, down, forward, pull •  CW, CCW, left, right, sway-back, sway-forward, Disappear •  Not as fast as Expresiv 45   ‹#› Uses for Cognitive Thought Identification • Whole new input method • Variable intensity for fine control • Observe subconscious responses • More training improves accuracy • Improves the model • Trains your brain to focus 46  

26. 4/6/15   24   ‹#› The Pieces 47   Emotiv EPOC Brain-Computer Interface Google Glass Wearable Computer Parrot.AR Drone 2 Simi-Autonomous ‹#› About Appmethod •  Integrated Development Environment •  Multi-platform visual designer •  Native compiler for Win32, Win64, OS X, iOS & Android •  Framework for rapid prototyping and accelerated development with C++ or Object Pascal •  Architecture allowing shared code & UI across platforms •  Advanced language features like ARC, Generics, etc. 48   www.appmethod.com Starts at $25/mo per platform or C++ Android Phone for Free

27. 4/6/15   25   ‹#› Laptop   EDK   My   App   How it Works 49   Brain   Electrical   Signals   Wireless  Signal   AT*REF=1,290718208   Provides   WiFi     Hotspot   Feedback   Emotiv EPOC SDK + Parrot AR.Drone + Google Glass Demonstration Tweet @Appmethod and #IoT or G+, LinkedIn, Facebook, etc.

28. 4/6/15   26   ‹#› Emotiv EPOC SDK • Platforms ◦ Windows ◦ Mac OS X ◦ Ubuntu ◦ Fedora ◦ iOS & Android coming soon 51   • Languages ◦ C / C++ / Objective-C ◦ C# (.NET) ◦ Python ◦ Object Pascal / Delphi ◦ Java coming soon ‹#› MindTunes •  Three disabled musicians •  Collaborated with DJ Fresh and Julien Castet (tech) •  Created dance track •  http://youtu.be/HyF4ZxGhPHw •  Full documentary •  http://youtu.be/PgfxKZiSCDQ 52  

29. 4/6/15   27   ‹#› More Information •  Email: [email protected] •  Emotiv EPOC: www.emotiv.com •  Parrot AR.Drone: projects.ardrone.org •  Appmethod: www.appmethod.com •  My blog: delphi.org •  Full Slide Deck: http://www.slideshare.net/jimmckeeth/jim-mc-keeth- wearable-thought-input •  Code: https://github.com/jimmckeeth/Delphi-Emotiv-EPOC 53  

#iot presentations

Eica 2016.2 - Industria 4.0
03. 11. 2016
0 views

Eica 2016.2 - Industria 4.0

Related presentations


Other presentations created by TechWellPresentations

Exploring Usability Testing
30. 01. 2014
0 views

Exploring Usability Testing

Alan Page: On Testing
30. 01. 2014
0 views

Alan Page: On Testing

The Craft of Bug Investigation
30. 01. 2014
0 views

The Craft of Bug Investigation

Key Test Design Techniques
30. 01. 2014
0 views

Key Test Design Techniques

Why Software Drives Us Crazy
10. 07. 2014
0 views

Why Software Drives Us Crazy

Mobile App Testing Secrets
15. 07. 2014
0 views

Mobile App Testing Secrets