Human Computer Interface: Revolution, Revolution, Revolution.

In der Softwareentwicklung heißt es eigentlich immer nur GUI (Graphical User Interface), gesprochen „Guuuuuhi“ wenn man Bedienelemente für Software entwickelt. Doch seit das Multitouchdisplay von Jefferson Y. Han im Februar letzten Jahres auf der TED Conference vorgestellt wurde, und er die ersten Displays davon an das US Militär verkauft hat, geht die Entwicklung in riesigen Schritten voran und das GUI vverändert sich. So werden Menüs jetzt durch eine Handbewegungsgeste statt Rechtsklick erzeugt und selbst statistische Daten kommen einem zum Greifen nahe. Vermutlich heißt es dann bald Manual Gesture Interface (MGI), um die Hand als Interface zu beschreiben.

Hier klicken für Film 1 (from Wallstreet Journal) und hier für Film 2 (YouTUBE)
Bild anklicken für große Darstellung.

Minority Report Videos (vom hallo user blog)

• Video 1
• Video 2
• Video 3

Das alles ist nicht mehr evolutionär sondern revolutionär, weil zugleich alte Bedienkonzepte vollständig über Bord geworfen werden. „Endlich keine doofe Maus mehr!“ möchte man da erleichtert rufen und denkt als „Hacker“ insgeheim an weniger Peritendinitis durch Mausgebrauch. Eine weitere, neue, grandiose Vorstellung des Human Computer Interface der Zukunft gibt es in zwei recht neuen Videos zu sehen. Foto davon siehe oben im Vergleich mit Fiction-Displays; ganz rechts das reale Display (Beitrag gefunden via netzlernen.ch).

Der „Brightcove“-Film, der auch auf der Hauptwebseite der Firma Perceptive Pixel zu sehen ist zeigt aber nicht den blick hinter die Kulissen. Wer sich für die echten Details dieser Technologie namens FTIR interessiert, der muss eine andere Webseite aufsuchen: Nämlich diese hier! Das ist quasi die Bauanleitung für die MEMEX direkt vor den eigenen Augen. Die Beschreibung des Verfahrens ist simpel:

…multi-touch sensing enables a user to interact with a system with more than one finger at a time, as in chording and bi-manual operations. Such sensing devices are inherently also able to accommodate multiple users simultaneously, which is especially useful for larger interaction scenarios such as interactive walls and tabletops.

Der erste Film davon wurde hier gezeigt. Echte Hintergrundinfo zu dem Frustrated Total Internal Reflection Verfahren gibt es dort auch. „Star Trek“ und „Minority Report“ waren gestern würde ich sagen.

Update 24.6.2007
Wie sehr Multitouch-Technologie das HCI verändert zeigt auf eindrucksvolle Weise der neuste Demo-Film über das iPhone (Teil 1), (Teil 2) von Apple. Gesten wie „Durchstreichen“, um Einträge im Adressbuch zu löschen sind einfach intuitiv! Aus Mangel an Platz für viele Fenster, werden planare Flächen „übereinander“ gelegt, wenn man andere Funktionen aufruft. Auch der gute alte Breadcrumb-Trail, den wir schon von Hänsel und Gretel kennen kommt zum Einsatz. Das beste jedoch ist, ein riesiger Störfaktor bei der Kommunikation mit dem Gerät fällt weg: Das Pointing-Device. Der Finger hat keine technischen Probleme, navigiert schnell und präzise. Die Vision von Bill Gates „Information At Your Fingertips“, die er in seiner Keynote Speech auf der COMDEX Keynote Speech auf der COMDEX in Las Vegas/Nevada am 14. November 1994 hielt, wird nun verwirklicht durch andere geniale Erfinder, wie Jeff Han einerseits und Firmen wie Apple Inc. andererseits.

Information At Your Fingertips (Full Transcript + Video)

Update 8.7.2013: I updated this part of my post after I found out that the link to the transcript did not work anymore. As a consequence I added the full copy of the transcript here, plus a small video snippet and the full speech video at the bottom. Please recognize, that Microsoft has also a page where it hosts this speech on there own servers. Another link

Information At Your Fingertips – 2005
Bill Gates’ Keynote Speech Fall/COMDEX
Alladin Hotel, Las Vegas, Nevada Nov. 14, 1994
Transcript

It’s great to be here at Comdex. It’s a chance for us every year to get together and see all the change that’s been taking place. With this fast moving industry the pace of change has never been greater. In fact, we can find change everywhere today. With last Tuesday’s election, even Bill Clinton and the democrats are facing some change. The changes now are being discussed not simply in the context of data processing or the computer industry but in a much broader context. In the context of how we communicate, how we organize, how we educate, how we run business. The changes that would be brought by this will go well beyond the group we find here. However, this industry will be at the center of all of it. With software innovation and competitive hardware, we’ll be shaping this so-called information highway.

Now, this next era that we’re moving into is quite different than the original PC era. That one was done simply by having a few companies that believed–spring it on the world–and it was a complete surprise. Here we have thousands of companies and it’s talked about all the time. You can hardly pick up a magazine without reading about what’s going on here. It’s often cast in the light of who’s doing a deal with who. What companies are getting together. Who’s getting ahead. Who’s winning and who’s losing. I don’t think that’s really the context that helps us understand where we ought to go.

I think the best approach is to really look at what will it be like once technology is pervasive. What will the benefits be? Then step back from there to see how we can get out to that. I decided to take this year’s speech and dedicate it to that vision, to showing those scenarios. That means I’m going to go this full hour without mentioning a single Microsoft product–if I can control myself, that is. Really just look out ahead. I did do this one other time in 1990 when I introduced the original “Information at Your Fingertips” concept. There I included some fun scenarios where people were doing a lot of things. Most of what I showed in that presentation has already come true today: object orientation, document centricity, fax/ mail integration, improved ease of use. One notable thing I had in there that has not taken place is pen computing, the tablet computer where signatures and handwritten notes were being recognized. I still believe very much that that will happen, and will be important, but that’s one case where the time frame wasn’t exactly right.

The last four years have seen very substantial growth in this business. Every year the increase is fantastic because we’re at a selling rate worldwide of over 40 million PCs a year. Many people have predicted that we had reached saturation. That there wasn’t the opportunity to keep going. Yet, whether it’s in business or home or in education, the growth has been there and I expect it to continue. It is a very healthy situation. Lots of winners and losers, but an industry that’s not only growing from a revenue point of view, which we see here, but also from a capability perspective. As prices are coming down, computers and software are having a broader and broader impact, empowering people with very rich information tools. Key to this growth is the pace of innovation. One way that we can see that, that’s most dramatic, is by looking at what’s happened to the microprocessor.

Going back to the original PC we see that the 8088 had .33 MIPS of capability. It only let us do a very limited number of things. With the 286 we got enough power to move up and do applications that had never been done on 8 bit PCs. The 386 that came in 1986 for the first time allowed us to do graphical interface and that’s when Microsoft and many companies put out graphical-type products which have now moved into the mainstream.

After the 386, we moved up to the 486, taking us to 20 MIPS. Here we started to see collaborative applications, sharing of rich information between applications. Machines that could really run those applications together and give you substantial benefits. Now this year we’ve seen the emergence of the latest generation, the Pentium, that takes us over 100 MIPS. That pace of improvement is exponential improvement. It’s not just a linear increase, it’s exponential. It’s very hard for us to appreciate what that means. In fact, this pace is going to continue. So as we look out ahead it’s hard to say specifically what the performance will be, but there will be incredible performance that applications will be able to take advantage of. We’ll use that to the benefit of the user. To search out the important information. To provide new interface techniques.

There is no slowing as we look at the next decade in terms of the processor speed or memory size or storage capabilities of these systems. It’s really up to us to think, how do we want to take advantage of that. Of course, at the center of this will be the idea of digital convergence. That is, taking all the information–books, catalogs, shopping approaches, professional advice, art, movies, and taking those things in their digital form, ones and zeroes, and being able to provide them on demand on a device looking like a TV, a small device you carry around or what the PC will evolve into. All of these form factors will count. But we’ll need to have a common architecture so we can take all the authoring, the work done to prepare this media and make it easily available to people using these different devices. So this is what we’re reading about all the time, with all the different stories.

The communications companies have to think about this because it is their future. Media companies, from TV to cable, to Hollywood studios, need to get involved because it is their future. The consumer electronics industry is coming into this and will need to be one of the ones building these devices. Certainly, for the PC industry, this is what it’s all about. Lots of advances, lots of things that we can draw on to have growth in years ahead. So even at 40 million units a year we still have lots of frontiers to conquer and lots of impact.

This video I put together relates to the title of today’s speech which is Information At Your Fingertips 2005. I picked that year not because that’s when you’ll finally start to see these things happening. I picked it because I think that’s a time frame where these devices will be pervasive. That people will expect them. There will be everyone using them. Small businesses, consumers of all types will have them. The critical mass of information will be there to make them very straightforward to work with and get a lot of power out of them. This 2005 is a decade away.

I think there’s a lot of people who are going to say that I’ve been too conservative here because, in fact, I’m not showing any technologies that need to be invented anew. Everything here you can find in some limited form already. Some people will criticize this as being too far out, too futuristic. As long as I get a balance of people criticizing it equally on both counts I’ll feel like I’ve hit the right point.

If we look to our past, our belief in graphical interface, which we announced in 1983, it took seven years to happen. Our belief in CD-ROM technology, where we held our first conference talking about that in 1986, it’s only this year that you can say that’s taken place. So we’re often optimistic. But I think the key here is not so much the specific year but having the scenarios in mind and having an agreement as an industry on what the opportunities look like. Let’s go ahead and start the little story that I’ve prepared.

Video of Bill’s speech (part 1):

Video of Bill’s speech (part 2):

Update 12.7.2007
Ein neues Interview mit Jeff Han ist bei Technology Review erschienen (via Tim SchlotfeldtBruysten) Wenn man nach dem Video einfach weiterguckt, bekommt man weitere visionäre Ideen auf den Screen gezaubert.

Update 31.7.2007
Eine konsequente Weiterentwicklung der neuen Displays würde ich mir im Sinne von StarTrek TNG wünschen als Personal Access Display Device kurz PADD genannt. Als universelles Gerät zum Lesen und Schreiben. Folgende Abbildung zeigt die PADDs der VOYAGER Series. Weitere nette Seiten zu StarTrek Interfaces bzw. Treknology gibt es hier.

Update 13.12.2007
Eine echte Lösung für das Touch-interface „for the Rest of us“ ist das folgende (Wii-Remote + IR-emittierender Stift) von
Johnny Chung Lee:

(via Martin Ebner / youtube)

Update 23.11.2008
Statt mit Touch-Interfaces, sind auch noch ganz andere Interfaces derzeit in Entwicklung. Eines davon ist ein Gesten-basiertes Interface namens g-speak der Firma oblong, bei dem man einzig mit Handbewegungen wie im Film Minority Report den übergroßen Bildschirm steuert.

g-speak overview 1828121108 from john underkoffler on Vimeo.

g-speak overview 1828121108 from john underkoffler on Vimeo. (via Timo Heuer)

Aber das ist lange nicht das „Ende der Fahnenstange“ in Sachen Interfaces. Wie wäre es zum Beispiel mit einem Scratch-Interface wie es Johnny Lee nachfolgend demonstriert?

Danke an Thomas für den Tipp!

und letztlich bewegen wir uns gerade mit riesigen Schritten auf das Brain Computer Interface (BCI) zu, wie man in meinem Podcast zum Gehirn (Thetarock’r 04) nachlesen kann.

Update 11.1.2009
Weitere marktfähige Lösung das iTablet von PQ Labs.
https://www.holoboard.ai/

Quelle: crunchgear
Weitere Details beim Hersteller

Update 7.7.2013
Ist schon eine Weile her, da habe ich Oblivion im Kino gesehen. Den muss man übrigens im Kino gucken, so bildgewaltig wie der ist. Und der Film hat ähnlich wie Avatar krasse Userinterfaces gezeigt. Der reinste Augenschmaus und Anregung genug für eigene interfaces. Hier ist ein Zusammenschnitt einer ganzen Menge UI Einstellungen. Aus dem Film. Gefunden hab ich das bei Stanislav, meinem ESA Kollegen.

OBLIVION GFX Montage from GMUNK on Vimeo.

Avatar – Aufbruch nach Pandora

In Avatar gab es diese ultracoolen Holoscreens die sich mitten in den Raum projezierten. Hier gibt es einen Beitrag dazu mit dem Titel „CG arena : Prime Focus Adds Graphic Dimension to ‘Avatar’“ in dem bissel der Hintergrund der Effekte erklärt wird. Hinter vielen coolenn Effekten steht die Firma Prime Focus.

Source: http://www.cgarena.com/archives/news/newsimages/avatar2.jpg; Holoscreen in Avatar

Hier gibt es mal einen Zusammenschnitt der coolsten Effekte in Hollywoodfilmen.

Fantasy User Interfaces

Update 9.7.2013: Today another site with fancy user interfaces crossed my road on twitter: KIT Fui via @daringfireball.

Fantasy User Interfaces, Fictional User Interfaces, Fake User Interfaces, Futuristic User Interfaces. Regardless of what the F stands for, they all represent the same thing, the user interfaces and heads up displays found in many popular movies and television shows.

See what Microsoft produced again „Productivity Future Vision (2011)“:
Link to Website (Link)