• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    October 2017
    M T W T F S S
    « Dec    
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    3031  
  • Archives

The Ergonomics Of Panoramic Interactions Continued

TouchShield SlideBruno’s comment about touch-screens got me thinking. While most users still interface with the computer via mouse, keyboard and a one-way display, things are going to change fast in the coming years. The old KVM (Keyboard/Video/Mouse) user interface is being replaced by more powerful and natural tools. The point&click / drag&drop metaphors popularized by Apple’s Macintosh since 1984 after the invention of the ball mouse at the Xerox Parc are due for an update. Ever smaller and powerful mobile devices, accelerometers, touch screens, 3D screens. How will they interface between the user and the VR Panorama?

The solutions I have observed so far simply hard wire the behavior of these new devices to the mouse. This is no different than my 1998 Wacom Tablet (which still works!). The simulation of the mouse limits the interaction designer to define the device in relationship to mouse behavior and either mimic it or its inverse.  After half a century it is time to break those limits; to look at the interactions anew and to design device/context specific metaphores; to mold the intearction around the human. I see a combination of relevant factors, including the device’s physical characteristics and the context in which it is used. A touch-screen on a desktop requires a different metaphore than one on a smartphone. And what to do when two competing input devices with conflicting metaphores are attached, such as an accelerometer (3D mouse) and a touch-screen?

For the desktop touch-screen and for the laptop touch-screen I tend to agree with Bruno that the Google StreetView metaphore is the way to go, at least until the computer can discern if the index finger is at a nearly perpendicular angle and fully straightened as in a pointing; or it has a smaller angle and a slightly more curved posture as in a natural dragging movement.

Things become more touchy (pun intended) with mobile devices which typically have an accelerometer and a touch-screen. Which one should drive the VR panorama interaction and how? To me the most natural would be to use the accelerometer and point the iPhone in the direction I want to see, but in some situation such explicit movements are embarassing, unconvenient, inappropriate, or all of the above; and the more discreet dragging by finger on the touch-screen is the right way to go.

Whichever it is, I think modern panoramic viewer should make provisions to accommodate both behaviors – dragging and pointing. Ideally the system would tell the VR player in what context it is playing and the VR player would adapt, using an appropriate metaphores. Currently the browser only let the VR player assume the presence of a pointing device, and the devices all interface by mimicking the mouse. In the current situation making the mouse behavior a parameter, as implemented in the KRpano viewer, is the best thing to do. When a reliable detection mechanism can tell the player what device is attached, the choice may be automated.

I look forward to see the results of León’s Google Summer of Code project adding QTVR playback and Wiimote interaction capabilities to the VLC media player. In the meantime I got help from the Liquidware guys in my still unsuccessful attempts to make their Antipasto Arduino IDE work on my Ubuntu notebook. The TouchShield Slide touch-screen rocks and I am keen to toy on new interfaces with it.

Accepted Students for Google Summer of Code 2009

MascotteThe dice have rolled. For the third year in a row the Hugin community will mentor five Google Summer of Code students as they go about writing free software:

  • James Legg of Fast Preview fame will tackle the Layout Panorama Model mentored by Bruno Postle, improving the handling of stacks.
  • Tim Nugent of Celeste fame will tackle Straight Line Detection for Lens Calibration mentored by Tom Sharpless, improving the handling of fisheye lenses.
  • Lukáš Jirkovský, too young in 2008, has become a significant contributor to our community over the past year and will tackle Ghost Removal for Enfuse, mentored by Andrew Mihal.
  • Dev Ghosh will add Mosaic Mode to Hugin/Panotools, mentored by Daniel German.
  • Last but not least Yulia Kotseruba will add functionality like multiple blending to Lighttwist, mentored by Sébastien Roy.

I am particularly proud of the collaboration with Sébastien Roy on Lighttwist. First, because I believe there is a cultural fit between his Vision3D laboratory and the Hugin community. Second, because it is good to see academics releasing the result of their research (which, let’s not forget, is often funded with public taxpayer’s money) to the general public under an Open Source license.  And last but not least, because I think that Lighttwist is the natural extension to Hugin. I would dare to advance that Lighttwist is to panoramic photography in 2009 what QuickTimeVR was to it in 1999. In a private email exchange Ken Turkowski, inventor of QuickTimeVR and one of our team, acknowledge the analogy.

Then there is a sixth project that I care about. We have teamed up once again (and we hope to be luckier this time) with VideoLAN to bring QTVR playback to this popular media player. This time with an extra bonus: Wiimote control. León Moctezuma, who in 2007 added SPi-V playback to FreePV, will build on his expertise of FreePV and add the Wiimote on top of it, mentored by Antoine Cellerier of VideoLAN.

What’s next for the students? this article I wrote exactly one year ago applies.

Good Luck, students, I hope to congratulate you all in a few months.

Flash Panoramas, the more the merrier.

The beginnings

In the beginning there was QuickTimeVR. Or was it ptviewer? Those were the days! At the turn of the millenium fans of both technologies fought endless verbal battles about which one is best to display their full spherical artwork. Then two things happened that left them in the cold:

  • now defunct iPix forced Helmut Dersch to pull the plug on panotools and ptviewer. The first Open Source panorama authoring and publishing solution survived thanks to the contributions of Fulvio Senore (ptviewer) Jim Watters, Bruno Postle, Daniel M. German and other contributors (panotools). Helmut’s software was ahead of time. He is now back in the community with new ideas.
  • business logic at Apple pulled the plug on QTVR development. It has lingered unsupported inside QuickTime, until recent updates crippled some functionality dear to VR-artists.

Life went on

  • Starting with the release of Windows XP Service Pack 1, Microsoft removed Java from its system and initiated Java’s decline in ubiquity. It has continuously lost market share since then and is now down at about 84%. Ten years after the inception of the web there was no widely deployed standard yet to display VR content!
  • 3D accelerated video cards became mainstream, and with them the market share of Adobe Shockwave increased too. Probably the first 3D accelerated panorama viewer, SPi-V was released November 22 2004 by Aldo Hoeben.
  • A flurry of viewing technologies came and went. None of them achieved more than single-digit market share. Noteworthy is DevalVR that attracted a passionate following of discerning users for its smooth panning and small footprint.

A new Open Source viewer is born September 14, 2005 when Pablo d’Angelo starts the FreePV Open Source Panoramic Viewer Project with Fulvio Senore and Thomas Rauscher. It is the first viewer to play QTVR on Linux and raises a lot o

f hopes in the community. A Google Summer of Code 2007 project by Leon Moctezuma added SPi-V playing capabilities, but the viewer is still experimental and suffers of the same problem the flurry of other viewing technologies: lack of market penetration. Keep fingers crossed, this year it may become a Google Summer of Code again, integration with the VLC media player.

Flash to the rescue!

flash logo

Flash based panorama players have existed for a while, though most of them did not correct perspective properly and where apt for either flat pictures (like Zoomify), or for cylindrical panoramas.

With the arrival of Flash 8 in August 2005 (although Linux users had to wait until January 2007, when Flash 9 for Linux was released), full spherical panoramas became possible. First generation full spherical players include Thomas Rauscher’s Pano2QTVR and Immervision’s PurePlayer Flash. Flash 8 was not completely up to the challenge yet. The audience reported seeing snakes instead of straight line.

With Flash 9 quality improved dramatically. Denis V. Chumakov’s FPP became the most popular Flash 9 player.

Flash is the most widely distributed plugin, with a market penetration of 98%. Adobe has done almost everything right to get Flash widely accepted. It’s a unique value proposition of ubiquity, features and flexibility.

In March 2007 I predicted a mushrooming of Flash based panorama viewers within 12-18 month, similarly to Flash based mp3 players. Today, Patrick Cheatham and Zephyr Renner made my prediction come true with the release of an Open Source viewer based on the Papervision3D engine. I hope it is the start of a growing community effort.

Meanwhile, Adobe works on Flash 10 that will include hardware accelerated 3D. Exciting times ahead!