• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    September 2017
    M T W T F S S
    « Dec    
     123
    45678910
    11121314151617
    18192021222324
    252627282930  
  • Archives

History

Using Google Books, Jim Watters located some historic sources on how to make panoramic photos.

Thank you, Jim!

LGM 2009: Conclusions

Time to sum up my feelings about Libre Graphics Meeting (LGM) 2009 and set targets for the next twelve months, inspired and encouraged by Jon Phillips.

LGM2009: what went well

  • Venue: Excellent support, good infrastructure, and extremely useful recordings of the conference (thank you, Kaveh Bazaragan). The offer by Louis Desjardins to hold LGM regularly in this setting should be considered seriously. Kaveh’s offer to host us in India is very attractive as well. Never been to that part of the world…
  • Format: Single track is just right! Stick to the clock. Session length appropriate. Good mix of sessions, lightning talks, BoF. Plenty of opportunities to get things done and network.
  • Attendance: A good mix of artists and developers.

LGM2009: what could have been better

  • Echo in the local media (I tried to get noticed with the Ultra Wide Views exhibition but strictly speaking I’m not local).
  • Interaction with the local public (my impression is that there was less local public than at LGM2007).
  • Outreach beyond the world of Free graphics. Everybody has imaging devices nowadays and we need to convey to them that this software is for them too.

To do for LGM2010

  • Decide on the venue ASAP. This is the single most important issue on the critical path to leading to LGM2010. The discussion on the mailing list seems to have fade out?
  • Reach out to the general public! Ginger Coons suggested tupperware parties? Let’s show schoolkids that they need not pirate software to be creative. Let’s share our creations with the general public. Let’s get the media buzz going. Concentrate a few speeches geared at the general public on the Saturday. Make the exhibition last longer than the conference, with the conference being its culminating point. Entice the general public to learn about free graphics.
  • Keep the buzz going – my feeling is that there is not enough going on in between conferences. I’d like to see more activity on the mailing list. We need more fund raising in these difficult times.

Hugin at LGM2009

Lucky circumstances had it that I could offer free lodging to those of our team that were attending. Being that close was a unique opportunity to discuss the challenges ahead, both immediate (with two of the new Google Summer of Code students due to start coding soon) and in the longer term (with Andrew Mihal, Pablo d’Angelo, Jim Watters). While I would prefer to focus on the creative process rather than on the tool-making process, we are still at a stage where tools require more attention than I’d like.

  1. The first exhibition of panoramic artwork was a good start. It provides a stepping stone for a V2 at LGM2010 (or 2011?). I’ll summarize key learnings at some point.
  2. Our tool-making process is not ticking regularly enough yet. We’ve discussed a fix for our release cycle (we do not subscribe to ffmpeg’s philosophy and we are far behind compared with the best practices of the Libre Graphics pack). Pablo had a good idea for a new naming convention. Hugin-0.8 is due very soon. After that we’ll initiate the changes.
  3. Our organization needs to grow to the next level. Here again, we’ll be looking at best practices in the Libre Graphics world and shamelessly copy you.

To do for Hugin 2010

  1. Get more artists to physically attend the conference and present their work. I liked Stani’s presentation; an inspiration for next year. I did not really plan jumping in for Tom and presenting his new tool. Next year we shall also have at least one tutorial session to introduce people (possibly from the general public) to the joys of panorama photography.
  2. Deliver the tools to the users at a faster pace. Users have been waiting almost a year to have convenient access to tools introduced during Google Summer of Code 2008. We intend to shorten that time with the current crop of Google Summer of Code projects.
  3. Structure our organization to better handle donations (foundation? ideally piggy-back on something existing rather than reinvent the wheel) and go more systematically about recruiting which is currently also not diversified enough.
  4. Help another small Libre Graphics related project to jump on the bandwagon. This year Hugin made the conscious decision to allocate one of its Google Summer of Code slots to Lighttwist. I believe that giving a lift to like-minded projects and helping them establish reputation as recipients of student slots within the GSoC universe is in the common interest. For 2010, should Google elect to run the program again and should we be accepted as a mentoring organization again, I already spoke with Rafal Mantiuk. He would love to have a student work on pfstools. Whether I can pull my friends at Hugin in this direction remains to be seen – the connection between Lighttwist and Hugin is a more obvious one than between pfstools and Hugin. But I am confident that this is of strategic interest to Libre Graphics as a whole, beyond Hugin.

Umbrella Organization

One fact that can’t go unnoticed is the complete absence from LGM of one of the biggest Free Software sponsors. And I am not talking money. Leslie Hawthorn of the Google Open Source Program Office was a two hours drive away, in Ottawa, at BSDCan. Why couldn’t LGM attract Google Open Source Program Office to hold a presentation? What can we do to improve this, and maybe even convince Google to return to be a full sponsor of LGM like it was in 2008?

I think one of the issues is that we’re still too splintered. Libre Graphics would be better off if we can set up some sort of umbrella organization to handle our common interests.

  • Umbrella organizations such as Gnome, KDE, FreeBSD and others are heavy weights of the Google Summer of Code program. The logic is simple: Leverage. Google leverages the mentoring organizations to manage projects. The more student projects a mentoring organization takes care of, the more attractive it is for Google to work with it. An umbrella organization as an added layer makes the program scale better. It reduces the strain on Google’s resources for a given quantity of students while providing the necessary quality assurance. I’ve added a feature request in that sense to Melange’s tracker. An umbrella organization could allocate slots within projects that taken alone would have little chance to be selected as mentoring organization, and many Libre Graphics projects qualify in this category.
  • Hugin is not the only project that needs a formal way to handle money. Other projects have already been down that road. Why reinvent the wheel? We could learn from their best practices regarding tax exempt status and other issues. Or even better: why start a foundation if there are existing ones? We’ll be looking not just to learn from existing foundations, but also to explore whether there is a natural fit for us to join forces. Pablo agrees that joining an existing foundation is the best option for us, and my wish would be to join an umbrella foundation for Libre Graphics.
  • I hope I’m not the only one who wants to help other small Libre Graphics projects on the bandwagon. As a team we can help more efficiently.
  • Last but not least, the umbrella organization could give a base to the Libre Graphics teams organizing the local meetings. As such, we already have a candidate leader for the umbrella organization: Louis Desjardins has done a terrific job at LGM2007 and LGM2009 – plus all the support he has given to the LGM2008 organization. I did not discuss this endorsement with him and I hope my proposition does not  anger him: Louis for president!

Academic Track

One of the directions discussed at the end of LGM 2009 is to improve ties to the academic world. There is a natural fit between motivated students (not only in computer science) and Open Source. With Micheal Terry of University of Waterloo (a pioneering institution that it lets students keep intellectual property to their work, enabling them to release under Free licenses); Nicolas Robidoux of Laurentian University; Sébastien Roy of Université de Montréal; there were at least three professors presenting at LGM. And let’s not forget Benoît Ozell of École Polytechnique who made LGM’s infrastructure possible. We need more. Universities can leverage Free software as teaching instrument and research ground; as well as a lower cost, high quality, self-maintained infrastructure (not a negligible point in these times of shrinking budgets). Free software’s benefit would be a reneweable source of recruits: as generations of students come and go, some of them may stick around. Others will just make a small part of the journey with us as a way to start early establishing a track record for their future professional career. Tht0s good too. I am proud of the ties Hugin developed to Lighttwist and the 3D Vision Lab at Université de Montréal. We need more such symbiotic relationships.

Usability and User Interfaces

One of the hottest topics throughout the conference was user interfaces.

Jon seems to be sold on the web based stuff. Listen to his talk. I’m skeptic. The cloud buzz reminds me of early day eCommerce buzz, and more specifically a Powerpoint slide (I think it was a McKinsey consultant) ranking goods and services in order of eMarketability. That slide made perfect sense to me: goods where the touch/feel/smell experience is important (e.g. cigars) don’t sell well over the net. Twelve years and a few bubbles later another smart consultant may have already a similar slide for the cloud. Great for new media publishing; for transparent data backups and other back-end services. I bet that graphics apps in the cloud are not far away from cigars in eCommerce. For services that already have an inherent dependence on bandwidth, the cloud is indeed a great thing. Redundancy is good.

I found Michael Terry’s lightning talk about kinematic templates one of the most refreshing and interesting contributions. Time to apply these same principles to general menu navigation. Apple does it the hardware way by putting the menus always on top of the screen so that reaching for a menu I don’t have to slow down and target the item precisely as if it was in the middle of the screen. With screens becoming ever larger, the approach has its limitation. Not every pixel on the screen has equal weight/importance and this should be mapped to a sort of gravity map in mouse movements. Drawings / kinematic templates are the first and most immediate application to this. Kudos!

Video Editing

Before LGM I had this perception of void in the area of video editing. I did my research six months ago when I bought this great camcorder to share the joys of a grandson discovering the world with my parents across the pond. I want to spend time video editing, not trying to make the tools work. Result: I bought SONY Vegas 9 Platinum Edition to replace the useless stuff that comes bundled with the camcorder and has the limitation that it does not work with anything else but the bundled camcorder (serial number check!).

Jean-François Fortin Tam gave an excellent presentation about PiTiVi. Interestingly, he was perviously also a Vegas user – Vegas is in my opinion the least worse of the pack of video editing applications, all licenses considered. PiTiVi looks promising, but it seems to be only available for Linux, still lacks some basic tools such as fade transitions, and as Jean-François said, tog et something useable you still have to make an effort to make the tools work by getting them from the repository. I have not tried it yet, but it may become an interesting alternative in a few month, when the project will release something stable that will be picked up by the different Linux distributions.

What really inspired me was Bassam Kurdali’s speech describing the use of Blender as a video editor. During Bassam’s talk I grabbed a video to my notebook and tried to follow in his footstep. And guess what? I’m sold. From a user interface perspective, things worked very well for me. Only a technical issue for which I filed a bug report is keeping me from making my first steps in Blender as a video editor.

User Interfaces Revisited

Using Blender was an eye-opener to me. My greatest fear, which turn out to be unwarranted, was that a complex tool would have a steep learning curve. What I found out was the other way around. Blender has a user interface metaphore constent across the board (even across different operating systems, which is a plus) which makes it so easy to take the plunge! It interface effectively between the users and the very broad and sometimes extremely complex set of functionalities. It scales nicely and once a few basic concepts are understood, it is very effective. Blender rocks!

This brings me to another (last) topic in this long post: UI consistency. As Ginger Coons said in one of my preferred talks designers like Adobe. In my opinion, one of the factors that contribute to this liking is the consistent user interface. When you know how to handle one tool, you know how to handle all of them. Many Libre Graphics software have a rather rough user interface (and I include Hugin in the pack). The notable exception is Blender.

The UIs of our tools are too disparate. I know my following wish won’t happen any time soon. I know all of our tools have their own unique histories, cultures, dependencies, likes and dislikes. But I am allowed to dream, right? So let me drop the bomb: If I could, I’d put a layer of Blender UI on top of all of them. Because the Blender UI is so great. It’s consistent across the tools and even across operating systems. My guess is that this is the result of a well thought-through design because the people at Blender were confronted with a bigger problem than the rest of us. They had to interface between their users with a set of functionaities that is far more extended and often times more complex than any other graphics tool. The result of their design effort is a UI with a clean and consistent metaphore. Learn that, and you’re half-way on the learning curve. It is easy to learn and efficient to use.

Conclusions

LGM 2009 was great. Dreaming is permitted. But most important, we need to take consistent actions throughout the year and build on top of what has been achieved to make LGM 2010 even better. For myself, I’m off for a one month trip back home. When I’m back, I’ll tackle those priorities that are in the realm of what I can do. For the rest, I am ready to team up with whoever shares the vision of a stronger, united, Libre Graphics Universe.

LGM Day 1

We’re at LGM. Yesterday  I picked up the Ultra Wide Views, Fahim in downtown and Pablo coming from Germany at the airport. We spent the late afternoon / early evening mounting all the canvas and the few photopaper prints; then we looked at Sébastien, Vincent and Jean-Ambroise set up the LightTwist driven Cyclorama. We had a bite with Benoît Ozell, his support for everything happening here at École Polytechnique de Montréal is fantastic. Dev Ghosh joined us after his plane landed.  Jim Watters arrived after dinner. I worked until 3 AM on my notebook.

This morning most things worked as planned but still neded some oiling. The exhibit of 360° views from a dozen artists on the Cyclorama was ready just in time for after Sébastien’s speech, with some minor details corrected afterwards. It was delighting to receive the visit of .Jean-Pierre Lavoie, one of the most successful panorama photographers in Canada, who has pioneered usage of immersive medias in the news. LightTwist made a great debut.

I am running on my last juices, but all the interesting people keep me going. It’s good to see familiar faces; to put a face on a name known from the mailing lists; to meet new people and hear about new ideas. This conference is refreshing. I’m capturing some of the moments on video, but won’t have time to process it for a while.

Andrew Mihal arrived a few minutes ago and Yulia Kotseruba is scheduled to arrive late tonight. This will complete one of the largest Hugin developers line-ups in a single location. Friday night is the conference’s official dinner, but we’ll have our team dinner Thursday night in a yet to be defined place.

Vedutismo – Practical Application

Equirect

equirectangular 360°

The above is an equirectangular projection of a full spherical panorama. You can view it from the inside here.

In the past few days, libpano has seen the addition of four new projections. I added them to hugin’s experimental version in the subversion repository.

Panini 180°

Panini 180°

Panini is credited to Bruno Postle and Thomas K. Sharpless. It was added to libpano13 by Daniel M. German, and it is probably the most interesting of the new projections. It replicates the perspective of vedute, stunning detailed large painting with a very broad field of view.

Architectural 360°

Architectural 360°

Architectural is an idea of Daniel M. German. A hybrid mix of Miller (which is almost conformal) on the top half and Lambert Equal Area on the bottom half. It presumes the space around you (in the bottom half of the pano) is less important than the space above the horizon, uses a smaller surface than the Equirectangular while giving more space to the top half.

Oprtographic 180°

Orthographic 180°

Jim Watters committed the last two, which are fisheye-like variations – the orthographic and equisolid projections.

Equisolid 180°

Equisolid 180°

When the dust settles

It’s a few week before the official coding starts. The students are bonding nicely with the community. Ideas are flowing, and some patches too. I can’t wait to see what they will do when they are officially on Google’s payroll and assigned to work full time on hugin.

While Pablo and Alexandre P. are at LGM, time to introduce properly this year’s team that will participate in the Google Summer of Code. There is a lot to be written, and more may come over the next few months as the summer unfolds. For now, here is an overview of the six projects we’re working on, of the people, and of exciting things to come.

The Projects

There are six projects this year in our portfolio, even though only five are listed on the Hugin/panotools entry at Google. The sixth project is a joint effort with VideoLAN on their leading cross-platform media player. It is listed on their page at Google.

OpenGL hugin preview

The preview windows is central to the panorama making process. It is here where the author looks at his composition before rendering it. It is here that framing decisions are being made, interactively. Right now the interaction is slow.  James Alastair Legg, mentored by Pablo d’Angelo, will improve the panorama preview windows by giving it the speed of OpenGL. We expect near real-time interaction for the author when composing his panorama.

Automatic Feature Matching for Panoramic Images

Critically important to stitching panoramas is to identify overlapping images in two features and align them in space. In the past, this was tedious, manual work. Then control point detectors came along. Those available to hugin users are still tainted by patents. It’s a two step process: detecting features and matching them. In Google Summer of Code 2007 Zoran Mesec wrote the detector, MatchPoint. This year Onur Küçüktunç will write the matcher, mentored by Alexandre Jenny, author of the original autopano and of Autopano Pro. A beautiful example of cooperation and co-existence between the proprietary and open source world. We expect to have a fully patent free control point detection process.

Masking in GUI

Overlap regions between two images are the nature of stitched panoramas. Since the world is in movement, such overlaps often present challenges that won’t match. The current solution is to render each image on a separate layer, and then mask out manually one of the two images so to display only one frozen instance of the moving object. This can often be a painful work at the single pixel level. Fahim Mannan, mentored by Daniel M. German, will introduce a simpler workflow: just mark the moving object with a couple of approximate brush strokes and his code will work out the exact object boundaries automatically. We expect an improvement for those using hugin to stitch action panoramas.

Batch Processing

Photographers often come back from the field with tons of photographs to stitch. A lot of this could be automated. Even more so with the up and coming pano-videography. Marko Kuder, mentored by Zoran Mesec, will improve hugin’s batching abilities. We expect to be able to process repetitive tasks without human intervention.

Machine-based Sky Identification

Some areas of photographs are better suited for control points than others. The sky, with its moving clouds, definitely not, as good control point don’t move between images. Timothy Nugent, mentored by Yuval Levy, will train a support vector machine (SVM) to identify clouds in the sky as a bad area for control points, so that it can be masked out before triggering the control point detection. Once working the method can be extended to other features as well, such as foliage and water.

Panorama Viewing in VideoLAN

19 years after Tim Berners-Lee invented the web there is still no universal format to view panoramas on the web. Apple’s QuickTimeVR, the original technology to display full spherical panoramas, is not available on platforms other than Windows and the Macintosh, and it is no longer developed by Apple. A lot of good things have happened with Flash panoramas in the last year. Nevertheless, a lot of legacy content out there is captive of the QuickTime format, like the World Wide Panorama. In Google Summer of Code 2007 Leon Moctezuma added functionality to FreePV. This year Michael Ploujnikov mentored by Yuval Levy will integrate panorama viewing in VLC, a leading cross-platform media player. We expect Linux users and users of other alternative platforms to have access to the majority of QTVR content soon.

The team

I’m happy to pass the admin role to Alexandre Prokoudine this year. We had more available mentors, student applications and project ideas that we would have loved to follow through, but resources are limited also for large corporations and Google is already very generous with us. We would have loved to see students working on leading edge image processing under the supervision of Andrew Mihal of enblend/enfuse fame, or John Cupitt of VIPS. Maybe next year. Other mentors that registered with our organization on the Google Code page and are left without student are Bruno Postle, Jim Watters and Ken Turkowski. We are a team, and like last year I expect a lot of help and community support to the six lucky students.

Partners

Cooperation is a topic I particularly care about for this edition of the Google Summer of Code. We are leveraging the Summer of Code to reach beyond our small world. I am proud that we found an ally in the VideoLAN team, a larger mentoring organization. Granted, we are natural allies: hugin/panotools is used to create media; and VLC is used to play it. Nevertheless, this cross-collaboration, whose idea is the result of a meeting with Jean-Baptiste Kempf at the 2007 Google Summer of Code Mentor Summit, is IMO a demonstration that the whole is worth more than the sum of the parts and that an initiative like the Google Summer of Code adds much more value to the world of FOSS than what can be stated in (highly appreciated) numbers. 175 mentoring organization, 1125 students, 90 countries.

And in our small world we’re working a partnership deal to further motivate and propel the hugin/panotools team, similar to what we did last year. Stay tuned for an announcement.

Flash Panoramas, the more the merrier.

The beginnings

In the beginning there was QuickTimeVR. Or was it ptviewer? Those were the days! At the turn of the millenium fans of both technologies fought endless verbal battles about which one is best to display their full spherical artwork. Then two things happened that left them in the cold:

  • now defunct iPix forced Helmut Dersch to pull the plug on panotools and ptviewer. The first Open Source panorama authoring and publishing solution survived thanks to the contributions of Fulvio Senore (ptviewer) Jim Watters, Bruno Postle, Daniel M. German and other contributors (panotools). Helmut’s software was ahead of time. He is now back in the community with new ideas.
  • business logic at Apple pulled the plug on QTVR development. It has lingered unsupported inside QuickTime, until recent updates crippled some functionality dear to VR-artists.

Life went on

  • Starting with the release of Windows XP Service Pack 1, Microsoft removed Java from its system and initiated Java’s decline in ubiquity. It has continuously lost market share since then and is now down at about 84%. Ten years after the inception of the web there was no widely deployed standard yet to display VR content!
  • 3D accelerated video cards became mainstream, and with them the market share of Adobe Shockwave increased too. Probably the first 3D accelerated panorama viewer, SPi-V was released November 22 2004 by Aldo Hoeben.
  • A flurry of viewing technologies came and went. None of them achieved more than single-digit market share. Noteworthy is DevalVR that attracted a passionate following of discerning users for its smooth panning and small footprint.

A new Open Source viewer is born September 14, 2005 when Pablo d’Angelo starts the FreePV Open Source Panoramic Viewer Project with Fulvio Senore and Thomas Rauscher. It is the first viewer to play QTVR on Linux and raises a lot o

f hopes in the community. A Google Summer of Code 2007 project by Leon Moctezuma added SPi-V playing capabilities, but the viewer is still experimental and suffers of the same problem the flurry of other viewing technologies: lack of market penetration. Keep fingers crossed, this year it may become a Google Summer of Code again, integration with the VLC media player.

Flash to the rescue!

flash logo

Flash based panorama players have existed for a while, though most of them did not correct perspective properly and where apt for either flat pictures (like Zoomify), or for cylindrical panoramas.

With the arrival of Flash 8 in August 2005 (although Linux users had to wait until January 2007, when Flash 9 for Linux was released), full spherical panoramas became possible. First generation full spherical players include Thomas Rauscher’s Pano2QTVR and Immervision’s PurePlayer Flash. Flash 8 was not completely up to the challenge yet. The audience reported seeing snakes instead of straight line.

With Flash 9 quality improved dramatically. Denis V. Chumakov’s FPP became the most popular Flash 9 player.

Flash is the most widely distributed plugin, with a market penetration of 98%. Adobe has done almost everything right to get Flash widely accepted. It’s a unique value proposition of ubiquity, features and flexibility.

In March 2007 I predicted a mushrooming of Flash based panorama viewers within 12-18 month, similarly to Flash based mp3 players. Today, Patrick Cheatham and Zephyr Renner made my prediction come true with the release of an Open Source viewer based on the Papervision3D engine. I hope it is the start of a growing community effort.

Meanwhile, Adobe works on Flash 10 that will include hardware accelerated 3D. Exciting times ahead!

undistorted alignment

CP-screenshot

Control Points (CP), also known as Reference Points, are the holy grail of image stitching. They are identified in the overlap area between two images so that the images can be aligned to each other in space.In the old days, panorama photographers would tediously map these points manually, as a set of pixel coordinates on adjacent images. Even before that, they would patiently use scotch tape. Then came automated software.

The area is still the focus of much research, and was the subject of a Google Summer of Code project by Zoran Mesec, mentored by Dr. Herbert Bay, the inventor of the SURF algorythm. The project is still under development and Zoran has added new code to the hugin repository a few weeks ago, just before his exams.

The problem of most current software for automatic generation of CP is that it has troubles with those areas where the two photos have different kind of distortion. This is inevitable with fisheye lenses that are very popular for the production of full spherical panoramas.

A solution to this problem is to compare the photos after remapping them into a conformal projection. Bruno Postle announced recently an implementation of a perl tool that runs the usual automatic generation of CP on temporary stereographic versions of photos. Despite all the apparent overhead, it is not significantly slower than conventional software with the same feature detection algorithm.

For the moment it is a perl script, dependent on some extra CPAN perl modules and ImageMagick. And it runs from the command line. But it won’t take long until Hugin can feed it with the appropriate parameters. I’ve been delaying my next installer snapshot in the hope to be able to add it.