• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    April 2014
    M T W T F S S
    « Dec    
     123456
    78910111213
    14151617181920
    21222324252627
    282930  
  • Archives

LGM 2009: Conclusions

Time to sum up my feelings about Libre Graphics Meeting (LGM) 2009 and set targets for the next twelve months, inspired and encouraged by Jon Phillips.

LGM2009: what went well

  • Venue: Excellent support, good infrastructure, and extremely useful recordings of the conference (thank you, Kaveh Bazaragan). The offer by Louis Desjardins to hold LGM regularly in this setting should be considered seriously. Kaveh’s offer to host us in India is very attractive as well. Never been to that part of the world…
  • Format: Single track is just right! Stick to the clock. Session length appropriate. Good mix of sessions, lightning talks, BoF. Plenty of opportunities to get things done and network.
  • Attendance: A good mix of artists and developers.

LGM2009: what could have been better

  • Echo in the local media (I tried to get noticed with the Ultra Wide Views exhibition but strictly speaking I’m not local).
  • Interaction with the local public (my impression is that there was less local public than at LGM2007).
  • Outreach beyond the world of Free graphics. Everybody has imaging devices nowadays and we need to convey to them that this software is for them too.

To do for LGM2010

  • Decide on the venue ASAP. This is the single most important issue on the critical path to leading to LGM2010. The discussion on the mailing list seems to have fade out?
  • Reach out to the general public! Ginger Coons suggested tupperware parties? Let’s show schoolkids that they need not pirate software to be creative. Let’s share our creations with the general public. Let’s get the media buzz going. Concentrate a few speeches geared at the general public on the Saturday. Make the exhibition last longer than the conference, with the conference being its culminating point. Entice the general public to learn about free graphics.
  • Keep the buzz going – my feeling is that there is not enough going on in between conferences. I’d like to see more activity on the mailing list. We need more fund raising in these difficult times.

Hugin at LGM2009

Lucky circumstances had it that I could offer free lodging to those of our team that were attending. Being that close was a unique opportunity to discuss the challenges ahead, both immediate (with two of the new Google Summer of Code students due to start coding soon) and in the longer term (with Andrew Mihal, Pablo d’Angelo, Jim Watters). While I would prefer to focus on the creative process rather than on the tool-making process, we are still at a stage where tools require more attention than I’d like.

  1. The first exhibition of panoramic artwork was a good start. It provides a stepping stone for a V2 at LGM2010 (or 2011?). I’ll summarize key learnings at some point.
  2. Our tool-making process is not ticking regularly enough yet. We’ve discussed a fix for our release cycle (we do not subscribe to ffmpeg’s philosophy and we are far behind compared with the best practices of the Libre Graphics pack). Pablo had a good idea for a new naming convention. Hugin-0.8 is due very soon. After that we’ll initiate the changes.
  3. Our organization needs to grow to the next level. Here again, we’ll be looking at best practices in the Libre Graphics world and shamelessly copy you.

To do for Hugin 2010

  1. Get more artists to physically attend the conference and present their work. I liked Stani’s presentation; an inspiration for next year. I did not really plan jumping in for Tom and presenting his new tool. Next year we shall also have at least one tutorial session to introduce people (possibly from the general public) to the joys of panorama photography.
  2. Deliver the tools to the users at a faster pace. Users have been waiting almost a year to have convenient access to tools introduced during Google Summer of Code 2008. We intend to shorten that time with the current crop of Google Summer of Code projects.
  3. Structure our organization to better handle donations (foundation? ideally piggy-back on something existing rather than reinvent the wheel) and go more systematically about recruiting which is currently also not diversified enough.
  4. Help another small Libre Graphics related project to jump on the bandwagon. This year Hugin made the conscious decision to allocate one of its Google Summer of Code slots to Lighttwist. I believe that giving a lift to like-minded projects and helping them establish reputation as recipients of student slots within the GSoC universe is in the common interest. For 2010, should Google elect to run the program again and should we be accepted as a mentoring organization again, I already spoke with Rafal Mantiuk. He would love to have a student work on pfstools. Whether I can pull my friends at Hugin in this direction remains to be seen – the connection between Lighttwist and Hugin is a more obvious one than between pfstools and Hugin. But I am confident that this is of strategic interest to Libre Graphics as a whole, beyond Hugin.

Umbrella Organization

One fact that can’t go unnoticed is the complete absence from LGM of one of the biggest Free Software sponsors. And I am not talking money. Leslie Hawthorn of the Google Open Source Program Office was a two hours drive away, in Ottawa, at BSDCan. Why couldn’t LGM attract Google Open Source Program Office to hold a presentation? What can we do to improve this, and maybe even convince Google to return to be a full sponsor of LGM like it was in 2008?

I think one of the issues is that we’re still too splintered. Libre Graphics would be better off if we can set up some sort of umbrella organization to handle our common interests.

  • Umbrella organizations such as Gnome, KDE, FreeBSD and others are heavy weights of the Google Summer of Code program. The logic is simple: Leverage. Google leverages the mentoring organizations to manage projects. The more student projects a mentoring organization takes care of, the more attractive it is for Google to work with it. An umbrella organization as an added layer makes the program scale better. It reduces the strain on Google’s resources for a given quantity of students while providing the necessary quality assurance. I’ve added a feature request in that sense to Melange’s tracker. An umbrella organization could allocate slots within projects that taken alone would have little chance to be selected as mentoring organization, and many Libre Graphics projects qualify in this category.
  • Hugin is not the only project that needs a formal way to handle money. Other projects have already been down that road. Why reinvent the wheel? We could learn from their best practices regarding tax exempt status and other issues. Or even better: why start a foundation if there are existing ones? We’ll be looking not just to learn from existing foundations, but also to explore whether there is a natural fit for us to join forces. Pablo agrees that joining an existing foundation is the best option for us, and my wish would be to join an umbrella foundation for Libre Graphics.
  • I hope I’m not the only one who wants to help other small Libre Graphics projects on the bandwagon. As a team we can help more efficiently.
  • Last but not least, the umbrella organization could give a base to the Libre Graphics teams organizing the local meetings. As such, we already have a candidate leader for the umbrella organization: Louis Desjardins has done a terrific job at LGM2007 and LGM2009 – plus all the support he has given to the LGM2008 organization. I did not discuss this endorsement with him and I hope my proposition does not  anger him: Louis for president!

Academic Track

One of the directions discussed at the end of LGM 2009 is to improve ties to the academic world. There is a natural fit between motivated students (not only in computer science) and Open Source. With Micheal Terry of University of Waterloo (a pioneering institution that it lets students keep intellectual property to their work, enabling them to release under Free licenses); Nicolas Robidoux of Laurentian University; Sébastien Roy of Université de Montréal; there were at least three professors presenting at LGM. And let’s not forget Benoît Ozell of École Polytechnique who made LGM’s infrastructure possible. We need more. Universities can leverage Free software as teaching instrument and research ground; as well as a lower cost, high quality, self-maintained infrastructure (not a negligible point in these times of shrinking budgets). Free software’s benefit would be a reneweable source of recruits: as generations of students come and go, some of them may stick around. Others will just make a small part of the journey with us as a way to start early establishing a track record for their future professional career. Tht0s good too. I am proud of the ties Hugin developed to Lighttwist and the 3D Vision Lab at Université de Montréal. We need more such symbiotic relationships.

Usability and User Interfaces

One of the hottest topics throughout the conference was user interfaces.

Jon seems to be sold on the web based stuff. Listen to his talk. I’m skeptic. The cloud buzz reminds me of early day eCommerce buzz, and more specifically a Powerpoint slide (I think it was a McKinsey consultant) ranking goods and services in order of eMarketability. That slide made perfect sense to me: goods where the touch/feel/smell experience is important (e.g. cigars) don’t sell well over the net. Twelve years and a few bubbles later another smart consultant may have already a similar slide for the cloud. Great for new media publishing; for transparent data backups and other back-end services. I bet that graphics apps in the cloud are not far away from cigars in eCommerce. For services that already have an inherent dependence on bandwidth, the cloud is indeed a great thing. Redundancy is good.

I found Michael Terry’s lightning talk about kinematic templates one of the most refreshing and interesting contributions. Time to apply these same principles to general menu navigation. Apple does it the hardware way by putting the menus always on top of the screen so that reaching for a menu I don’t have to slow down and target the item precisely as if it was in the middle of the screen. With screens becoming ever larger, the approach has its limitation. Not every pixel on the screen has equal weight/importance and this should be mapped to a sort of gravity map in mouse movements. Drawings / kinematic templates are the first and most immediate application to this. Kudos!

Video Editing

Before LGM I had this perception of void in the area of video editing. I did my research six months ago when I bought this great camcorder to share the joys of a grandson discovering the world with my parents across the pond. I want to spend time video editing, not trying to make the tools work. Result: I bought SONY Vegas 9 Platinum Edition to replace the useless stuff that comes bundled with the camcorder and has the limitation that it does not work with anything else but the bundled camcorder (serial number check!).

Jean-François Fortin Tam gave an excellent presentation about PiTiVi. Interestingly, he was perviously also a Vegas user – Vegas is in my opinion the least worse of the pack of video editing applications, all licenses considered. PiTiVi looks promising, but it seems to be only available for Linux, still lacks some basic tools such as fade transitions, and as Jean-François said, tog et something useable you still have to make an effort to make the tools work by getting them from the repository. I have not tried it yet, but it may become an interesting alternative in a few month, when the project will release something stable that will be picked up by the different Linux distributions.

What really inspired me was Bassam Kurdali’s speech describing the use of Blender as a video editor. During Bassam’s talk I grabbed a video to my notebook and tried to follow in his footstep. And guess what? I’m sold. From a user interface perspective, things worked very well for me. Only a technical issue for which I filed a bug report is keeping me from making my first steps in Blender as a video editor.

User Interfaces Revisited

Using Blender was an eye-opener to me. My greatest fear, which turn out to be unwarranted, was that a complex tool would have a steep learning curve. What I found out was the other way around. Blender has a user interface metaphore constent across the board (even across different operating systems, which is a plus) which makes it so easy to take the plunge! It interface effectively between the users and the very broad and sometimes extremely complex set of functionalities. It scales nicely and once a few basic concepts are understood, it is very effective. Blender rocks!

This brings me to another (last) topic in this long post: UI consistency. As Ginger Coons said in one of my preferred talks designers like Adobe. In my opinion, one of the factors that contribute to this liking is the consistent user interface. When you know how to handle one tool, you know how to handle all of them. Many Libre Graphics software have a rather rough user interface (and I include Hugin in the pack). The notable exception is Blender.

The UIs of our tools are too disparate. I know my following wish won’t happen any time soon. I know all of our tools have their own unique histories, cultures, dependencies, likes and dislikes. But I am allowed to dream, right? So let me drop the bomb: If I could, I’d put a layer of Blender UI on top of all of them. Because the Blender UI is so great. It’s consistent across the tools and even across operating systems. My guess is that this is the result of a well thought-through design because the people at Blender were confronted with a bigger problem than the rest of us. They had to interface between their users with a set of functionaities that is far more extended and often times more complex than any other graphics tool. The result of their design effort is a UI with a clean and consistent metaphore. Learn that, and you’re half-way on the learning curve. It is easy to learn and efficient to use.

Conclusions

LGM 2009 was great. Dreaming is permitted. But most important, we need to take consistent actions throughout the year and build on top of what has been achieved to make LGM 2010 even better. For myself, I’m off for a one month trip back home. When I’m back, I’ll tackle those priorities that are in the realm of what I can do. For the rest, I am ready to team up with whoever shares the vision of a stronger, united, Libre Graphics Universe.

LGM Day 3, Day 4, Follow Up

Days went by so quick, so here is a summary.

Day three was intense. The presentations were very interesting, particularly those of Andrew Mihal about Enblend-Enfuse and about GPU stitching. In the evening we had the conference supper, a very pleasant social event at which I learned from about Øyvind’s Now By Then installation and got a chance to purchase one of the last “Architecture Fiver“‘s  that Stani brought along for his talk. He also patiently gave me an insight into the thinking pattern of curators, lifting my morale from the previous day.

On day four I had a near-death experience. Since Tom Sharpless could not attend, I picked up his slide and hosted the talk. In the morning I prepared my notebook in dual display mode, so that on one display I can run the demonstrations of Panini, MathMap and Hugin, while on the other display I had my notes, which I kept editing until shortly before my talk. Without saving. Disaster strikes when I connect the notebook to the projector. My notebook’s native resolution is 1400×1050, too much for the projector to handle. Must restart X. Lose all presets and notes! I froze on the spot. Had to cut on a few gimmicks such as recording the talk with the catadioptric lens or shooting a stitched panorama during the talk (I made the move with the camera around the tripod, but had no available brain cycles to even think of setting the exposure or pressing the real button. I felt like a zombie and was disappointed at my performance which I felt was terrible, although a look at its recording comforted me: it was not that bad after all, even if I forgot to say half the things I wanted to say, the live demo and the interaction with the public were not that catastrophic.

The good news from day four was that Sébastien and Vincent debugged the flat display in extremis so that people walking out of for the lunch break passed by and could admire Guillaume’s stunning Boulevard Bancel (we’ll gladly admit that once this was on screen and working we pulled the cables on the slide show so that nothing can go wrong. Proper slide show the next time).

Since the cafeteria was closed on Sunday, we had to order pizzas for lunch, which was great as it inspired more exchanges before the last presentations. In the end, I even got the honor of shooting the official group picture, and to offer some of the present teams a panorama inside a panorama in the Cyclorama, while our team helped folding up the exhibit canvases.

After protracted good byes, I drove eastwards with Alexandre and Pablo. Alexandre only joined us for a tour of Québec-City. Pablo continued with us to Boréalie before I drove him back to Montréal for his flight on Wednseday. The rest of the week, and the weekend, I had to catch up with pent up business. Particularly the weekend was difficult, with a few difficulties upgrading servers remotely from FreeBSD 6.3 to FreeBSD 7.2 (the worse thing is that all manipulations worked well on the guinea-pig server in the office that is pretty much an exact mirror of the servers to be upgraded).

I still made time on Sunday evening for another quick hop to Montréal, and I am happy I did. Joergen Geerds was visiting for the weekend with his girlfriend. Interesting people.

After

Vedutismo – Practical Application

Equirect

equirectangular 360°

The above is an equirectangular projection of a full spherical panorama. You can view it from the inside here.

In the past few days, libpano has seen the addition of four new projections. I added them to hugin’s experimental version in the subversion repository.

Panini 180°

Panini 180°

Panini is credited to Bruno Postle and Thomas K. Sharpless. It was added to libpano13 by Daniel M. German, and it is probably the most interesting of the new projections. It replicates the perspective of vedute, stunning detailed large painting with a very broad field of view.

Architectural 360°

Architectural 360°

Architectural is an idea of Daniel M. German. A hybrid mix of Miller (which is almost conformal) on the top half and Lambert Equal Area on the bottom half. It presumes the space around you (in the bottom half of the pano) is less important than the space above the horizon, uses a smaller surface than the Equirectangular while giving more space to the top half.

Oprtographic 180°

Orthographic 180°

Jim Watters committed the last two, which are fisheye-like variations – the orthographic and equisolid projections.

Equisolid 180°

Equisolid 180°

Vedutismo

hugin-logoInspired by a discussion of Bruno Postle and Tom K. Sharpless, Daniel M. German added a new projection to libpano, the library underlying hugin for geometrical transforms. The new Panini projection, named after one of the most famous vedutismo painters, would not be immediately available within hugin.

For me, this was the chance to get a little bit deeper into the code. I had a free hour. I had a flu and did not want to pass it on to my baby. So I set on a journey that looking back would only take a few minutes to an experienced coder. And next time it will indeed take me much less. But next time I might leave the challenge to another contributor – maybe a potential Google Summer of Code student looking to commit a patch as part of the selection process.

I started from a fresh and clean build according to the standard instructions for Ubuntu. The file system layout is such that a top level src folder contains a libpano13 folder and a hugin folder with their respective sources fresh from SVN. Since this is an additional projection, I looked at something similar for orientation to what I need to do – in this case how other libpano projections are implemented in hugin.

First I looked at Daniel’s most recent changes to libpano, specifically this change in the list of features that libpano gives to external programs calling it. I happened to find a minor bug in it and posted a patch. And I understood from it that I was to add the preojection #12, PANINI.

I opened a terminal window and searched throughout the hugin source tree for the implementation of the Lambers projection:

$ find . -exec grep -l "LAMBERT" {} \;

returned the list of files to look into, with the exception of those files with .svn/ in the path, as they are part of the subversion system.

I found and looked into the following files:

./src/PTBatcherGUI/ProjectListBox.cpp
./src/hugin_base/panotools/PanoToolsInterface.cpp
./src/hugin_base/panodata/PanoramaOptions.cpp
./src/hugin_base/panodata/PanoramaOptions.h
./src/hugin1/hugin/OutputProjectionInfo.cpp
./src/hugin1/hugin/ChoosyRemapper.cpp
./src/hugin1/hugin/VertexCoordRemapper.cpp
./src/hugin1/tests/test_projections.cpp

I looked inside to understand what they were doing and how I shall change them to accommodate the new projection.

In ./src/hugin_base/panodata/PanoramaOptions.h I added PANINI to the enumeration – just search for LAMBERT to see where it is likely that changes are needed.

In ./src/hugin_base/panodata/PanoramaOptions.cpp I had to add the projection and return sensible value for the maximum horizontal and vertical field of views (HFOV and VFOV). I was not sure. Daniel sugested 220° and 180°. Anything wide enough is OK, if users find that it goes to far, the limit can be adjusted later on. UPDATE: a second look at the code shows that the info comes from libpano13′s queryfeature.c, so no need to edit that.

In ./src/PTBatcherGUI/ProjectListBox.cpp I just added the projection to the list of displayed PTBatcher’s GUI options. I have not tested PTBatcher.

In ./src/hugin_base/panotools/PanoToolsInterface.cpp I added an additional case for PANINI and the _panini value, analog to the other projections. To make sure that I did not have to do additional changes, I grepped for _lambert to confirm that this is just the symbol passed on to libpano and nothing more has to be done in hugin. Indeed I found the symbol in libpano.

For the next three files, I needed to know a little bit about the projection, specifically about its behaviour at the poles. Since AFAIK it does not have “stretchy poles”, I did not make any change in ./src/hugin1/hugin/OutputProjectionInfo.cpp nor in ./src/hugin1/hugin/ChoosyRemapper.cpp. I was not sure about whether it streches across the 360° seam. Since it is a projection that has similarities to RECTILINEAR, I replicated that behavior in ./src/hugin1/hugin/VertexCoordRemapper.cpp. Probably a mistake – the fast preview show gibberish for the new Panini projection, but some of the more expert coders may look into that, and it does not affect the traditional preview nor the rendering output.

Last but not least, ./src/hugin1/tests/test_projections.cpp. It seems that the function of this file is to test the transforms, and it has been neglected for a while: the tests stopped at Lambers Azimuthal – no tests for Albers Conic Equal Areas nor Miller Cilindrical, both of which have been added later to libpano. It was simply an updater of the counter’s upper limit to PANINI that extended the test to these projections, as well as to the new one.

All changes done, cross fingers and continue with the standard build / install process. If something does not work, try to find out what it is. Ask the community!

I was lucky. Everything built and installed.

The next step is to prepare a patch and post it for peer review in the community:

$ svn diff hugin > hugin.panini.patch

Since this is a minor change that does not seem to break anything, and since I have commit privilege, and since we have no policy about svn commits of unstable / insufficiently tested code, I decided to take the sortcut and committed.

$ svn ci -m "support for libpano13's panini" hugin

So simple, if you know what to do. I hope this tutorial is helpful and will motivate you to put your hands under the hood. It is specific to Linux but most likely works with very little changes on OS X as well. Conceptually it is the same in Windows, using Windows’ Explorer instead of the find/grep command line duo, and TortoiseSVN’s diff functionality.

The most important is: try it! Sooner or later you’ll get it right.

“Only those who risk going too far can possibly find out how far one can go.”
- T.S. Eliot (1888-1965)

By the time I finished, Daniel already added a new projection, called Architectural. Will somebody be faster than me and use this tutorial to add it?

Control Points Generators, the more the merrier.

There is a big choice of control point generators in the latest hugin installer. Many of them are experimental. Some work better for some situations than others. Variety is positive as there is likely to be a solution to most situations. But it is also daunting. Which one to choose, and how? here a few pointers. Moreover, the default preferences settings as set by the installer are written down, so you don’t have to be afraid of changing them and not know how to change them back.

Match-n-Shift

Match-n-Shift is Bruno Postle’s idea. Images are remapped to a conformal projection before the identification process. Despite the added step, match-n-shift is not significantly slower. For fisheye lenses, the quality of control points is better than non-remapped generators. Currently it only works with the good old autopano and it is subject to the same patents constraints, but efforts are underway to make it work with Matchpoint.

Default preference settings: -f %f -v %v -c -p %p -o %o %i

Autopano-SIFT Perl

Autopano-SIFT Perl is the latest incarnation of the classic SIFT-based control point generator that has been used by hugin users for years, and its use is limited by the SIFT patent in some countries, notably the United States. The shell wrapper script has been rewritten in Perl by Bruno Postle so that it can be compiled for and run on Windows (where Perl is optional). Linux and OSX users have had access to this tool for a long time.

Default preferences settings: –noransac –points %p –output %o %i

Autopano-SIFT-C

Autopano-SIFT-C is Tom K. Sharpless reengineering of Autopano-SIFT Perl. Tom is passionate about precision and efficiency. He initially used the full scale image to achieve best possible precision, but there is a cost to that: double the maximum dimension and it quadruples the time needed to process that single picture. This gets compounded as the pictures are matched against each other. Autopano-SIFT-C current default is 1600 pixels, but in my installer I have set it to 800 pixels, the value used by match-n-shift and by autopano-SIFT Perl, which yields excellent results.

Autopano-SIFT-C is more efficient and thus slightly faster, though I could not notice the difference with my average project. It also does some remapping, but I found match-n-shift much more elegant. The main difference is that Autopano-SIFT-C has a monolithic approach. Everything happens within a single file: scaling, reprojection, detection, matching, so it is less flexible to combine and cooperate with other tools.

Last but not least, the algorithm is still SIFT, so patents may apply.

Default preferences settings: –maxdim 800 –maxmatches %p %o %i
–maxdim (default 1600) => set to 800 to compare with a-c-c

Edit: I’ve been made aware of a recently introduced expandos in hugin that I had forgotten. –maxmatches %p %o %s will use a conformal projection. A practical show of how right Agos’ comment below is!

Matchpoint

Matchpoint was Zoran Mesec’s 2007 Google Summer of Code project. He wrote it as a drop-in replacement for the generatekeys program that came with autopano, so it was easy for Bruno Postle to adapt the Autopano-SIFT-C wrapper to it. Unfortunately Matchpoint still has a problem dealing with transparency masks, so it can’t be used with match-n-shift.

Matchpoint is the only feature detector in this article to be free of patent. It is still experimental but yields useful results. In the upcoming Google Summer of Code participation Onur Küçüktunç will develop a feature matcher to go with, mentored by Alexandre Jenny, the author of Autopano Pro.

Default preferences settings: –noransac –points %p –output %o %i

Pan-o-matic

Panomatic is the newest kid on the block. It is an implementation of the SURF algorythm (which is also patent protected in some jurisdictions) by Anael Orlinski. It is built to extract maximum power from modern multi-core processors and comes in two flavors. The standard Pan-o-matic runs on modern CPUs, that means Pentium 4 and newer or AMD Athlon XP and newer. The NOSSE version runs also on older CPUs.

Default preferences settings: -o %o %i

A few interesting options for those who may want to try:

  • -n<number> sets the of cores to use (default: autodetect)
  • –fullscale use full scale images to detect keypoints
  • –ptgui activate compatibility with PTGui

So which one should I choose?

The default is Autopano-SIFT-C. It is robust and yields good results. But if you are using fisheye lenses, Match-n-Shift is probably a better option. If you have a modern powerful multi-core CPU, you may want to give pan-o-matic a try.

All of them, though are subject to patents – either SIFT or SURF. It is your responsibility to make sure that you are not violating a patent in your jurisdiction of residence.

Matchpoint solves the patent issue for feature detection, and hopefully by the end of the Google Summer of Code we’ll have a patent-free solution also for the feature matching.