• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    April 2014
    M T W T F S S
    « Dec    
     123456
    78910111213
    14151617181920
    21222324252627
    282930  
  • Archives

Google Summer of Code 2009 Wrap Up

Another summer is over. The maples in my neighborhood are changing color and we just finished our third Google Summer of Code (GSoC) participation, the best ever. Like the previous two editions, GSoC has been a catalyst beyond the five project slots generously allocated to us by Google. GSoC has been more than money. More than an opportunity to attract fresh contributors and get things done. It has been first and foremost a transformational opportunity, making an inherently insider organization an open and outward looking one.

Allocation

Google generously allocated five slots to us and this year we decided to allocate one of them to a smaller Open Source project. It seems to me that there is an ideal mentoring organization size (or at least minimum size) for GSoC participation. Smaller mentoring organizations require a similar amount of administrative effort as larger ones. Sharing is a way of enabling smaller projects to participate while limiting the burden on Google’s Open Source Programm Office.

There are many small Open Source projects in our related field, and in talks I had with them most expressed interest. I decided to work with Professor Sébastien Roy’s Vision 3D Lab at Université de Montréal and his LightTwist project for a few reasons:

  • I needed to make a case to our community that sharing is in our interest. Indeed there was a very short discussion about LightTwist’s relevance to the Hugin community.
  • Meetings and contacts between members of the Hugin community and of the Vision 3D Lab go back to Libre Graphics Meeting (LGM) 2007. We share a similar culture / attitude.
  • Last but not least, I was looking for more academic contacts to promote GSoC recruiting efforts.

The experience worked better than expected. We even organized a joint display of Hugin artwork projected with Lighttwist at LGM 2009. I warmly recommend other organizations to do the same: Next year give a lift to a smaller project!

Recruiting

This was my biggest deception this year. I did put a lot of effort upfront – contacting professors at relevant faculties in my region and beyond. The turnout – at least for us – was zero. Recruiting is indeed our bottleneck. This year again we had more mentors than allocated students, so we rotated mentoring responsibilities. Our recruiting needs improvement.

Students

One of the biggest question was to hire new students or returning ones? I am thankful to my fellow mentors for changing my mind on this one, and I hope our faithful returning students will forgive me. Initially I even convinced one of last year’s students to participate as a mentor. Then I asked him to submit a project proposal. Thank you, Tim, for your flexibility! Student’s life is IMO the most beautiful period in one’s lifetime, enjoy it while it lasts!

Selection

Three months are a short period. To make the most out of it candidates must be up to speed from day one. Following the lead of other mentoring organizations, we introduced qualification tasks. Despite some controversy amongst a few vocal students, it has worked well for us. Our goal was to determine basic proficiency with the tools. Other organizations, notably x264, have put more effort in the qualifying tasks so that they also determine if the candidate has the skill to complete the task. It may be more work, but it is definitely best practice IMO, and we should consider putting in this upfront effort.

Meetings

Remote collaboration works better after a physical handshake. I interviewed all serious candidates by phone and try to organize as many face to face meeting with community members as possible. More by lucky circumstances than by design we managed to meet all students this year: two at LGM in Canada and three at a dedicated meeting in the UK. These meetings have been very productive. Even face to face meeting did not prevent us from failing one student, but they are useful and we should try to bridge physical distance in the future as well.

Projects

Our two returning students did not have to get up to speed with the code base and could take on more ambitious projects. And indeed they did, beyond my dreams and expectations.

James Legg, mentored by Bruno Postle, refactored code deep into the core to implement his new layout model. The model works. Minor quirks in specific situations will be fixed. This project corrects a design weakness, unintended heritage of the pre-HDR time.

Tim Hugent, mentored by Tom Sharpless, produced an automatic lens calibration tool. The calibrated lens model still need a lot of validation work through practical use in the field. The potential benefits are improved precision and, beyond panorama making (and Hugin) better images from less than perfect lenses. This project was the first step in an exciting new direction.

Lukáš Jirkovský, mentored by Andrew Mihal, brought deghosting (the removal of moving artifacts from multiple exposures) to the next level and made it available to enfuse, the preferred choice for photo realistic exposure bracketing. Also this project will continue after the end of GSoC.

Yulia Kotseruba, mentored by Sébastien Roy, added new blending algorithms to LightTwist, making it possible to extend the projection surface vertically and opening up new fields of uses such as dome projection or, given enough wall space, a low-cost high-resolution IMAX replacement. Work continues and will be published in a scientific review.

Last but not least a word about Dev Ghosh, mentored by Daniel German. Unfortunately we had to fail Dev because he had not achieved the agreed (and revised) milestone. To his credit he got the mathematics right – his MatLab implementation works. Chances are that the code will be implemented in our code base, sooner or later.

A big THANK YOU to all the students for their effort and dedication!

Update: The only reason to fail Dev was quantity. The quality of his work was excellent. Daniel has implemented it in Libpano and it may become the first chunk of 2009 code to be officially released, with the upcoming libpano 2.9.15.

Integration

We found the right recipe for this last year and we recidivate. During GSoC students focus on implementation. They will integrate their code after GSoC. As an encouragement, they will be sponsored a panoramic head from Nodal Ninja, an industry leader that sees the benefits of supporting Open Source software that drives sales of its hardware. At the time of writing, the first project has been already integrated in trunk. Thank you, Bill Bailey, marketing director for Fanotec.

Community Development

On this year’s mentoring organization application form there was one interesting question about “criteria used to select community members“. This lead to some introspection, following which I drafted this community charter. Thank you, Google Open Source Program Office, for giving us food for thoughts.

Speed

GSoC projects have accelerated the pace of development. Our sequential development process with trunk freezes and long release cycles was no longer adequate. I’ve taken it upon myself to re-engineer it and at the time of writing we’re getting ready to release for the first time under the new, parallel process. I hope we can integrate and release all the code developed for GSoC 2009 before Christmas, and that our project team will increase it’s capacity to absorb and release code changes.

Generations

After graduating university I was offered a job at a company that I still admire (I accepted another job because it gave me the opportunity to expatriate). That company had a three year cycle: the first year you’re hired as a junior and trained on the job. if you did it well the second year your boss moves on and you put your imprint on the job, adding to it what you have learned. On the third year, you train a junior to replace you and if everything went well you’re promoted to the next job. This helped the company stay nimble and up to date. We need a similar attitude in our project. People come and go, and we need to groom the next generation of committers before the current generation fades out; and to document our processes and know how for the perusal of the next generation. GSoC has been a great recruiting ground and our next project leader may be amongst this year’s GSoC students.

Conclusion

Before our GSoC participations, joining Hugin was a steep learning curve. Still today we get consistent feedback from GSoC candidates and students that getting up to speed with the code base is the biggest hurdle they face. Things have improved. In 2007 with an SDK for Windows developers. In 2008 with the equivalent for OSX developers and with build documentation for different platforms. This year it was the release process, and there is more on the plate for the years to come – much of this thanks to GSoC. Thank you, Google!

Exhibition Video

The first Ultra Wide Views exhibition is over. Held at École Polytechnique de Montréal in the context of the Libre Graphics Meeting (LGM) 2009, it was a great success. Thanks to the hard work of  the Vision3D Lab at Université de Montréal; and of Mardigrafe Inc.; and thanks to the infrastructure kindly provided by École Polytechnique de Montréal; and last but not least thanks to the many participating artists.

Panoramic artwork has been shown on two state of the art projecting displays (a 360° cylinder and a high resolution array of projectors) and in the form of two dozens of high resolution giclées. The general public has seen images in a quality never seen before. The developers and artists participating at LGM where inspired and entertained by this display of practical applications of their work.

I look forward to continue collaborations with these artists and partners on more bleeding edge panoramic projects; and I look forward to LGM 2010 hoping that exhibiting art along the conference will become a tradition.

LGM Day 3, Day 4, Follow Up

Days went by so quick, so here is a summary.

Day three was intense. The presentations were very interesting, particularly those of Andrew Mihal about Enblend-Enfuse and about GPU stitching. In the evening we had the conference supper, a very pleasant social event at which I learned from about Øyvind’s Now By Then installation and got a chance to purchase one of the last “Architecture Fiver“‘s  that Stani brought along for his talk. He also patiently gave me an insight into the thinking pattern of curators, lifting my morale from the previous day.

On day four I had a near-death experience. Since Tom Sharpless could not attend, I picked up his slide and hosted the talk. In the morning I prepared my notebook in dual display mode, so that on one display I can run the demonstrations of Panini, MathMap and Hugin, while on the other display I had my notes, which I kept editing until shortly before my talk. Without saving. Disaster strikes when I connect the notebook to the projector. My notebook’s native resolution is 1400×1050, too much for the projector to handle. Must restart X. Lose all presets and notes! I froze on the spot. Had to cut on a few gimmicks such as recording the talk with the catadioptric lens or shooting a stitched panorama during the talk (I made the move with the camera around the tripod, but had no available brain cycles to even think of setting the exposure or pressing the real button. I felt like a zombie and was disappointed at my performance which I felt was terrible, although a look at its recording comforted me: it was not that bad after all, even if I forgot to say half the things I wanted to say, the live demo and the interaction with the public were not that catastrophic.

The good news from day four was that Sébastien and Vincent debugged the flat display in extremis so that people walking out of for the lunch break passed by and could admire Guillaume’s stunning Boulevard Bancel (we’ll gladly admit that once this was on screen and working we pulled the cables on the slide show so that nothing can go wrong. Proper slide show the next time).

Since the cafeteria was closed on Sunday, we had to order pizzas for lunch, which was great as it inspired more exchanges before the last presentations. In the end, I even got the honor of shooting the official group picture, and to offer some of the present teams a panorama inside a panorama in the Cyclorama, while our team helped folding up the exhibit canvases.

After protracted good byes, I drove eastwards with Alexandre and Pablo. Alexandre only joined us for a tour of Québec-City. Pablo continued with us to Boréalie before I drove him back to Montréal for his flight on Wednseday. The rest of the week, and the weekend, I had to catch up with pent up business. Particularly the weekend was difficult, with a few difficulties upgrading servers remotely from FreeBSD 6.3 to FreeBSD 7.2 (the worse thing is that all manipulations worked well on the guinea-pig server in the office that is pretty much an exact mirror of the servers to be upgraded).

I still made time on Sunday evening for another quick hop to Montréal, and I am happy I did. Joergen Geerds was visiting for the weekend with his girlfriend. Interesting people.

After

LGM Day 1

We’re at LGM. Yesterday  I picked up the Ultra Wide Views, Fahim in downtown and Pablo coming from Germany at the airport. We spent the late afternoon / early evening mounting all the canvas and the few photopaper prints; then we looked at Sébastien, Vincent and Jean-Ambroise set up the LightTwist driven Cyclorama. We had a bite with Benoît Ozell, his support for everything happening here at École Polytechnique de Montréal is fantastic. Dev Ghosh joined us after his plane landed.  Jim Watters arrived after dinner. I worked until 3 AM on my notebook.

This morning most things worked as planned but still neded some oiling. The exhibit of 360° views from a dozen artists on the Cyclorama was ready just in time for after Sébastien’s speech, with some minor details corrected afterwards. It was delighting to receive the visit of .Jean-Pierre Lavoie, one of the most successful panorama photographers in Canada, who has pioneered usage of immersive medias in the news. LightTwist made a great debut.

I am running on my last juices, but all the interesting people keep me going. It’s good to see familiar faces; to put a face on a name known from the mailing lists; to meet new people and hear about new ideas. This conference is refreshing. I’m capturing some of the moments on video, but won’t have time to process it for a while.

Andrew Mihal arrived a few minutes ago and Yulia Kotseruba is scheduled to arrive late tonight. This will complete one of the largest Hugin developers line-ups in a single location. Friday night is the conference’s official dinner, but we’ll have our team dinner Thursday night in a yet to be defined place.

Accepted Students for Google Summer of Code 2009

MascotteThe dice have rolled. For the third year in a row the Hugin community will mentor five Google Summer of Code students as they go about writing free software:

  • James Legg of Fast Preview fame will tackle the Layout Panorama Model mentored by Bruno Postle, improving the handling of stacks.
  • Tim Nugent of Celeste fame will tackle Straight Line Detection for Lens Calibration mentored by Tom Sharpless, improving the handling of fisheye lenses.
  • Lukáš Jirkovský, too young in 2008, has become a significant contributor to our community over the past year and will tackle Ghost Removal for Enfuse, mentored by Andrew Mihal.
  • Dev Ghosh will add Mosaic Mode to Hugin/Panotools, mentored by Daniel German.
  • Last but not least Yulia Kotseruba will add functionality like multiple blending to Lighttwist, mentored by Sébastien Roy.

I am particularly proud of the collaboration with Sébastien Roy on Lighttwist. First, because I believe there is a cultural fit between his Vision3D laboratory and the Hugin community. Second, because it is good to see academics releasing the result of their research (which, let’s not forget, is often funded with public taxpayer’s money) to the general public under an Open Source license.  And last but not least, because I think that Lighttwist is the natural extension to Hugin. I would dare to advance that Lighttwist is to panoramic photography in 2009 what QuickTimeVR was to it in 1999. In a private email exchange Ken Turkowski, inventor of QuickTimeVR and one of our team, acknowledge the analogy.

Then there is a sixth project that I care about. We have teamed up once again (and we hope to be luckier this time) with VideoLAN to bring QTVR playback to this popular media player. This time with an extra bonus: Wiimote control. León Moctezuma, who in 2007 added SPi-V playback to FreePV, will build on his expertise of FreePV and add the Wiimote on top of it, mentored by Antoine Cellerier of VideoLAN.

What’s next for the students? this article I wrote exactly one year ago applies.

Good Luck, students, I hope to congratulate you all in a few months.

Beam Me Up, Vincent!

Friday afternoon I went to Montreal to finalize a few things for the upcoming Ultra Wide Views 2009 exhibition. First we tested some images on Lighttwist. I did not expect that Vincent would surprise me with the first trial of spherical images. He only asked for one on Thursday. He took us on a very promising tour.

The soundtrack is from Freaked System and can be legally downloaded on Jamendo. I can’t repeat often enough how amazed I am at the quality of the music that can be found on Jamendo.

After putting our feet back on Earth, we walked to nearby École Polytechnique. Benoît Ozell has been very helpful organizing the logistics for LGM and by extension for the exhibition. We looked at some rooms for the Lighttwist projection and settled on… two. Yes, the intention is to have two different projection installations:

  • The initially planned 360° Cyclorama, for which I opened up contributions today – they will be soon visible in the gallery. They will complement videos and potentially even a live performance. 14 FullHD projectors, it’s going to be hot in that room.
  • A high resolution projection of the flat artworks, some of which are also meant to be printed. Most likely four FullHD projectors combined – Sébastien’s ambition is to demonstrate the affordability of Lighttwist over alternatives of similar resolution and wants to connect all those pixels to a single PC.

Things are shaping up very nicely. If you have appropriate material, get yourself an account and start uploading. Be part of this unique, ground-breaking exhibit of Ultra Wide Views.

Authoring for the Cyclorama

The Cyclorama is a cylindrical 360° display. Seven (or fourteen) projectors, driven by Lighttwist, surround the audience with a high definition image or movie – ambiophonic sound included. The following tutorial explains how to output panoramas for the Cyclorama with Hugin. I start with a full spherical panorama, but any Hugin project will do. Make sure the image covers 360° horizontal FoV (field of view) and at least 30° vertical FoV >30°. Images with smaller FoV will leave part of the screen blank.

Start Hugin and load the image(s). Since equirectangular images usually don’t have EXIF data, enter the image type and FoV as in the window below:

01_load_equirect1

Visually check the equirectangular in the Fast Preview window:

02_equirect_fast_previewOn the Stitcher tab, enter the following parameters:

  • Projection: Cylindrical
  • Field of View: horizontal 360°, vertical 38.5°
  • Panorama Canvas Size: width 7200, height: 800
  • Crop: 0,0,7200,600

03_stitcher_tab1The resolution of the Cyclorama is currently 7200 x 600 pixel, equivalent to 360°x29.3° in a cylindrical projection centered around the horizon. However we do not center around the horizon: the screen will be slightly higher to avoid shadows from the audience inside the Cyclorama. Set the horizon at 1/3 (or 200 pixels) from the bottom (although the human eye is very forgiving). To do so we output slightly more vertical FoV and crop it. Test the result in the Fast Preview window:

04_cyl_fast_previewThe white line is the crop line. When you’re happy with the result, stitch!

060608cesarea03a_cyclo

Exhibition

As a last step, log on to Ultra Wide Views, select the Cyclorama 2009 exhibition on the upload form and beam the result over. Thank you! It may be shown on the Cyclorama during LGM. The exhibition’s theme will be Horizons. The panorama used for this tutorial is the Mediterranean Sea seen from the coast of Israel in Cesarea. On the beach are the remnants of a two thousands years old Roman aqueduct.

Important Notes:

  1. Make sure your input image has enough resolution for an output at 7200 pixels horizontally. Full spherical panoramas in the range above 6000 pixels are OK.
  2. Hugin does not apply interpolation filters (yet). Ideally you either first resize the panorama to 7200 horizontal pixels and then run it through the above process, or you output at recommended resolution and resize it later. To do so, set the appropriate FoV values, then click on Calculate Optimal Size. Stitch. Open the result in your preferred image editor, resize and crop. Double-check that the resulting cropped image is 7200×600. Warning: some image editors do not resize well the 360° seam.
  3. Not all content is appropriate for cylindrical projections. The crop lines at +20°/-10° vertically often cut away important detail. Choose a panorama that fits the medium.

Lighttwist

Lighttwist is a volumetric projection system. It is currently used to project on the Cyclorama, a large cylindrical display (click the picture below to get inside it).

inside_lighttwist

It was invented by Université de Montréal professor Sébastien Roy and is being further developed under his direction by Vincent Chapdelaine-Couture and Louis Bouchard. It was presented to some of us two years ago on the fringes of LGM 2007. In 2009, LGM is coming back to Montréal and Lighttwist is back… with more than a twist.

Sébastien and his team joined us for the Google Summer of Code. In a week we’ll know if this summer we’ll see a Lighttwist related project sponsored by Google.

He also responded to the call for presentations to LGM and will likely present the latest development at LGM. The number of projectors have doubled from seven to 14. For the same cylinder. What’s this twist?

Last but not least, we’re working out the details to run a show on Lighttwist at LGM, with a live video performance, a couple of short movies and a slide show of panoramas. If you produce panoramic movies, or if you produce panoramic images (in cylindrical projection, 360°x29.3° – 7200x600px with the horizon at 1/3 from the bottom), contact me to be featured at the show.

With so many projectors, related computers and big screen, Lighttwist may seem intimidating. But it is not. This weekend Vincent published an addendum to the user guide for the minimalist Lighttwist configuration – on a single PC. Add a few video cards and a webcam and you’re all set for a start into volumetric projection. Did I mention that Lighttwist is Open Source?