• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    April 2008
    M T W T F S S
    « Mar   May »
     123456
    78910111213
    14151617181920
    21222324252627
    282930  
  • Archives

Control Points Generators, the more the merrier.


There is a big choice of control point generators in the latest hugin installer. Many of them are experimental. Some work better for some situations than others. Variety is positive as there is likely to be a solution to most situations. But it is also daunting. Which one to choose, and how? here a few pointers. Moreover, the default preferences settings as set by the installer are written down, so you don’t have to be afraid of changing them and not know how to change them back.

Match-n-Shift

Match-n-Shift is Bruno Postle’s idea. Images are remapped to a conformal projection before the identification process. Despite the added step, match-n-shift is not significantly slower. For fisheye lenses, the quality of control points is better than non-remapped generators. Currently it only works with the good old autopano and it is subject to the same patents constraints, but efforts are underway to make it work with Matchpoint.

Default preference settings: -f %f -v %v -c -p %p -o %o %i

Autopano-SIFT Perl

Autopano-SIFT Perl is the latest incarnation of the classic SIFT-based control point generator that has been used by hugin users for years, and its use is limited by the SIFT patent in some countries, notably the United States. The shell wrapper script has been rewritten in Perl by Bruno Postle so that it can be compiled for and run on Windows (where Perl is optional). Linux and OSX users have had access to this tool for a long time.

Default preferences settings: –noransac –points %p –output %o %i

Autopano-SIFT-C

Autopano-SIFT-C is Tom K. Sharpless reengineering of Autopano-SIFT Perl. Tom is passionate about precision and efficiency. He initially used the full scale image to achieve best possible precision, but there is a cost to that: double the maximum dimension and it quadruples the time needed to process that single picture. This gets compounded as the pictures are matched against each other. Autopano-SIFT-C current default is 1600 pixels, but in my installer I have set it to 800 pixels, the value used by match-n-shift and by autopano-SIFT Perl, which yields excellent results.

Autopano-SIFT-C is more efficient and thus slightly faster, though I could not notice the difference with my average project. It also does some remapping, but I found match-n-shift much more elegant. The main difference is that Autopano-SIFT-C has a monolithic approach. Everything happens within a single file: scaling, reprojection, detection, matching, so it is less flexible to combine and cooperate with other tools.

Last but not least, the algorithm is still SIFT, so patents may apply.

Default preferences settings: –maxdim 800 –maxmatches %p %o %i
–maxdim (default 1600) => set to 800 to compare with a-c-c

Edit: I’ve been made aware of a recently introduced expandos in hugin that I had forgotten. –maxmatches %p %o %s will use a conformal projection. A practical show of how right Agos’ comment below is!

Matchpoint

Matchpoint was Zoran Mesec’s 2007 Google Summer of Code project. He wrote it as a drop-in replacement for the generatekeys program that came with autopano, so it was easy for Bruno Postle to adapt the Autopano-SIFT-C wrapper to it. Unfortunately Matchpoint still has a problem dealing with transparency masks, so it can’t be used with match-n-shift.

Matchpoint is the only feature detector in this article to be free of patent. It is still experimental but yields useful results. In the upcoming Google Summer of Code participation Onur Küçüktunç will develop a feature matcher to go with, mentored by Alexandre Jenny, the author of Autopano Pro.

Default preferences settings: –noransac –points %p –output %o %i

Pan-o-matic

Panomatic is the newest kid on the block. It is an implementation of the SURF algorythm (which is also patent protected in some jurisdictions) by Anael Orlinski. It is built to extract maximum power from modern multi-core processors and comes in two flavors. The standard Pan-o-matic runs on modern CPUs, that means Pentium 4 and newer or AMD Athlon XP and newer. The NOSSE version runs also on older CPUs.

Default preferences settings: -o %o %i

A few interesting options for those who may want to try:

  • -n<number> sets the of cores to use (default: autodetect)
  • –fullscale use full scale images to detect keypoints
  • –ptgui activate compatibility with PTGui

So which one should I choose?

The default is Autopano-SIFT-C. It is robust and yields good results. But if you are using fisheye lenses, Match-n-Shift is probably a better option. If you have a modern powerful multi-core CPU, you may want to give pan-o-matic a try.

All of them, though are subject to patents – either SIFT or SURF. It is your responsibility to make sure that you are not violating a patent in your jurisdiction of residence.

Matchpoint solves the patent issue for feature detection, and hopefully by the end of the Google Summer of Code we’ll have a patent-free solution also for the feature matching.

17 Responses

  1. We should really, really implement a common interface. Users should choose their preferred control point finder, but the task should be no more difficult than selecting it. No arguments, no strings.

  2. You give as default option for autopano and matchpoint -noransac. Why?

    AFAIK this option should be a default for fish-eye lenses, but for “normal” lenses the “–ransac 1” option should be preferable.
    Is this correct or ..?

  3. @Harry: RANSAC is a method to exclude outliers from a statistic sample. To my understanding using RANSAC may exclude some bad points from the sample. I recommend *always* looking at the list of control points and manually remove bad ones such as those on moving objects.

  4. Thanks for your response. My comment was maybe a bit too short.
    I know what ransac does or is supposed to do.
    I agree that you should always look at the generated control points, but I understood that the “–ransac 1 ” option already removed some of the “garbage” (as mentioned this understanding came from the link you mentioned among quite some other web articles).
    From what I’ve found on the internet I understood that due to the deformation in fish-eye images (especially at the edges) this method was not “solid” enough to detect the “real” outliers and that it was better not to use it at all for those fish-eye images.

    However, when using “normal” rectilineair lenses it does a good job according to the articles and topics I read.
    I use it almost always on my “rectilineair” images and I have the “idea” (no statistic analysis at all) that it helps.

    Note: maybe the new conformal approach with match-n-shift and autopano-sift-c for fish-eye images can improve the input images so much that ransac also works fine on the intermediate images.

  5. @Harry: it seems logical and likely that the new conformal reprojection will make RANSAC perform on fisheye images like it does on rectilinear images. That said, the problem as I understand it is in the sample size, i.e. the quantity of CPs. I use pre-calibrated lens parameters and get away with 3-5 CPs per image pair. The risk is then for a significant quantity of bad CPs to be taken for good CPs by RANSAC, removing the good CPs. Bottom line: if you use a lot of CPs, RANSAC helps in most cases. If you use few CPs, it may get you in a situation where adding the CPs manually is a more efficient choice. I’ll do more testing with match-n-shift and if I find that it helps, I’ll change the default on RANSAC, but it is obvious that my installer is geared at fisheye lens users like me, while your bundle is geared at rectilinear lens users like you, and what we need is a simple algorithmic mechanism to identify the type of project and set the appropriate parameters.

  6. match-n-shift disables ransac if the input field-of-view is more than 60 degrees.

    This isn’t based on any testing, just my experience that ransac doesn’t help much with wide-angle fisheyes, and a presumption that the same is true for rectilinear photos too.

  7. I am a bit confused by the number of tools supplied and their purposes. Nice to have this overview including parameters, but where to enter them?
    There seem to be tools that need other tools like Matchpoint. (“Unfortunately Matchpoint still has a problem dealing with transparency masks, so it can’t be used with match-n-shift.” -> with what should it be used? And how? Where to specify?)
    In the hugin preferences dialogue there are only the “traditional” two options to choose from. So where to enter the path and the arguments?
    In the dialogue there is an explanation of arguments containing %s (“PanoTools script”). Reading the edit concerning autopano-sift-s I understand that %s means “use a conformal projection” for a-s-c. So do same argument letters mean different things depending on what autopano program they are used with? Very confusing! We need checkboxes.

  8. Choosing a CP gen should be a drop-down affair with options in check boxes.
    Leave the command line parameter for truly advanced stuff.

  9. @kosmarnik: you are completely right – and the drop down should be in the stitcher tab (since the choice is project-dependent) while the command line parameters can stay in the preferences.

  10. @Levy:
    It’s a simple thing to add, when do you think it’ll get in the build?

  11. @kosmarnik: it’s not that simple when we are in a feature and string freeze and working to have the first stable release in over a year. You are welcome to join the mailing list for further discussions, and even more welcome to fetch the code from SVN and dig in it.

  12. In the cp generator the presence of leafs and trees should be detected, perhaps with something similar to sky detection, because trees are likely to move themself for the breeze.

  13. Hi Yuval,

    do you think you can hyperlink each project?

  14. @Joergen: as far as I remember they are all available with the hugin installer, at least for Windows and maybe for Mac as well.

  15. sorry to revive this old post but I have a hunge problem. I can find the problem all over the internet but not the solution. Whatever control point generator I use I get the same error message namely that the temporary file could not be opened.
    But hugin is looking for the wrong file name, its looking for name.0.oto and the generated tmp file does not have that 0.oto ending :-(

  16. @Ovidiu: This is indeed an old post. Things have changed and the upcoming Hugin 2010.2.0 (currently at the second beta) will bring a new multi-row detection mode. I’ll have to write an update :-)

    For your specific support question about temp files: you need to provide more information (e.g.: what CP generator did you use? what version of Hugin? what operating system?, etc.) and this is not the right place. Head out to http://groups.google.com/group/hugin-ptx and post your support requests there.

  17. Hi guys,

    i would be glad if somebody could give me a hint, how to get match-n-shift working on win7?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s