• Subscribe

    Subscribe to This Week In Panospace by eMail.
    Subscribe in a reader
  • License

    Creative Commons License
    This work is © 2008-2012
    by Yuval Levy
    and licensed under a
    Creative Commons License.
  • Entries

    July 2018
    M T W T F S S
    « Dec    
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    3031  
  • Archives

Linear Panoramas (Mosaic) Tutorial

Another development cycle is coming to an end.  Bruno released libpano13 2.9.17 and paved the way for Hugin 2010.2.0.   This version of Hugin will bring a lot of new features, amongst others a versatile masking tool and finer grained control of the control point (CP) detection process, courtesy of  Thomas Modes.  Good news for Windows user as well: galvanized by Emanuele Panz and his new NSIS based installer that solves the issues with distribution of patented software, also the Windows users community is on the forefront this time.

Time to do some cool stuff with Hugin:  mosaics.  The basic functionality for mosaics has been in the repository for about a year, and it has matured well.  This morning I set out to shoot and stitch a mosaic.

Stitched mosaics differ from traditional stitched panorama in two major ways:

  • In an ideal mosaic, all images depict a single, perfectly flat subject.
  • The images are taken from different points of view.

Mosaics present a few challenges of their own.  Think of them as protrusions and intrusions from/into the perfectly flat imaginary plane that you are trying to shoot.

Shooting

Not much is flat in the real world.  If parallax prevented seamless concordance in a traditional stitched panorama, depth and perspective prevent seamless concordance in a mosaic.  While parallax can be almost completely eliminated with accurate camera positioning on the no-parallax point, perspective is a given and will be with us throughout the process and into the finished product.  You need to be aware of it while shooting a mosaic more than you need to be aware of moving objects when shooting a stitched panorama.  The tools to deal with it are the same: overlap (plenty!); masking; and, inevitable in this case, retouching/editing.

For this kind of shooting, I first walk parallel to the subject watching how protrusions and intrusions  move relative to the imaginary ideal plane as I move along.  I look for vertical (not necessarily straight)  lines in the plane that could act as stitching seams, i.e. will be unencumbered by protrusions in two adjacent shots.  If I can’t find such a virtual seam line I will have a hard time masking and retouching the seam.  In a second pass I shoot images straight at the subject, one between every pair of the still virtual seam lines, with a lot of overlap.  In a third pass, I take pictures at different angles to “get around” protrusions such as trees that hide areas of interest of the subject.

All images were shot hand held.  Like for traditional panoramas, constant exposure and color are important.

Order

I found that order matters.  I get best results when starting from an anchor image in the middle of the mosaic and working my way to the edges.  To simplify the work flow, once the images are loaded I re-order them in the Images tab, making sure the anchor image is the top one.

Setting the Stage

Start a recent version of Hugin.  You will be switching forth and back between the main window and the fast preview window, so if you have two displays, open them side by side.  Unlike Terry Duell’s suggestions I found that it is not necessary to add images individually to the project, but a little bit of pre-planning is necessary.  There will be some inevitable forth and back as this is an iterative process.

Generate Control Points

Bad news first:  you most likely will have to enter CPs by hand.  That said, it’s easy.

It is paramount that all CPs are on the same plane and no currently existing CP detector known to me meets the condition.  Even small depth errors (e.g. relief on a wall) has big impact.  Avoid CPs between non-contiguous images.  Three or four well distributed (i.e. not on a straight line) CPs are enough.  If vertical lines are present, add one vertical CPs on each image that has them.  Conceptually, the three corners of a triangle in space are enough to define a plane and you are looking to define the common plane between two images.

Start with a Calibrated Lens

I found mosaics optimization to be much more sensible to the starting parameters than spherical optimization; and the outcome to be much more sensible to lens distortion than traditional panoramas.  Get lens distortion out of the way before starting.  If you have saved calibration data for your lens load it now.  Else, here’s a quick fix.

In the Fast Preview window:

  • Hide all images but one with plenty of lines that should be straight in the real world.
  • Select rectilinear projection.
  • Center the preview on the visible image and zoom in on it with the field of view (FoV) sliders.

Your goal is to make straight lines look straight, and the Fast Preview window will give you valuable visible feedback. In the Control Points tab, set a few line CPs:

  • Select the visible image into both the left and right panes and start adding CPs on lines.
  • For vertical lines, click on one end of a vertical line in the left pane and on the other end in the right pane.
  • For all other lines do the same, but check that the CP-type drop down says “another line” and the table says “line” and a number.  The name of horizontal CPs is misleading.  Horizontal lines are to identify the horizon itself, not lines parallel to it.

With six well distributed lines you can already get a good approximation.  In the Optimizer tab:

  • Select to optimize “the custom parameter below”
  • Select “only use control points from images visible in the preview window”
  • Deselect all parameters
  • Select only Pitch and Roll of the above image
  • Run a first optimization (and adjust the view in the Fast Preview window if necessary)
  • Deselect all parameters
  • Select Barrel
  • Run a second optimization
  • Select Pitch, Roll and Barrel and optimize them all
  • Now deselect Barrel, and leave the lens parameters alone

There is more to proper lens calibration, but this should be enough approximation for the project at hand.  Confirm visually that the lines are straight in the Fast Preview window before continuing.

Start from the Center

Start with only your anchor/calibration image visible  in the middle of the Fast Preview (in rectilinear projection) and all other images hidden.

In the Optimizer tab:

  • Select to optimize “the custom parameter below”
  • Select “only use control points from images visible in the preview window”
  • Optimize for Pitch and Roll of the anchor image.  Make sure it’s X/Y/Z parameters are 0 (in the Images tab).

Optimize Image after Image

Repeat the following for each image, always verifying the visual result in the Fast Preview after each optimization.

In the Fast Preview window:

  • Visually check that the current result is approximately OK
  • Adjust the FoV sliders if necessary
  • Make the next image visible

In the Images tab:

  • Select the image that just became visible in the Fast Preview window
  • Edit it’s X value.  Enter a negative value if it is to the left or a positive one if it is to the right.  Over time you will get a feel for the magnitude.  Start with +/-1.  Make sure it is roughly positioned where it belongs, slightly more to the outside than to the inside.

In the Optimizer tab

  • Select to optimize “the custom parameter below”
  • Select “only use control points from images visible in the preview window”
  • Optimize for Yaw, Pitch,  Roll, X, Y, Z of the newly visible image

Repeat for all images.  At the end do one more optimization run, optimizing Yaw, Pitch, Roll, X, Y, Z for all images but the anchor image, and only Pitch and Roll for the anchor image.

Improve Slowly

This is a step by step, image by image process.  And we’re not finished yet.  You may find that the project needs further tweaking.  It’s not an issue of mathematical precision but rather one of controlling the position and warping in critical areas where the different perspectives from the different pictures collide.

Terry suggested dragging the image in the Fast Preview window instead of the above step in the Images Tab.  And indeed the Fast Preview has a Move/Drag tab with mosaic mode.  Unfortunately dragging an image will automatically drag all other images connected by CPs, so unless you do this only once before adding CPs, dragging won’t work. I found the way through the numeric input in the Images tab to be preferable.  It enabled me to go back and rework the positioning of individual images without losing the work done on CPs.

Move/drag in the Fast Preview window is useful associated with cropping functionality.  I use manual cropping in this case, it’s fast and intuitive.  When the mosaic looks good in the Fast Preview window, head to the Stitcher tab and render it.  Make sure to output both the blended panorama and all remapped images by checking the appropriate check boxes.  Hit the Stitch Now button, enter a prefix and go get a deserved break while the computer churns away.

Masking

Hugin has a built in masking tool.  Masking in the input images is more useful for panoramas than for mosaic, because in a well shot panoramas it is reasonable to assume that the images align naturally and the masking is only needed to exclude / include details.  With mosaics, the assumption is wrong, and I prefer to do masking in output images, already warped and aligned, together with editing and some other unspecified fiddling to achieve a visually pleasant result.

Post Processing

Below is a scaled rendering of the blended mosaic.  No post-processing applied yet.  The original is about 21.000 pixels wide.  It’s not bad, but “perspective collisions” are visible at the seams and will require retouching.  It’s time to prepare the project for your favorite image editor.

Copy all the rendered images into a single TIFF file.  On the command line, enter:

$ tiffcp prefix* layeredmosaic.tif

Open the resulting file with Gimp.  The layers will be perfectly aligned.  Add mask layers to them and start editing, or export to a Photoshop file if you are more comfortable with that image editor.  My Photoshop CS 2 (quite old, but still does the job) is not able to open properly the multi-page TIFF file.

Improvements

This was a quick proof of concept / tutorial.  If I was to do something serious with this image, I would probably re-shoot it, taking advantage of the building regularity and adding more images, one in between every pair of columns.

Conclusion

Reaching this advanced stage in the image creation process was relatively easy with Hugin.  Now, depending on your level of perfectionism and intended purposes, there are still hours of editing ahead before the file can be declared good to print.  I don’t have enough wall space available for this 3.5m / 11 feet long mosaic.  If you have and are interested in this or similar commissioned work, I’d like to hear from you.

Bend to Please

Equirect

equirectangular 360°

Earlier this year I used the above full spherical panorama to test the new projections as they were added to libpano. You can view it from the inside here.

Those new projections devised by Tom, Bruno and Daniel, were not all. In his quest to satisfy our sense of what the world really looks like without being limited by the narrow field of view of the traditional rectilinear perspective, Tom Sharpless asked for permission to modify and use the image. A few weeks later he came up with a real WOW! He is currently working on implementing this and more into his Panini Perspective Tool.

Tom was meant to discuss his work at LGM but prior commitments and the lack of a teleporting facility makes it impossible for him to attend. We agreed that I will present his slides, and host a discussion on Panoramic Perspective Control. The main idea: use of panoramic photography techniques, coupled with computing, to achieve new images that break out of the physical boundaries set by traditional optics and appeal to the human sense of aesthetics.

As part of the talk, I’ll show how I followed into Tom’s footsteps and produced the image below.

PVsqueezed to be optically pleasant.

Tools

You will need

  • Tom’s Panini Perspective Tool, or if you want to do this in high resolution a recent (2009) version of Hugin.
  • A recent version of the GIMP. I had to build 2.6.x on my own box until Ubuntu 9.04 came along. Building the GIMP from code is not trivial but relatively well documented around the web.
  • A recent version of MathMap. Unfortunately MathMap is not yet packaged with Ubuntu. The 1.3.4 tarball will do. To my knowledge, nobody has got MathMap to work on Windows yet.
  • Tom’s PVSqueeze.mm script. Put it in the appropriate MathMap folder or copy&paste it into the “Expression” tab.

Process240° x 150° equirectangular pannini

  1. As so often, we start from an equirectangular image, the universal panorama format.
  2. Load the equirectangular in Hugin and extract an Equirectangular Pannini view of the area of interest. The flexibility of extracting an Equirectangular Pannini view is higher in Panini, but Panini’s resolution is limited by the video card’s memory. Hugin can do higher resolutions but has the disadvantage that the eye distance (horizontal compression) in libpano is fixed (patch, anyone?)
  3. Load the perspective into the GIMP.
  4. From the GIMP menu, open the Filter MathMap.
  5. Load the PVSqueeze filter.
  6. Click Preview to see the low-resolution live image.
  7. In MathMap’s User Values tab, adjust the values and see the live image morph until you like it.

MathMapping

From MathMap’s homepage: “MathMap is a GIMP plug-in which allows distortion of images specified in a simple programming language. For each pixel in the generated image, a script is evaluated which returns a color value. The script can either refer to a pixel in the source image or can generate colors completely independent of the source.” MathMap’s is a project of Mark Probst.

PVSqueeze

Using Tom’s PVSqueeze filter:

  1. Set the horizontal field of view (hfov) and eye distance (eyeDist) sliders roughly right for your input image.
  2. Adjust the vanishing point (VP) position (VPX, VPY), and the slopes of the 4 perspective edges (XXangl) so they line up with the appropriate edges in your image. There aren’t any drawn lines, but the change of image shape is pretty easy to see. Note you have to set the projection center Y identical to the VP Y by hand, or you get a black triangle in the middle (bug; but misaligning them can help you see just where the VP is).
  3. Adjust the shift modifiers for the upper and lower Vees to suit.  Basically these swing the pixel shift direction between straight up and down — which preserves verticals like chair legs — and radial with regard to the VP — which makes most floor patterns look nicer but also creates the curved diverging lines. The shift direction varies with vertical position, from vertical at the VP to radial farther out.  XVlim controls the vertical level at which this transition starts, and XTwid the width of the transition zone.  You can set them for pure vertical or pure radial shift, or some blend.
  4. Hit OK, review the full size result in the Gimp window; save or go back and adjust some more.

Next

So we used some smart math and arbitrary parameters to bend a captured panoramic image to please our taste. In doing this, we connected back to the Renaissance tradition of perspective on a mathematical basis. The resulting high-resolution picture is currently being printed on canvas to be displayed at the Ultra Wide Views exhibition that opens May 6 in Montréal, Canada. At Libre Graphics Meeting I will show more detail and I hope we can discuss how to classify the different projection types and make them more accessible to graphic artists.

Hugin 102: Fisheye Lens Basics

The basic tutorial applies for fisheye lenses lenses too, with a few differences.

First, we need to enable Rotational Search. In the Preferences make sure that Rotational search is enabled in the Control Points Editor tab.

After loading the images into a project, we need to check if hugin knows that those are fisheye images. In the Camera and Lens tab, look at the Lens type (f) column. If the lens type is wrong, select any of the lines and change its type to either Circular fisheye or Full frame fisheye in the Lens section of the Geometric tab. Which of the two is easy: circular if the edges of the sensor frame are outside of the edges of the lens, and full frame if not.

For full frame fisheye, this is it – continue with the basic tutorial. For circular fisheye there is one additional step in the Crop tab.

Select all the images that will have the same crop (i.e. same orientation) and drag the mouse inside the image to match the circle approximately around the edge of the lens. You can leave the Always center Crop on d,e box checked. In the above screenshot it is unchecked because I use it to correct for small shifts of the no-parallax-point position in relation to the pano head nodal point, but that’s for another tutorial.

Hugin 101: Setting Control Points Manually

A version of hugin that does not have automatic control point (CP) detection ships with the Nodal Ninja while we are waiting for the outcome of a Google Summer of Code project to produce the second part of a completely patent-free CP generator. Nevertheless, using it is easy. Because manual CPs are more important in hugin than in commercial stitching tools, the mechanism to set them has been engineered to be as efficient as possible. Once you get used to it, it is really just two approximate clicks and one right-click.

This is a beginner’s tutorial for how to manually set control points in hugin, but it is also useful for the pros dealing with a difficult stitch that require careful manual placement of CPs.

Initially, hugin Nodal Ninja Special Edition starts with the assistant.

Click on the 1. Load Images… button, select some images and open them.

If you try to click on the 2. Align… button, you will see a notification and the process will end.

If after dismissing the notification you are not on the control points tab, please select it. Make sure the checkboxes auto fine-tune and auto estimate are set.

The process in a nutshell:

  • click on the approximate position of the CP on the left image.
  • click on the approximate position of the CP on the right image.
  • right-click to add the point to the list.
  • repeat until you have enough CPs.

In detail:

When you click on the left CP, hugin will zoom in on it. Once zoomed, you may displace the CP to a precise position intended, but this is not necessary.

When you click on the right CP, just make sure that the box surrounding the mouse pointer also surrounds the area where the CP should be located. Hugin automatically fine-tunes inside that box, so that even if the left CP was not in the precise position intended, both CPs match the same position.

Fine tuning also kicks in after moving one of the points. And if for any reason fine tuning does not find the corresponding point, it will give a warning.

When the control point is where you want it, right-click and it will be added to the list of control points. Repeat until you have enough control points. Then select the next image pair and repeat for all adjacent image pairs. Simple, isn’t it?

Edit the Nadir, Part II

In Part I we saw how Hugin can be used to extract the nadir from a full spherical panorama for retouching. While there are a number of other methods (as kindly indicated by readers of this blog in the comments), this method has one strategic advantage: it can be used to extract a layered view. This is what we will do next.

Below is a nadir view of a full spherical panorama. It has already been aligned. It was shot with a Sigma 8mm fisheye lens, six shots around at 60° distance, so there are six images around the nadir.

All steps described in part I until just before clicking the Save project and stitch button have been performed. Before clicking that button, tick the Remapped Images box in the appropriate colum. Then Save the project and stitch. In this example, when prompted for a filename prefix we enter “nadir”.

layered nadir panorama preview

After the rendering, there are seven new tiff files: nadir.tif is the composite, and nadir0000.tif – nadir0005.tif are the individual layers.

Now, start a command line interface. A lot of photographers seems to have a phobia of the command line interface, but it really does not take more than basic typographic skills to use it. At the prompt, work yourself into the folder where the seven tiff files have been created with the cd command. Once there, enter the command

$ PTtiff2psd -o nadir.psd nadir.tif nadir0000.tif nadir0001.tif /
  nadir0002.tif nadir0003.tif nadir0004.tif nadir0005.tif

This will merge the tiff files into a single Photoshop file. At the time of writing, there is a bug in PTtiff2psd – it will fail if more than five tiff files are merged. You can also load all the tiff files into Photoshop and use a Photoshop action to import them into a single file.

nadir layer edit photoshop

Edit the nadir in Photoshop. Specifically, edit the masks to determine the seam placement between the individual images. When finished save it. Then flatten the layer and either use the Panotools Photoshop Plug-ins or the steps illustrated in part I after the edit to merge the nadir back into the equirectangular image.

Edit the nadir

This is a full spherical panorama stitched with hugin. There is a visible white hole at the nadir. Difficult to edit, because it is strechted. This tutorial will demonstrate how to extract the nadir view, edit it, and insert it back into the panorama.

initial pano preview

Start hugin. In the Images tab, click on the Add individual images… button. Select the stitched equirectangular and load it. Hugin will promt for the Camera and Lens data. Enter the values below and click OK.

equirectangular lens

Click the Num. Transf. button in the Panorama preview window, enter the values below and click OK.

numerical transform

In the Stitcher tab, set the projection to Rectilinear and the Field of View to 90 horizontal and 90 vertical, like in the screenshot below. Press the Calculate Optimal Size button and the Save project and stitch button.

extract nadir

Open the resulting file with your preferred image editor. I currently use Photoshop in ubuntu. Edit and save as TIFF.

nadir edit

Back in hugin. In the Images tab, click on the Add individual images… button. Either you have saved the previous project with the equirectangular, or you can reload the equirectangular as in the first step of the tutorial. Add the edited nadir. At the Camera and Lens data prompt enter the values below and click OK.

nadir lens

Move the nadir image up the list on top of the equirectangular image. Select it in the list and set its pitch to -90.

nadir insert

Check the final panorama in the Panorama preview window. Then in the Stitcher tab, output the remapped images. Load them as layers into an image editor and they should align properly. Save and publish. Click on the thumbnail at the following page to interact with this panorama as a full screen VR.

This is just one method – there are plenty of them. The advantage of this method is that it can be used to quickly generate a layered nadir view – very useful if you have a number of shots covering the area and you want to be able to mask the one or the other. If you don’t need layered output, you’re better off using the Panotools Photoshop Plugins. Advanced tutorials will follow.

Hugin dans Gimp 2 efficace 2e édition

J’ai contribué une section sur l’assemblage de panoramiques avec hugin à la deuxième édition du livre Gimp Efficace de Cédric Gemy, disponible chez Amazon.fr.

I contributed a section about stitching panoramas with hugin to the second edition of Cédric Gemy‘s book Gimp Efficace.

Cover Gimp 2 Efficace