Panorama scripting in a nutshell - PanoTools.org Wiki

# Panorama scripting in a nutshell

From PanoTools.org Wiki

This is a short document describing the various Open Source tools and techniques available for working with panoramas in a non-GUI environment. The examples are based on Linux experience, but most of them should work on OS X or Windows with minor modifications at most.

This document is also useful for understanding what happens 'under the hood' in hugin itself and recommended reading for anyone looking to work with the hugin sourcecode.

# Why would you do that?

Creating and manipulating panoramas is something that should obviously be done in a graphical tool[*]. This is true, but perhaps you find that you are spending more time operating a computer than being a photographer. This is an indication that something is wrong - The computer exists to do the boring work and scripting is one way to make this happen.

# PanoTools 'Scripting'

The file format first used in Helmut Dersch's Panorama tools is often referred to as a 'script', but a 'PanoTools' script is really a data file describing a 'panorama stitching project'.

Although these original PanoTools are less used today, the file format is alive and well with several variations used as project files by PTGui[*], PTAssembler[*], hugin and related tools. These files are simple plain-text, sometimes it is useful to modify them directly with a text editor - Further on we will introduce various methods for manipulating them via the 'programming' kind of scripts.

# Simple command-line stitching

Lets start with a basic but common use case: Photographers with quality panoramic tripod heads[*] can reproduce identical sets of photos every time, in this situation there is no need to create a new stitching project for every panorama, it is enough to use a single project as a template.

This example takes a pre-existing project called template.pto, created with three photos. It uses nona for remapping, and enblend for blending the remapped photos into a finished TIFF image:

     nona -o out -m TIFF_m template.pto DSC_1234.JPG DSC_1235.JPG DSC_1236.JPG
     enblend -o finished.tif out0000.tif out0001.tif out0002.tif

The first nona command creates three remapped TIFF images called out0000.tif, out0001.tif etc... the list of input images can be substituted with any set of identically sized photos. The second enblend command uses these TIFF files as input and merges them to a file called finished.tif.

# Creating hugin projects on the command-line

The entire panorama creation process can be performed on the command-line, and therefore scripted, doing this involves a series of steps similar to the workflow in a GUI tool.

## Generating the project file

Most of hugin tools take a pto file as input and output. So the first step is to create this pto file. For this purpose use pto_gen.

      pto_gen *.jpg

will put all jpg files in the current directory into one project file with a default name. You can also specify single images and your own project name

      pto_gen -o project.pto image1.jpg image2.jpg image3.jpg

You can specify more options to the project (see pto_gen for details).

## Generating control points

There are a number of tools available for generating control points from groups of photos, like cpfind. Here is an example using cpfind

      cpfind --multirow -o project.pto project.pto

Hint: Other control points generator, such as autopano-sift, autopano-sift-c[*], panomatic and match-n-shift[*] take a list of images instead of a project. In this case the pto generation and control point generation are one step. Here is an example for autopano-sift-c:

     autopano-sift-c --projection 0,50 project.pto DSC_1234.JPG DSC_1235.JPG DSC_1236.JPG

This will create a .pto project with control points (if any) linking the three photos, note that the projection format (f0, rectilinear) of the input photos and approximate horizontal angle of view (v50, 50 degrees) have to be specified.

## Pruning control points

This still isn't a finished panorama, at the very least this involves optimising the camera positions and then rendering the output.

You could go ahead and optimise this project file straight away, but this can be a bit hit and miss. First it is a good idea to clean up the control points. There are currently two useful tools for cleaning control points: celeste removes points from areas of sky and ptoclean[*] removes points with large error distances, first celeste_standalone:

     celeste_standalone -i project.pto -o project.pto

Then ptoclean[*]:

     ptoclean -v --output project.pto project.pto

Note that in both cases we are overwriting the input project.pto file with the cleaned output. If you want you can create intermediate temporary files and keep each step in case you need to backtrack.

(Other control point cleaners are ptscluster[*] and the new cpclean)

## Optimising positions and geometry

Up to now, the project file simply contains an image list and control points, the images are not yet aligned, you can do this by optimising geometric parameters with the autooptimiser tool:

     autooptimiser -a -l -s -o project.pto project.pto

Now you could just skip ahead and render the output if photometric optimisation isn't required.

By adding the -m switch also photometric optimisation is done. So the output project is optimised for positions, geometry and photometric parameters in one run:

     autooptimiser -a -l -s -m -o project.pto project.pto

(PToptimizer[*] is an alternative geometrical optimiser)

## Optimising photometric parameters

Photos have photometric exposure variables which can be optimised in much the same way as aligning positions. Doing this can remove brightness, white balance[*] and vignetting differences between photos, giving better blending or allowing creation of HDR images.

For doing an automatic photometric optimisation like Hugin's assistant, run autooptimiser with -m switch:

     autooptimiser -m -o project.pto project.pto

For full manual control for photometric parameters use vig_optimize. This program does the equivalent job for photometric parameters as autooptimiser does for geometric parameters. You need to indicate in the .pto project which parameters are 'variables'. For this task you can edit the .pto file or use a tool such as ptovariable[*]:

     ptovariable --vignetting --response --exposure project.pto
     vig_optimize -o project.pto project.pto

## Setting output options

For setting the output option (like on the stitcher tab) use pano_modify:

       pano_modify -o project.pto --center --straighten --canvas=AUTO --crop=AUTO project.pto

This command will straighten the pano, center it on the canvas, calculates the optimal size and calculates the optimal output region.

## Example work flow similar to assistant in Hugin GUI

We will use the same project file for all single steps. We always overwrite the same project file. You can also use another filename for every step.

1) Generate pto file

        pto_gen -o project.pto *.jpg

2) Find control points with cpfind, with celeste to ignore clouds

        cpfind -o project.pto --multirow --celeste project.pto

3) Control point cleaning

        cpclean -o project.pto project.pto

4) Find vertical lines

        linefind -o project.pto project.pto

5) Optimize position, do photometric optimization, straighten panorama and select suitable output projection

        autooptimiser -a -m -l -s -o project.pto project.pto

6) Calculate optimal crop and optimal size

        pano_modify --canvas=AUTO --crop=AUTO -o project.pto project.pto

7) Now create stitching makefile

        pto2mk -o project.mk -p prefix project.pto

8) And finally create the final panorama

        make -f project.mk all

# Stitching hugin projects on the command-line

## Rendering images

The hugin tool for remapping and distorting the photos into the final panorama frame is nona, it uses the .pto project file as a set of instructions:

     nona -m TIFF_m -o project project.pto

This nona command creates one remapped TIFF image for each of the input photos, these will be named project0000.tif, project0001.tif, project0002.tif etc...

(PTblender is an alternative renderer to nona)

## Blending

nona can do rudimentary assembly of the remapped images, but a much better tool for this is enblend, feed it the images, it will pick seam lines and blend the overlapping areas:

     enblend -o project.tif project0000.tif project0001.tif project0002.tif ...

Finally this produces the stitched panorama: project.tif

# Makefile stitching system

This simple stitching process works well on the command-line, but it gets increasingly complicated when dealing with bracketed stacks of photos, these are necessary for exposure blending with enfuse or HDR creation with hugin_hdrmerge[*].

Luckily hugin understands about stacks and is capable of breaking-up a project into each of the stitching tasks: rendering, assembling stacks and blending. These targets are written to a standard Makefile formatted file whenever a project is saved or stitched, e.g when you save project.pto using the hugin GUI, project.pto.mk is saved also. Stitching is then performed not by the hugin GUI, but by GNU make which processes the targets in the Makefile, resulting in the final stitched panorama.

## Creating Makefiles with pto2mk

This functionality is also available for use with any .pto project with the command-line pto2mk tool:

      pto2mk -o project.pto.mk -p project project.pto
      make -f project.pto.mk

One of the advantages of using make on the command-line is that re-running this command will only redo necessary work. If none of the input files have changed, then make will finish immediately; however if one of the input or intermediate files has changed, say by modifying an alpha channel, then only the tasks necessary to integrate that image into the final panorama will be redone.

## make options

make supports multiple targets, by default hugin creates an 'all', it also creates a 'clean' target that deletes intermediate files. So you can clean up afterwards like so:

     make -f project.pto.mk clean

Internal variables can be overriden by environment variables using the '-e' flag, so to stitch using the alternative nona-mask[*] tool instead of nona, use something like this:

     NONA=nona-mask make -e -f project.pto.mk

Alternatively variables can be simply appended to the end of any make command, this does exactly the same thing:

     make -f project.pto.mk NONA=nona-mask

In a shell such as bash, setting the environment will modify internal variables for the rest of the session:

     export ENFUSE='enfuse -l 29'

(note that this will override any existing useful enfuse options)

## Parallel make

make supports parallel processing for machines with multiple cores/processors with the '-j' parameter, so for a 16 core system this command will stitch using 16 parallel processes:

     make -j 16 -f project.pto.mk NONA='nona -t 1'

(note that by default nona uses all available cores, in this case it needs to be limited to one thread so that 16 instances of nona can run at the same time)

## Stitching queues

Processing a series of projects one after another is simple enough, this command will stitch all the projects in a folder:

     for file in *.pto.mk; do make -f $file; done

For more advanced command-line stitching, a tool like distmake will manage a queue of Makefile stitching jobs, and even load-balance multiple machines over a network.

## Makefile plugins

A Makefile can be extended with additional targets by creating a second Makefile that includes the original hugin .pto.mk Makefile. For instance a plugin Makefile called 'Makefile.psd.mk' might define a new target called 'ldr_psd' that assembles a multilayer PSD file for manual blending in a tool such as the Gimp[*]:

     include $(PTO).mk
     ldr_psd : $(LDR_LAYERS) $(LDR_BLENDED)
             PTtiff2psd -o $(LDR_REMAPPED_PREFIX_SHELL)_layered.psd \
             $(LDR_LAYERS_SHELL) $(LDR_BLENDED_SHELL)

You can then use this plugin by calling the ldr_psd target and supplying the name of the relevant .pto project:

     make -f Makefile.psd.mk ldr_psd PTO=project.pto

Hugin installs with an example Makefile.equirect.mk plugin with some common add-on tasks such as creating basic QTVR, PanoSalado[*] and SPi-V[*] files.

## Useful Makefile variables

Each filesystem PATH in this table has a matching PATH_SHELL version with special characters escaped - This is the version that should be used in the 'command' part of each Makefile rule.

$(INPUT_IMAGES) | list of input files
$(INPUT_IMAGE_1) | first input file
$(PROJECT_FILE) | .pto project file
$(HUGIN_PROJECTION) | output projection type in panotools numeric format
$(HUGIN_HFOV) | output horizontal angle of view in degrees
$(HUGIN_WIDTH) | output width in pixels
$(HUGIN_HEIGHT) | output height in pixels
$(DO_LDR_BLENDED) | set if 'normal' stitching with no exposure blending set
$(DO_LDR_STACKED_BLENDED) | set if stitching with 'exposure blending' set
$(DO_HDR_BLENDED) | set if HDR merging set
$(LDR_REMAPPED_PREFIX) | normal blended output prefix
$(LDR_EXPOSURE_REMAPPED_PREFIX) | exposure blended output prefix
$(HDR_STACK_REMAPPED_PREFIX) | HDR merged output prefix
$(LDR_BLENDED) | normal blended output file
$(LDR_STACKED_BLENDED) | exposure blended output file
$(HDR_BLENDED) | HDR merged output file
$(LDR_LAYERS) | list of normal unblended output files
$(LDR_STACKS) | list of exposure blended stack files
$(LDR_EXPOSURE_LAYERS) | list of blended exposure layers
$(HDR_STACKS) | list of HDR merged stack files

# Holding it all together with panostart

The .pto.mk Makefile helps with the stitching at the end of the process, but doesn't help with the bits at the beginning such as: identifying which photos go in which panorama, generating control points, pruning and optimising.

panostart[*] is a prototype tool that does it all. Given a list of photos in a folder, it identifies likely panoramas by comparing EXIF timestamps and writes a meta-Makefile containing: rules for generating .pto projects with match-n-shift[*], rules for generating corresponding .pto.mk Makefiles with pto2mk, rules for stitching the projects with make, plus some other rules for creating useful distributable output.

A typical panostart[*] command for dealing with a folder full of photos taken with a pocket point-and-shoot camera on auto-exposure might look like this:

     panostart --output Makefile --projection 0 --fov 50 --nostacks --loquacious *.JPG

Processing this Makefile is simple:

     make

Some of the resulting panoramas will have position, cropping and stitching errors. Fix these by editing the relevant .pto projects in hugin or another tool, when you type make again on the command-line, only those projects that have changed will be restitched.

Another advantage of doing things this way is reproducibility: delete all the (enormous) TIFF output panoramas and all you need to keep permanently are the photos, the meta-Makefile and the associated .pto projects. If you ever need to recreate the TIFF panoramas in the future, just type make. If you know the name of the TIFF file you want, just type 'make project.tif' and only a single panorama will be processed.

# Panotools::Script for manipulating project files

The name is a historical mistake, Panotools::Script[*] is a perl library for working with hugin .pto projects, though it could be extended in the future to cover any of the related file formats. Many of the command-line tools described in this document are written with this library.

Here is a trivial script, all it does is open and save a .pto project, it also resets the output ('p' Panorama line) angle of view to 360 degrees (v360), the projection format to equirectangular (f2) and the output pixel size to 4096x2048 pixels (w4096 h2048):

     #!/usr/bin/perl
     use Panotools::Script;
     my $pto = new Panotools::Script;
     $pto->Read ($ARGV[0]);
     $pto->Panorama->Set (v => 360, f => 2, w => 4096, h = 2048);
     $pto->Write ($ARGV[0]);

Here's slightly more complex script, this iterates over the input images and increments the white balance[*] making the panorama slightly 'warmer':

     #!/usr/bin/perl
     use Panotools::Script;
     my $pto = new Panotools::Script;
     $pto->Read ($ARGV[0]);
     for my $image (@{$pto->Image})
     {
         my $Er_new = $image->{Er} * 100 / 95; # red value
         my $Eb_new = $image->{Eb} * 95 / 100; # blue value
         $image->Set (Er => $Er_new, Eb => $Eb_new);
     }
     $pto->Write ($ARGV[0]);

# Full list of command-line tools

## Control point generation

- cpfind - Hugins own control point detector
- autopano-sift-c[*] - All in one keypoint generator and matcher
- match-n-shift[*] - All in one wrapper around align_image_stack and autopano-sift-c[*]
- panomatic - All in one alternative to autopano-sift-c[*]
- align_image_stack - Generates control points for stacked photos
- ptoanchor - add control points to a Hugin project
- ptochain - add control points to a Hugin project between consecutive photos
- ptobind - Join the ends of linked photos in a Hugin project
- ptofill - add control points to a Hugin project between likely overlapping photos
- generatekeys - Keypoint generator
- autopano - Keypoint matcher, part of autopano-sift-c[*] and not to be confused with autopano

## Project file modification

- pto_gen - Generates a new pto project from image files
- match-n-shift[*] - (see above) Default usage is to initialise a new .pto project from EXIF data
- ptoset - Change global parameters and project options
- ptovariable[*] - Set typical optimisation variables
- pto_var - Manipulate image variable, can also set optimisation variables
- pto_lensstack - Manipulate lenses and stacks
- pto_mask - Add mask to project file
- pto_template - Apply template to project file
- ptsed - Set any project parameter, convert .pto into PanoTools format
- ptosort[*] - Sort control points or images, remove duplicate control points
- ptopath - Remove file paths in project files
- transform-pano[*] - Roll, pitch and yaw rotation of .pto projects
- ptocentre - Rotate projects to put images in panorama centre
- pano_modify Change output parameters of project file

- ptomerge[*] - Join two or more projects
- pto_merge Merges two or more Hugin project files (this is a different tool to ptomerge above)
- ptosplit[*] - Extract subsets of images from a project into a new project

## Control point pruning

- ptscluster - Clean 'bad' control points based on distances and position
- celeste_standalone - Clean sky control points
- ptoclean[*] - Clean 'bad' control points based on standard deviation error
- cpclean - An improved version of ptoclean[*], part of the hugin project

## Optimisation

- autooptimiser - Optimise geometry of .pto projects
- PToptimizer[*] - Optimize geometry of PanoTools format projects
- vig_optimize - Optimise photometric parameters of .pto projects

## Rendering

- nona - Default hugin rendering engine
- nona-mask[*] - Wrapper around nona for using external bitmap masks
- PTmender - Render PanoTools format projects

- PTAInterpolate[*] - Create intermediate images from PanoTools projects

## Blending

- enblend - Merge partially overlapping images with multiresolution splines
- enfuse - Merge overlapping images with exposure fusion
- PTroller - Merge partially overlapping images with no seaming
- PTmasker - Compute stitching masks for input by PTroller
- enblend-mask - Wrapper around enblend for using external bitmap masks
- enfuse-mask - Wrapper around enfuse for using external bitmap masks
- deghosting_mask - Creates mask for removing ghosting in images
- hugin_hdrmerge[*] - Merge bracketed images for HDR generation
- enblend-svg - Wrapper around enblend for specifying input images via SVG files

- process-masks - Allow 'positive' masking in external masks for enblend
- tif2svg - Assemble multiple TIFF images into SVG files for enblend-svg
- PTblender - Correct colour and brightness of overlapping images

## Misc

- panostart[*] - All-in-one panorama workflow via Makefiles
- gigastart - All-in-one assembly of multi-row panoramas
- pto2mk - Create platform specific Makefile for stitching a project
- gigatile[*] - Create a Makefile for stitching a project as multi-resolution tiles
- ptodummy[*] - Generate missing input photos to debug .pto projects
- calibrate_lens[*] - Automatic lens calibration using straight-line detection
- pafextract[*] - Tool to use panoglview as a zenith and nadir editor[*]

## Chromatic aberration

- fulla - Correct lens distortion and chromatic aberration
- tca_correct - Calculate chromatic aberration in a photo

## Querying

- ptograph - Draw undirected graphs of projects
- ptoget - Query any project parameter
- ptoinfo - Generate a report on a project file
- PTinfo - Display info about a panotools generated TIFF file
- panoinfo - Display information about installed libpano13[*] library
- pano_trafo - transform between image and panorama coordinates (and vice versa)

## TIFF files

- PTtiff2psd - Join multiple TIFF images into a multilayer PSD file
- PTtiffdump - Compare two TIFF files
- PTcrop - Remove unwanted empty space in TIFF files and replace with offsets
- PTuncrop - Replace offsets in TIFF files with empty pixels

## Output formats

- qtvr2img - Extract six cubefaces from a cubic QTVR
- cubic2erect - Render six cubefaces into a single equirectangular image
- qtvr2erect - Render a cubic QTVR into a single equirectangular image
- jpeg2qtvr - Assemble six JPEG cubefaces into a cubic QTVR
- erect2qtvr[*] - Render a single equirectangular image into a cubic QTVR

- erect2cubic - Create a .pto project for extracting cube faces from an equirectangular image
- erect2planet - Create a .pto project for extracting a 'little planet' from an equirectangular image
- erect2mercator - Create a .pto project for extracting a mercator view from an equirectangular image

## Older tools

Helmut Dersch's PTOptimizer and PTStitcher were the original command-line tools. These are not really recommendable any more as they require an older, buggier version of the libpano12[*] library, they are closed source so don't work on modern systems such as OS X or 64bit Linux, and are lacking in features compared to modern replacements such as autooptimiser and nona.

# Useful Image processing tools

- ImageMagick[*] - Many utilities for converting, manipulating and resizing images
- GraphicsMagick - A fork of ImageMagick with many improvements
- libtiff - Supplied with useful tools such as tiffcp for joining and recompression, and tifficc for adding ICC colour profiles
- exiftool[*] - All purpose reading, setting and copying of EXIF data
- exiv2 - EXIF manipulation, useful for setting the file modification date from the EXIF date
- jpegtran - Lossless rotation of JPEG images
- ufraw-batch - Command-line RAW processing
- dcraw[*] - Command-line RAW processing

--Bruno[*] 21:23, 2 May 2009 (UTC)

Retrieved from "http://wiki.panotools.org/index.php?title=Panorama_scripting_in_a_nutshell&oldid=15561[*]"