130 lines
6.8 KiB
Markdown
130 lines
6.8 KiB
Markdown
---
|
|
title: Pi pan-tilt for huge images, part 2: Hugin & PanoTools integration
|
|
author: Chris Hodapp
|
|
date: October 4, 2016
|
|
tags: photography, electronics, raspberrypi
|
|
---
|
|
|
|
In my [last post](./2016-09-25-pi-pan-tilt-1.html) I introduced some
|
|
of the project I've been working on. Those of you who thought a
|
|
little further on this might have seen that I made an apparatus that
|
|
captures a series of images from fairly precise positions, and then
|
|
completely discards that position information, hands the images off to
|
|
[Hugin][] and [PanoTools][], and has them crunch numbers for awhile to
|
|
calculate *the very same position information* for each image.
|
|
|
|
That's a slight oversimplification - they also calculate lens
|
|
parameters, they calculate other position parameters that I ignore,
|
|
and the position information will deviate because:
|
|
|
|
- Stepper motors can stall, and these steppers may have some
|
|
hysteresis in the gears.
|
|
- My pan and tilt axes aren't perfectly perpendicular.
|
|
- The camera might have a slight tilt or roll to it.
|
|
- The camera's [entrance pupil][] may not lie exactly at the center of
|
|
the two axes, which will cause rotations to also produce shifts in
|
|
position that they must account for. (More on this will follow
|
|
later. Those shifts in position can also cause parallaxing, which
|
|
is much more annoying to account for.
|
|
[No, it's not the nodal point. No, it's not the principal point.][npp])
|
|
|
|
That is, the position information we have is subject to inaccuracies,
|
|
and is not sufficient on its own. However, these tools still do a big
|
|
numerical optimization, and a starting position that is "close" can
|
|
help them along, so we may as well use the information.
|
|
|
|
Also, these optimizations depend on having enough good data to average
|
|
out to a good answer. Said data comes from matches between features
|
|
in overlapping images, say, using something like [SIFT][] and
|
|
[RANSAC][]. Even if we've left plenty of overlap in the images we've
|
|
shot, some parts of scenes can simply lack features like corners that
|
|
work well for this (see chapter 4 of
|
|
[Computer Vision: Algorithms and Applications][szeliski] if you're
|
|
really curious). We may end up with images for which optimization
|
|
can't really improve the estimated position, and here a guess based on
|
|
where we think the stepper motors were is much better than nothing.
|
|
|
|
If we look at the [PTO file format][pto] (which Hugin & PanoTools use)
|
|
in its i-lines section, it has pitch, yaw, and roll for each image.
|
|
Pitch and yaw are precisely the axes in which the steppers move the
|
|
camera (recall the pictures of the rig from the last post); the roll
|
|
axis is how the camera has been rotated. We need to know the lens's
|
|
angle of view too, but as with other parameters it's okay to just
|
|
guess and let the optimization fine-tune it. The nominal focal length
|
|
probably won't be exact anyhow.
|
|
|
|
Helpfully, PanoTools provides tools like `pto_gen` and `pto_var`, and
|
|
I use these in my script to generate a basic .pto file from the 2D
|
|
grid in which I shot images. All that's needed is to add up the steps
|
|
taken to reach each shot, convert steps to degrees, which for these
|
|
steppers means using 360 / 64 / 63.63895 = about 0.0884 (according to
|
|
[this][steps]), and making sure that the positive and negative degrees
|
|
correspond to the right direction in each axis.
|
|
|
|
With no refining, tweaking, or optimization, only the per-image
|
|
stepper motor positions and my guess at the lens's FOV, here is how
|
|
this looks in Hugin's fast preview:
|
|
|
|
[{width=100%}](../images/2016-10-04-pi-pan-tilt-2/hugin-steppers-only.jpg)
|
|
|
|
*(This is a test run that I did inside of [Hive13][], by the way. I
|
|
used the CS-mount [ArduCam][] and its included lens. Shots were in a
|
|
14 x 4 grid and about 15 degrees apart. People and objects were
|
|
moving around inside the space at the time, which may account for some
|
|
weirdness...)*
|
|
|
|
Though it certainly has gaps and seams, it's surprisingly coherent.
|
|
The curved-lines distortion in Hugin's GUI on the right is due to the
|
|
[projection][], and perfect optics and perfect positioning information
|
|
can't correct it. Do you recall learning in school that it's
|
|
impossible to put the globe of the world into a flat two-dimensional
|
|
map without distortion? This is exactly the same problem - which is
|
|
likely why Hugin's GUI shows all the pictures mapped onto a globe on
|
|
the left. That's another topic completely though...
|
|
|
|
Of course, Hugin pretty much automates the process of finding control
|
|
points, matching them, and then finding optimal positions for each
|
|
image, so that is what I did next. We can also look at these
|
|
positions directly in Hugin's GUI. The image below contains two
|
|
screenshots - on the left, the image positions from the stepper
|
|
motors, and on the right, the optimized positions that Hugin
|
|
calculated:
|
|
|
|
[{width=100%}](../assets_external/2016-10-04-pi-pan-tilt-2/hugin-comparison.png)
|
|
|
|
They sort of match up, though pitch deviates a bit. I believe that's
|
|
because I shifted the pitch of the entire thing to straighten it out
|
|
(or perhaps it did this automatically to center it), but I haven't
|
|
examined this in detail yet.
|
|
|
|
A full-resolution JPEG of the result (after automated stitching,
|
|
exposure fusion, lens correction, and so on), is linked below:
|
|
|
|
[{width=100%}](../assets_external/2016-10-04-pi-pan-tilt-2/hive13-20161004-fused.jpg)
|
|
|
|
It's 91 megapixels. The full TIFF image is 250 MB, so understandably,
|
|
I didn't feel like hosting it, particularly when it's not the
|
|
prettiest photo or the most technically-perfect one (it's full of lens
|
|
flare, chromatic aberration, overexposure, noise, and the occasional
|
|
stitching artifact).
|
|
|
|
However, you can look up close and see how well the details came
|
|
through - which I find pretty impressive for cheap optics and a cheap
|
|
sensor.
|
|
|
|
[ArduCam]: http://www.arducam.com/camera-modules/raspberrypi-camera/
|
|
[forum-raw-images]: https://www.raspberrypi.org/forums/viewtopic.php?p=357138
|
|
[raspistill]: https://www.raspberrypi.org/documentation/raspbian/applications/camera.md
|
|
[25mm-lens]: https://www.amazon.com/gp/product/B00N3ZPTE6
|
|
[Hugin]: http://wiki.panotools.org/Hugin
|
|
[PanoTools]: http://wiki.panotools.org/Main_Page
|
|
[entrance pupil]: https://en.wikipedia.org/wiki/Entrance_pupil
|
|
[npp]: http://www.janrik.net/PanoPostings/NoParallaxPoint/TheoryOfTheNoParallaxPoint.pdf
|
|
[steps]: https://arduino-info.wikispaces.com/SmallSteppers?responseToken=04cbc07820c67b78b09c414cd09efa23f
|
|
[SIFT]: https://en.wikipedia.org/wiki/Scale-invariant_feature_transform
|
|
[RANSAC]: https://en.wikipedia.org/wiki/RANSAC
|
|
[hive13]: http://hive13.org/
|
|
[projection]: http://wiki.panotools.org/Projections
|
|
[szeliski]: http://szeliski.org/Book/
|
|
[pto]: http://hugin.sourceforge.net/docs/manual/PTOptimizer.html
|