129 lines
5.8 KiB
Markdown
129 lines
5.8 KiB
Markdown
---
|
|
title: Pi pan-tilt for huge images, part 3: Image processing & raw workflow
|
|
author: Chris Hodapp
|
|
date: October 4, 2016
|
|
tags: photography, electronics, raspberrypi
|
|
---
|
|
|
|
This is the third part in this series, continuing on from
|
|
[part 1][part1] and [part 2][]. The last post was about integrating
|
|
the hardware with Hugin and PanoTools. This post still involves those
|
|
tools somewhat, but is more about the image processing pipeline I've
|
|
been using.
|
|
|
|
Thus far (aside from my first stitched image) I've been using a raw
|
|
workflow where possible. That is, all images arrive from the camera
|
|
in a lossless format, and every intermediate step works in a lossless
|
|
format. To list out some typical steps in this:
|
|
|
|
- Acquire raw images from camera with [raspistill][].
|
|
- Convert these to (lossless) TIFFs with [dcraw][].
|
|
- Process these into a composite image with [Hugin][] & [PanoTools][],
|
|
producing another lossless TIFF file (for low dynamic range) or
|
|
[OpenEXR][] file (for [high dynamic range][hdr]).
|
|
- Import into something like [darktable][] for postprocessing.
|
|
|
|
This is possibly overkill. I do it anyway.
|
|
|
|
# Acquiring Images
|
|
|
|
I may have mentioned in the first post that I'm using
|
|
[ArduCam's Raspberry Pi camera][]. This board uses a 5-megapixel
|
|
[OmniVision OV5647][ov5647]. (I believe they have
|
|
[another][arducam_omx219] that uses the 8-megapixel Sony OMX 219, but
|
|
I haven't gotten my hands on one yet.)
|
|
|
|
If you are expecting the quality of sensors and exposure control that
|
|
even an old entry-level DSLR camera provides, this board will probably
|
|
disappoint you. The sensor is tiny, it's noisy, and capturing still
|
|
images is quite slow (6-7 seconds, in my tests).
|
|
|
|
However, if you are accustomed to basically every other camera that is
|
|
within 2-3 times the price and is meant to interface directly with a
|
|
computer of some kind (USB webcams and the like), I think you'll find
|
|
it quite impressive:
|
|
|
|
- It comes in versions with CS lens mount, C mount, and M12 mount.
|
|
CS-mount and C-mount lenses are plentiful from their existing use in
|
|
security cameras, generally inexpensive, and generally good-quality
|
|
(and for a bit extra, ones are available with
|
|
electrically-controllable apertures and focus). M12 lenses
|
|
are... plentiful and inexpensive, at least. I'll probably go into
|
|
more detail on optics in a later post.
|
|
- [raspistill][] will provide 10-bit raw images straight off the
|
|
sensor (see its `--raw` option). Thus, we can bypass all of the
|
|
automatic brightness, saturation, contrast, and whitebalance which
|
|
are great for snapshots and video, but really annoying for composite
|
|
images.
|
|
- Likewise via [raspistill][], we may directly set the ISO speed and
|
|
the shutter time in microseconds, bypassing all automatic exposure
|
|
control.
|
|
|
|
I'm mostly using the CS-mount version, which came with a lens that is
|
|
surprisingly sharp. (TODO: Put example images here.) If anyone can
|
|
tell me how to surpass all this for $30 (perhaps with those GoPro
|
|
knockoffs that are emerging), please tell me.
|
|
|
|
Reading raw images from the Raspberry Pi cameras is a little more
|
|
convoluted, and I suspect that this is just how the CSI-2 pathway for
|
|
imaging works on the Raspberry Pi. In short: pass the `--raw` option
|
|
to raspistill, and it will produce a JPEG file which contains a
|
|
normal, lossy image, followed by a binary dump of 10-bit raw Bayer
|
|
data from the sensor.
|
|
|
|
# Converting Raw Images
|
|
|
|
People have already put considerable work into converting these rather
|
|
strange raw image files into something more sane (as the Raspberry Pi
|
|
forums document [here][forum1] and [here][forum2]) - like the
|
|
[color tests][beale] by John Beale, and 6by9's patches to dcraw, some
|
|
of which have made it into Dave Coffin's official [dcraw][].
|
|
|
|
I've had to use 6by9's version of dcraw, which is at
|
|
<https://github.com/6by9/RPiTest/tree/master/dcraw>. As I understand
|
|
it, he's trying to get the rest of this included into official dcraw.
|
|
|
|
On an older-revision ArduCam board, I ran into problems getting 6by9's
|
|
dcraw to read the resultant raw images, which I fixed with a
|
|
[trivial patch][dcraw-pr]. However, that board had other problems, so
|
|
I'm no longer using it. (TODO: Explain those problems.)
|
|
|
|
- TODO: Give dcraw options and why
|
|
- TODO: 8-bit vs. 16-bit
|
|
- TODO: Example of 10-bit raw (does it get anything?)
|
|
|
|
# Hugin & PanoTools Stitching
|
|
|
|
- Alongside exposure fusion, Hugin supports merging exposures into
|
|
HDR. Interestingly, this doesn't require bracketing. If the scene
|
|
contains a wide range of lighting across a larger area, and
|
|
individual shots are at different exposures to account for this,
|
|
this may still require HDR. (TODO: Example of this)
|
|
- Bracketing might still be a requirement, however, if a single shot
|
|
spans too wide an exposure range. Do keep in mind that this sensor
|
|
provides only 10 bits and is noisy.
|
|
|
|
# Postprocessing
|
|
|
|
- darktable already uses floating-point internally for everything, so
|
|
it gladly handles [OpenEXR][] files (which store pixel values in
|
|
floating point).
|
|
|
|
[part1]: ./2016-09-25-pi-pan-tilt-1.html
|
|
[part2]: ./2016-10-04-pi-pan-tilt-2.html
|
|
[raspistill]: https://www.raspberrypi.org/documentation/raspbian/applications/camera.md
|
|
[dcraw]: https://www.cybercom.net/~dcoffin/dcraw/
|
|
[Hugin]: http://wiki.panotools.org/Hugin
|
|
[PanoTools]: http://wiki.panotools.org/Main_Page
|
|
[OpenEXR]: http://www.openexr.com/
|
|
[hdr]: https://en.wikipedia.org/wiki/High-dynamic-range_imaging
|
|
[darktable]: http://www.darktable.org/
|
|
[ArduCam]: http://www.arducam.com/camera-modules/raspberrypi-camera/
|
|
[ov5647]: http://www.ovt.com/uploads/parts/OV5647.pdf
|
|
[arducam_omx219]: http://www.arducam.com/8mp-sony-imx219-camera-raspberry-pi/
|
|
[beale]: http://bealecorner.org/best/RPi/
|
|
[forum1]: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=44918
|
|
[forum2]: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=92562
|
|
[dcraw-6by9]: https://github.com/6by9/RPiTest/tree/master/dcraw
|
|
[dcraw-pr]: https://github.com/6by9/RPiTest/pull/1
|