--- title: Pi pan-tilt for huge images, part 3: Image processing & raw workflow author: Chris Hodapp date: October 4, 2016 tags: photography, electronics, raspberrypi --- This is the third part in this series, continuing on from [part 1][part1] and [part 2][]. The last post was about integrating the hardware with Hugin and PanoTools. This post still involves those tools somewhat, but is more about the image processing pipeline I've been using. Thus far (aside from my first stitched image) I've been using a raw workflow where possible. That is, all images arrive from the camera in a lossless format, and every intermediate step works in a lossless format. To list out some typical steps in this: - Acquire raw images from camera with [raspistill][]. - Convert these to (lossless) TIFFs with [dcraw][]. - Process these into a composite image with [Hugin][] & [PanoTools][], producing another lossless TIFF file (for low dynamic range) or [OpenEXR][] file (for [high dynamic range][hdr]). - Import into something like [darktable][] for postprocessing. This is possibly overkill. I do it anyway. # Acquiring Images I may have mentioned in the first post that I'm using [ArduCam's Raspberry Pi camera][]. This board uses a 5-megapixel [OmniVision OV5647][ov5647]. (I believe they have [another][arducam_omx219] that uses the 8-megapixel Sony OMX 219, but I haven't gotten my hands on one yet.) If you are expecting the quality of sensors and exposure control that even an old entry-level DSLR camera provides, this board will probably disappoint you. The sensor is tiny, it's noisy, and capturing still images is quite slow (6-7 seconds, in my tests). However, if you are accustomed to basically every other camera that is within 2-3 times the price and is meant to interface directly with a computer of some kind (USB webcams and the like), I think you'll find it quite impressive: - It comes in versions with CS lens mount, C mount, and M12 mount. CS-mount and C-mount lenses are plentiful from their existing use in security cameras, generally inexpensive, and generally good-quality (and for a bit extra, ones are available with electrically-controllable apertures and focus). M12 lenses are... plentiful and inexpensive, at least. I'll probably go into more detail on optics in a later post. - [raspistill][] will provide 10-bit raw images straight off the sensor (see its `--raw` option). Thus, we can bypass all of the automatic brightness, saturation, contrast, and whitebalance which are great for snapshots and video, but really annoying for composite images. - Likewise via [raspistill][], we may directly set the ISO speed and the shutter time in microseconds, bypassing all automatic exposure control. I'm mostly using the CS-mount version, which came with a lens that is surprisingly sharp. (TODO: Put example images here.) If anyone can tell me how to surpass all this for $30 (perhaps with those GoPro knockoffs that are emerging), please tell me. Reading raw images from the Raspberry Pi cameras is a little more convoluted, and I suspect that this is just how the CSI-2 pathway for imaging works on the Raspberry Pi. In short: pass the `--raw` option to raspistill, and it will produce a JPEG file which contains a normal, lossy image, followed by a binary dump of 10-bit raw Bayer data from the sensor. # Converting Raw Images People have already put considerable work into converting these rather strange raw image files into something more sane (as the Raspberry Pi forums document [here][forum1] and [here][forum2]) - like the [color tests][beale] by John Beale, and 6by9's patches to dcraw, some of which have made it into Dave Coffin's official [dcraw][]. I've had to use 6by9's version of dcraw, which is at . As I understand it, he's trying to get the rest of this included into official dcraw. On an older-revision ArduCam board, I ran into problems getting 6by9's dcraw to read the resultant raw images, which I fixed with a [trivial patch][dcraw-pr]. However, that board had other problems, so I'm no longer using it. (TODO: Explain those problems.) - TODO: Give dcraw options and why - TODO: 8-bit vs. 16-bit - TODO: Example of 10-bit raw (does it get anything?) # Hugin & PanoTools Stitching - Alongside exposure fusion, Hugin supports merging exposures into HDR. Interestingly, this doesn't require bracketing. If the scene contains a wide range of lighting across a larger area, and individual shots are at different exposures to account for this, this may still require HDR. (TODO: Example of this) - Bracketing might still be a requirement, however, if a single shot spans too wide an exposure range. Do keep in mind that this sensor provides only 10 bits and is noisy. # Postprocessing - darktable already uses floating-point internally for everything, so it gladly handles [OpenEXR][] files (which store pixel values in floating point). [part1]: ./2016-09-25-pi-pan-tilt-1.html [part2]: ./2016-10-04-pi-pan-tilt-2.html [raspistill]: https://www.raspberrypi.org/documentation/raspbian/applications/camera.md [dcraw]: https://www.cybercom.net/~dcoffin/dcraw/ [Hugin]: http://wiki.panotools.org/Hugin [PanoTools]: http://wiki.panotools.org/Main_Page [OpenEXR]: http://www.openexr.com/ [hdr]: https://en.wikipedia.org/wiki/High-dynamic-range_imaging [darktable]: http://www.darktable.org/ [ArduCam]: http://www.arducam.com/camera-modules/raspberrypi-camera/ [ov5647]: http://www.ovt.com/uploads/parts/OV5647.pdf [arducam_omx219]: http://www.arducam.com/8mp-sony-imx219-camera-raspberry-pi/ [beale]: http://bealecorner.org/best/RPi/ [forum1]: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=44918 [forum2]: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=92562 [dcraw-6by9]: https://github.com/6by9/RPiTest/tree/master/dcraw [dcraw-pr]: https://github.com/6by9/RPiTest/pull/1