Frame Gaps in Hyperspectral Orthomosaic

Frame gaps in hyperspectral products happen when the line-scanner's sample rate is insufficient for the aircraft's actual speed or attitude acceleration during flight. Because hyperspectral cameras are line-scan (aka pushbroom), they capture only a single pixel tall, N pixel wide (where N is typically 640 or 1020) frame at a user-defined static framerate. When a flight is planned at a set speed, that is the desired speed of the aircraft, but not what it will always perfectly hold. The aircraft may fly slightly faster or slower at times, especially in windy conditions, when flying excessively fast or slow, or when utilizing a high ratio of the aircraft's payload capacity. If the aircraft flies slightly faster than the desired speed for brief periods of time, the static framerate set for the hyperspectral camera is no longer at the correct framerate for the faster speed. This results in frame gaps; frames that were not captured because the sensor was capturing too slow for the current speed.

Additionally, instability in attitude, especially pitch, causes the same phenomenon. Even if the aircraft is holding speed well, a quick acceleration in pitch angle can result in missed scan lines as the sensor rotated past the next frame in sequence before it could collect that frame.

The few keys to tackling frame gaps from an operations perspective are to avoid excessively fast or slow flight (2m/s ≤  X ≤  8m/s). Propeller rotation can become unsteady with lower inertia at low speeds, and difficult to fine tune at high speeds as it approaches its max speed with the payload weight. These numbers are rough guidelines and will differ from aircraft to aircraft. We can also utilize a concept known as oversampling, increasing the framerate of the sensor beyond the nominal calculation for a given speed. For example, 60m altitude at 5m/s speed gives a nominal framerate with the Nano HP of 179 Hz, or 5.58ms per scan-line. If we set the framerate 10-20% faster, ~ 200-225 Hz or 5-4.5ms, we can capture scan-lines more frequently, thus giving ourselves forward overlap. The nominal frame period calculations give us a forward oversample of 0, meaning we basically need a cable cam to capture without gaps.

Depending on which visualization software you use, frame gaps will often show up as either black or white pixels, such as in the figure below, and contain no spectral information because they were not imaged. On the processing side, we have one option available: adjusting the back-projection error value. Max Backprojection Error is the maximum allowable backprojection error (in pixels) when matching EOPs to the trajectory. This parameter is used to ensure that the EOPs used for orthorectification are of sufficient quality. A smaller value will result in a more accurate orthorectification but may result in fewer EOPs being used, which can lead to gaps in the output mosaic. By increasing this parameter from the default 1 pixel to, say, 1.5-2 pixels, we'll fill some of these gaps by finding less spatially accurate solutions, but without fabricating data like a nearest neighbor sampling would do.