Video processing for UAVs and drones may be performed on the outputs of both electro-optical and infrared (EO/IR) camera payloads for a variety of applications. This processing may be performed either on board the aircraft, or at a base station following transmission of the video by the drone.
Onboard Video Processing Solutions
Onboard video processing solutions using embedded edge computing devices provide the advantage of low latency, as there is no need to wait for the video to be streamed elsewhere and then for the results to come back. These onboard devices, however, add extra weight and power consumption to the drone, and so may not be suitable for SWaP (size, weight and power)-constrained platforms.
Onboard video processing devices may have single or multiple input and output channels, and may be able to deal with multiple video formats such as analogue, Full HD digital, and H.264 compressed video. These devices may be based around dedicated GPU (graphics processing unit) technology and be able to run sophisticated artificial intelligence and machine learning software.
Remote Video Processing Technology
Offloading processing to a ground station or cloud computing server allows UAVs to save on component weight and power drain. However, transmitting the video information will take up extra bandwidth, and if the processing is being performed for a mission- or safety-critical application such as navigation, a reliable always-on connectivity solution will be required to ensure continued functionality. Video data will also need to be compressed before it is transmitted, and running the required compression algorithms can be computationally intensive.
Video processing technology may be utilised to support a wide variety of drone applications, such as ISR (Intelligence, surveillance and reconnaissance), search and rescue, surveillance and autonomous operations. It can be used to pick out targets of interest such as animals, humans and hostile vehicles and drones, as well as to estimate the continued motion of these targets to aid in tracking. With the aid of infrared cameras, video processing can be used to detect urban fires and wildfires, as well as to enhance video footage under low-light conditions.
This technology can also be used to aid in autonomous operations such as precision takeoff and landing, detect-and-avoid, and drone swarming. It may be an essential part of navigation in GPS-denied environments, providing information for navigation algorithms such as SLAM (simultaneous localization and mapping) in which an autonomous vehicle such as a USV (unmanned surface vessel) or UGV (unmanned ground vehicle), maps its environment and localizes itself without the need for any external reference.
Video Processing Software
Video processing software for footage captured by drones and unmanned systems can be applied to a range of uses, such as upscaling HD video capabilities, removing frame-to-frame jitter, correcting rotation caused by roll, isolating objects from the background when processing tracking video, and aiding autonomous landing.