CaDNET: An End-to-End Plenoptic Camera-Based Deep Learning Pose Estimation Approach for Space Orbital Rendezvous

This article presents a novel deep learning-based approach for relative pose estimation using a focused plenoptic camera for space rendezvous operations of on-orbit servicing (OOS) applications. Plenoptic cameras, also known as light-field cameras, are similar to traditional cameras but have an array of microlenses in front of the sensor. This configuration offers several advantages, such as software-based refocusing and increased image quality in low-light conditions while maintaining an extended depth of field. Moreover, it enables the derivation of 3-D depth images from the same light field, making it possible to use a single camera as a stereo vision system for autonomous space rendezvous navigation challenges. We propose a robust deep learning solution suitable for uncooperative close-range rendezvous missions, such as debris removal, based on a bidirectional long short-term memory (BiLSTM) network and a convolutional neural network (CNN), to accurately estimate the target’s pose from images captured by a plenoptic camera mounted rigidly on the chaser satellite. We validate the proposed approach, named cascaded deep network (CaDNET), using on-ground data obtained from a designed experimental setup. Through the quality experimental results achieved, we demonstrate the feasibility of adopting the plenoptic camera as an AI-based relative navigation solution for space rendezvous missions.