Typhoon Edge: Multi-Camera Intelligence for Autonomous Orbital Rendezvous and Robotic Servicing

Future spacecraft performing orbital rendezvous, inspection, servicing and robotic manipulation will need to operate with increasing levels of autonomy. During close-proximity operations, a spacecraft may need to process information from several cameras simultaneously, understand the relative position and orientation of another spacecraft, identify interfaces or objects, monitor a robotic arm and react to changing conditions without relying entirely on commands from the ground.

Typhoon Edge is designed to provide the onboard computing capability required for this new generation of autonomous space missions.

By combining the deterministic processing and flexible interfaces of a Microchip PolarFire SoC FPGA with the high-performance AI capability of an NVIDIA Jetson AGX Orin Industrial, Typhoon Edge creates a heterogeneous computing platform capable of receiving, processing and interpreting data from multiple sensors and cameras directly onboard the spacecraft.

The Jetson AGX Orin Industrial module provides up to 248 TOPS of AI performance and is specifically positioned for demanding edge AI and robotics applications, including multiple concurrent AI pipelines and high-speed sensor processing.

From Multiple Cameras to One Understanding of the Environment

An autonomous servicing spacecraft may use several cameras, each providing a different perspective of the operational environment.

For example, a mission could include:

• A long-range navigation camera for detecting and tracking the target spacecraft during approach.
• A medium-range camera for relative navigation and target pose estimation.
• Stereo or multiple cameras for depth perception and three-dimensional reconstruction.
• A robotic arm camera for monitoring the end effector and its interaction with the target.
• A wide-angle situational-awareness camera for detecting unexpected objects or changes around the servicing vehicle.
• Dedicated inspection cameras for identifying docking fixtures, grapple points, connectors, damaged structures or other mission-specific features.

Rather than treating these cameras as isolated sensors, Typhoon Edge can act as a central onboard intelligence platform, bringing multiple data streams into a common processing architecture. The result is a more complete understanding of the spacecraft’s environment.

The PolarFire SoC: Deterministic Sensor and Data Handling

The PolarFire SoC FPGA provides the deterministic and highly programmable part of the Typhoon Edge architecture. This device can be used to manage camera and sensor interfaces, handle data movement and buffering, implement mission-specific processing functions and provide deterministic processing alongside the AI subsystem. This is particularly valuable when multiple high-bandwidth data streams must be managed simultaneously.

The FPGA architecture can be configured to perform functions such as camera data acquisition and synchronization, timestamping and correlation of sensor data, image pre-processing, region-of-interest extraction, data formatting and routing, sensor fusion support, real-time hardware processing, interface conversion between different payload and camera interfaces & deterministic control functions.

By handling these functions close to the incoming data source, the FPGA can prepare and organize sensor information before it is delivered to the AI processing environment. This creates an efficient division of responsibility: the PolarFire SoC manages deterministic, interface-intensive and mission-specific processing, while the NVIDIA accelerator performs computationally intensive AI and vision tasks.

Once cameras data has been prepared and delivered to the NVIDIA Jetson AGX Orin Industrial, Typhoon Edge can execute advanced computer-vision and AI workloads. Potential onboard processing functions include object detection and classification, spacecraft recognition, relative pose estimation, tracking, optical flow analysis & robotic arm tracking.

The 248-TOPS Industrial version of Jetson AGX Orin is designed to support multiple concurrent AI application pipelines, making it particularly relevant where several sensors or cameras need to be processed at the same time.

Enabling Autonomous Orbital Rendezvous

Orbital rendezvous is a complex operation because the servicing spacecraft must continuously understand its position relative to a moving target. As the distance decreases, the type of visual information required can change significantly. At longer distances, the system may focus on detecting and tracking the target spacecraft. During the approach phase, higher-resolution camera data can be used to estimate relative position, velocity and orientation. At close range, the system may need to identify specific structural features, docking points or grapple fixtures.

Typhoon Edge can support this progression by processing multiple camera streams and combining the information into a common onboard representation of the mission environment.

• Long-range camera → Target detection and tracking
• Navigation cameras → Relative position and attitude estimation
• Stereo or multi-view cameras → Depth and 3D environment information
• Close-range cameras → Grapple point and interface recognition
• Robotic arm cameras → Manipulation monitoring and verification

This type of sensor-driven architecture aligns with the fundamental requirements of autonomous servicing systems. NASA describes autonomous real-time relative navigation as combining sensors, algorithms and processing to support safe rendezvous, alongside avionics that ingest sensor data and support rendezvous and robotic tasks.

Autonomous Robotic Arm Operations

One of the most demanding applications for multi-camera onboard intelligence is the control and monitoring of a robotic arm in orbit.

A robotic servicing mission cannot always rely on a single camera. During a manipulation operation, one camera may lose visibility because of spacecraft geometry, shadows, reflections or obstruction by the robotic arm itself. Multiple viewpoints can provide redundancy and improved situational awareness. Typhoon Edge can process information from cameras positioned around the servicing spacecraft and robotic arm to help answer critical questions:

• Where is the target object?
• What is its orientation?
• Where is the robotic end effector relative to the target?
• Is the arm approaching the correct interface?
• Has the object moved unexpectedly?

AI models running on the Jetson can identify and track relevant objects, while data from multiple cameras can be combined to improve confidence in the estimated position and orientation of the target. The resulting information can then be supplied to higher-level guidance, navigation and control software or to the robotic control system.

A Closed-Loop Onboard Intelligence Architecture

A possible Typhoon Edge processing chain could operate as follows:

Multiple Cameras and Sensors

↓

PolarFire SoC FPGA

Camera acquisition, synchronization, deterministic processing, buffering and data routing

↓

NVIDIA Jetson AGX Orin Industrial

AI inference, computer vision, object recognition, pose estimation and sensor fusion

↓

Autonomy Software

Mission logic, decision-making, trajectory assessment and robotic task planning

↓

Spacecraft GNC and Robotic Control

Thrusters, attitude control, robotic arm and end-effector commands

↓

Updated Camera and Sensor Data

This creates a continuous onboard perception-and-action loop.