feature: Integrating the RAPTOR foundation policy (#26082)

* moving raptor bump compiles and raptor mode appears hovering with RAPTOR seems to work Using Raptor to execute offboard commands works (using multirobot f03825a5795a77c5a095f799eeb8e0b646fe7176 to feed the trajectory_setpoint). Requires more testing simplified rotmat runtime inference frequency multiple arming request response reflects actual readiness adjusting to fit IMU gyro ratemax relaxing control timing warning thresholds for SITL Using mode registration to signal if offboard commands should be forwarded to trajectory_setpoint instead of just hardcoding vehicle_status.nav_state == vehicle_status_s::NAVIGATION_STATE_OFFBOARD adopting new "request_offboard_setpoint" in raptor module replace offboard seems good mc_raptor: overwrite offboard parameter separate raptor config addendum Raptor off by default RAPTOR readme Loading raptor checkpoint from tar works. check if load was successful refactoring: cutting out the pure C interface to allow direct testing of the policy input/output behavior from the file, without fully loading it into memory first adapter not needed anymore ripping out test observation mode (not used in a long time) fixing warnings bump RLtools to fix the remaining warnings Loading RAPTOR checkpoint from sdcard seems to work on FMU-6C embedding Raptor policy into flash works again also printing checkpoint name when using the embedded policy cleaner handling of the checkpoint name back to reading from file ripping out visual odometry checks cleaner more debug but no success bump rlt bump pre next rebase we can publish the no angvel update because we latch onto it with the scheduled work item anyways this kind of runs on the 6c still bad SIH almost flying saving stale traj setpoint yaw new error. timestamp not the problem anymore bump rlt; SIH works with executor shaping up bumping blob (include tar checkpoint) cleaning up fixing formatting update readme * moving raptor bump compiles and raptor mode appears hovering with RAPTOR seems to work Using Raptor to execute offboard commands works (using multirobot f03825a5795a77c5a095f799eeb8e0b646fe7176 to feed the trajectory_setpoint). Requires more testing simplified rotmat runtime inference frequency multiple arming request response reflects actual readiness adjusting to fit IMU gyro ratemax relaxing control timing warning thresholds for SITL Using mode registration to signal if offboard commands should be forwarded to trajectory_setpoint instead of just hardcoding vehicle_status.nav_state == vehicle_status_s::NAVIGATION_STATE_OFFBOARD adopting new "request_offboard_setpoint" in raptor module replace offboard seems good mc_raptor: overwrite offboard parameter separate raptor config addendum Raptor off by default RAPTOR readme Loading raptor checkpoint from tar works. check if load was successful refactoring: cutting out the pure C interface to allow direct testing of the policy input/output behavior from the file, without fully loading it into memory first adapter not needed anymore ripping out test observation mode (not used in a long time) fixing warnings bump RLtools to fix the remaining warnings Loading RAPTOR checkpoint from sdcard seems to work on FMU-6C embedding Raptor policy into flash works again also printing checkpoint name when using the embedded policy cleaner handling of the checkpoint name back to reading from file ripping out visual odometry checks cleaner more debug but no success bump rlt bump pre next rebase we can publish the no angvel update because we latch onto it with the scheduled work item anyways this kind of runs on the 6c still bad SIH almost flying saving stale traj setpoint yaw new error. timestamp not the problem anymore bump rlt; SIH works with executor shaping up bumping blob (include tar checkpoint) cleaning up fixing formatting update readme updating gitignore * fixing format and declaring submodules as cmake dependencies * adding uORB message documentation * fixing comment alignment * Adding option to restrict mc_raptor to not listen to the trajectory_setpoint (use the position and yaw at activation time as reference instead) * bump RLtools; relax timing thresholds and adding real world readme * smooth traj tracking performance * Measuring trajectory_setpoint timing (providing stats in raptor_status); reverting accidental .gitignore modification * More ideomatic way of setting the path to the policy checkpoint * Reset trajectory_setpoint on raptor mode activation * Adding internal trajectory generation (feeding trajectory_setpoint over Mavlink is too noisy). Quite agile trajectory tracking, good performance * stable flight * Update msg/versioned/RaptorInput.msg Co-authored-by: Hamish Willee <hamishwillee@gmail.com> * adopting message formatting conventions * sort raptor.px4board * Archiving RegisterExtComponentRequestV1.msg * Add message versioning for VehicleStatus v2 and RegisterExtComponentRequest v2 * fixing formatting * making internal reference configurable via command * RAPTOR docs wip * raptor internal reference documentation * Finishing RAPTOR docs first draft * adding logging instructions * Fixing missing command documentation test error * fixing format * adding motor layout warning * raptor minimal subedit - prettier, images etc * Improve intro * Fix up Neural_Networks version * Mentioning "Adaptive" in the RAPTOR documentation's title * Adding clarifications about the internal reference trajectory generator * Removing "foundation policy" wording * Fixing new-line check * Removing redundant (evident through directory hierarchy) raptor_ from filenames * Unifying Neural Network docs (mc_nn_control and mc_raptor) under the "Neural Network" topic * Fix to standard structure * Making the distinction between mc_nn_control and mc_raptor more clear and fixing the comparison table * Removing trajectory_setpoint forwarding flag from external mode registration request and from the vehicle status * Trivial layout and wording fixes * fixing docs error --------- Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
2026-10-03 16:58:54 +08:00 · 2026-01-14 09:47:47 -08:00
parent 95b8328162
commit db2c6b2abe
33 changed files with 2269 additions and 126 deletions
@@ -822,9 +822,11 @@
    - [Camera Integration/Architecture](camera/camera_architecture.md)
    - [Computer Vision](advanced/computer_vision.md)
      - [Motion Capture (VICON, Optitrack, NOKOV)](tutorials/motion-capture.md)
-    - [Neural Networks](advanced/neural_networks.md)
-      - [Neural Network Module Utilities](advanced/nn_module_utilities.md)
-      - [TensorFlow Lite Micro (TFLM)](advanced/tflm.md)
+    - [Neural Networks](neural_networks/index.md)
+      - [MC NN Control Module (Generic)](neural_networks/mc_neural_network_control.md)
+        - [Neural Network Module Utilities](neural_networks/nn_module_utilities.md)
+        - [TensorFlow Lite Micro (TFLM)](neural_networks/tflm.md)
+      - [RAPTOR Adaptive RL NN Module](neural_networks/raptor.md)
    - [Installing driver for Intel RealSense R200](advanced/realsense_intel_driver.md)
    - [Switching State Estimators](advanced/switching_state_estimators.md)
    - [Out-of-Tree Modules](advanced/out_of_tree_modules.md)
@@ -1,119 +1 @@
-# Neural Networks
-
-<Badge type="tip" text="PX4 v1.17" /> <Badge type="warning" text="Experimental" />
-
-::: warning
-This is an experimental module.
-Use at your own risk.
-:::
-
-The Multicopter Neural Network (NN) module ([mc_nn_control](../modules/modules_controller.md#mc-nn-control)) is an example module that allows you to experiment with using a pre-trained neural network on PX4.
-It might be used, for example, to experiment with controllers for non-traditional drone morphologies, computer vision tasks, and so on.
-
-The module integrates a pre-trained neural network based on the [TensorFlow Lite Micro (TFLM)](../advanced/tflm.md) module.
-The module is trained for the [X500 V2](../frames_multicopter/holybro_x500v2_pixhawk6c.md) multicopter frame.
-While the controller is fairly robust, and might work on other platforms, we recommend [Training your own Network](#training-your-own-network) if you use a different vehicle.
-Note that after training the network you will need to update and rebuild PX4.
-
-TLFM is a mature inference library intended for use on embedded devices.
-It has support for several architectures, so there is a high likelihood that you can build it for the board you want to use.
-If not, there are other possible NN frameworks, such as [Eigen](https://eigen.tuxfamily.org/index.php?title=Main_Page) and [Executorch](https://pytorch.org/executorch-overview).
-
-This document explains how you can include the module in your PX4 build, and provides a broad overview of how it works.
-The other documents in the section provide more information about the integration, allowing you to replace the NN with a version trained on different data, or even to replace the TLFM library altogether.
-
-If you are looking for more resources to learn about the module, a website has been created with links to a youtube video and a workshop paper. A full master's thesis will be added later. [A Neural Network Mode for PX4 on Embedded Flight Controllers](https://ntnu-arl.github.io/px4-nns/).
-
-## Neural Network PX4 Firmware
-
-::: warning
-This module requires Ubuntu 24.04 or newer (it is not supported in Ubuntu 22.04).
-:::
-
-The module has been tested on a number of configurations, which can be build locally using the commands:
-
-```sh
-make px4_sitl_neural
-```
-
-```sh
-make px4_fmu-v6c_neural
-```
-
-```sh
-make mro_pixracerpro_neural
-```
-
-You can add the module to other board configurations by modifying their `default.px4board file` configuration to include these lines:
-
-```sh
-CONFIG_LIB_TFLM=y
-CONFIG_MODULES_MC_NN_CONTROL=y
-```
-
-:::tip
-The `mc_nn_control` module takes up roughly 50KB, and many of the `default.px4board file` are already close to filling all the flash on their boards. To make room for the neural control module you can remove the include statements for other modules, such as FW, rover, VTOL and UUV.
-:::
-
-## Example Module Overview
-
-The example module replaces the entire controller structure as well as the control allocator, as shown in the diagram below:
-
-![neural_control](../../assets/advanced/neural_control.png)
-
-In the [controller diagram](../flight_stack/controller_diagrams.md) you can see the [uORB message](../middleware/uorb.md) flow.
-We hook into this flow by subscribing to messages at particular points, using our neural network to calculate outputs, and then publishing them into the next point in the flow.
-We also need to stop the module publishing the topic to be replaced, which is covered in [Neural Network Module: System Integration](nn_module_utilities.md)
-
-### Input
-
-The input can be changed to whatever you want.
-Set up the input you want to use during training and then provide the same input in PX4.
-In the Neural Control module the input is an array of 15 numbers, and consists of these values in this order:
-
- [3] Local position error. (goal position - current position)
- [6] The first 2 rows of a 3 dimensional rotation matrix.
- [3] Linear velocity
- [3] Angular velocity
-
-All the input values are collected from uORB topics and transformed into the correct representation in the `PopulateInputTensor()` function.
-PX4 uses the NED frame representation, while the Aerial Gym Simulator, in which the NN was trained, uses the ENU representation.
-Therefore two rotation matrices are created in the function and all the inputs are transformed from the NED representation to the ENU one.
-
-![ENU-NED](../../assets/advanced/ENU-NED.png)
-
-ENU and NED are just rotation representations, the translational difference is only there so both can be seen in the same figure.
-
-### Output
-
-The output consists of 4 values, the motor forces, one for each motor.
-These are transformed in the `RescaleActions()` function.
-This is done because PX4 expects normalized motor commands while the Aerial Gym Simulator uses physical values.
-So the output from the network needs to be normalized before they can be sent to the motors in PX4.
-
-The commands are published to the [ActuatorMotors](../msg_docs/ActuatorMotors.md) topic.
-The publishing is handled in `PublishOutput(float* command_actions)` function.
-
-:::tip
-If the neural control mode is too aggressive or unresponsive the [MC_NN_THRST_COEF](../advanced_config/parameter_reference.md#MC_NN_THRST_COEF) parameter can be tuned.
-Decrease it for more thrust.
-:::
-
-## Training your own Network
-
-The network is currently trained for the [X500 V2](../frames_multicopter/holybro_x500v2_pixhawk6c.md).
-But the controller is somewhat robust, so it could work directly on other platforms, but performing system identification and training a new network is recommended.
-
-Since the Aerial Gym Simulator is open-source you can download it and train your own networks as long as you have access to an NVIDIA GPU.
-If you want to train a control network optimized for your platform you can follow the instructions in the [Aerial Gym Documentation](https://ntnu-arl.github.io/aerial_gym_simulator/9_sim2real/).
-
-You should do one system identification flight for this and get an approximate inertia matrix for your platform.
-On the `sys-id` flight you need ESC telemetry, you can read more about that in [DSHOT](../peripherals/dshot.md).
-
-Then do the following steps:
-
- Do a hover flight
- Read of the logs what RPM is required for the drone to hover.
- Use the weight of each motor, length of the motor arms, total weight of the platform with battery to calculate an approximate inertia matrix for the platform.
- Insert these values into the Aerial Gym configuration and train your network.
- Convert the network as explained in [TFLM](tflm.md).
+<Redirect to="../neural_networks/mc_neural_network_control" />
@@ -0,0 +1,21 @@
+# Neural Network Control
+
+PX4 supports the following mechanisms for using neural networks for multirotor control:
+
+- [MC Neural Networks Control](../neural_networks/mc_neural_network_control.md)<Badge type="warning" text="Experimental" /> — A generic neural network module that you can modify to use different underlying neural network and training models and compile into the firmware.
+- [RAPTOR: A Neural Network Module for Adaptive Quadrotor Control](../neural_networks/raptor.md)<Badge type="warning" text="Experimental" /> — An adaptive RL NN module that works well with different Quad configurations without additional training.
+
+Generally you will select the former if you wish to experiment with custom neural network architectures and train them using PyTorch or TensorFlow, and the latter if you want to use a pre-trained neural-network controller that works out-of-the-box (without training for your particular platform) or if you train your own policies using [RLtools](https://rl.tools).
+
+Note that both modules are experimental and provided for experimentation.
+The table below provides more detail on the differences.
+
+| Use Case                                                         | [`mc_raptor`](../neural_networks/raptor.md) | [`mc_nn_control`](../neural_networks/mc_neural_network_control.md) |
+| ---------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------ |
+| Pre-trained policy that adapts to any quadrotor without training | ✓ RAPTOR                                    | ✘                                                                  |
+| Train policy in PyTorch/TF                                       | ✘                                           | ✓ TF Lite                                                          |
+| Train policy in RLtools                                          | ✓                                           | ✘                                                                  |
+| Use manual control (remote) with NN policy                       | ✘ GPS/MoCap                                 | ✓ Manual attitude commands                                         |
+| Load policy checkpoints from SD card                             | ✓ Upload via MAVLink FTP                    | ✘ Compiled into firmware                                           |
+| Offboard setpoints                                               | ✓ MAVLink                                   | ✘                                                                  |
+| Internal Trajectory Generator                                    | ✓ (Position, Lissajous)                     | ✘                                                                  |
@@ -0,0 +1,119 @@
+# MC Neural Networks Control
+
+<Badge type="tip" text="PX4 v1.17" /> <Badge type="warning" text="Experimental" />
+
+::: warning
+This is an experimental module.
+Use at your own risk.
+:::
+
+The Multicopter Neural Network (NN) module ([mc_nn_control](../modules/modules_controller.md#mc-nn-control)) is an example module that allows you to experiment with using a pre-trained neural network on PX4.
+It might be used, for example, to experiment with controllers for non-traditional drone morphologies, computer vision tasks, and so on.
+
+The module integrates a pre-trained neural network based on the [TensorFlow Lite Micro (TFLM)](./tflm.md) module.
+The module is trained for the [X500 V2](../frames_multicopter/holybro_x500v2_pixhawk6c.md) multicopter frame.
+While the controller is fairly robust, and might work on other platforms, we recommend [Training your own Network](#training-your-own-network) if you use a different vehicle.
+Note that after training the network you will need to update and rebuild PX4.
+
+TLFM is a mature inference library intended for use on embedded devices.
+It has support for several architectures, so there is a high likelihood that you can build it for the board you want to use.
+If not, there are other possible NN frameworks, such as [Eigen](https://eigen.tuxfamily.org/index.php?title=Main_Page) and [Executorch](https://pytorch.org/executorch-overview).
+
+This document explains how you can include the module in your PX4 build, and provides a broad overview of how it works.
+The other documents in the section provide more information about the integration, allowing you to replace the NN with a version trained on different data, or even to replace the TLFM library altogether.
+
+If you are looking for more resources to learn about the module, a website has been created with links to a youtube video and a workshop paper. A full master's thesis will be added later. [A Neural Network Mode for PX4 on Embedded Flight Controllers](https://ntnu-arl.github.io/px4-nns/).
+
+## Neural Network PX4 Firmware
+
+::: warning
+This module requires Ubuntu 24.04 or newer (it is not supported in Ubuntu 22.04).
+:::
+
+The module has been tested on a number of configurations, which can be build locally using the commands:
+
+```sh
+make px4_sitl_neural
+```
+
+```sh
+make px4_fmu-v6c_neural
+```
+
+```sh
+make mro_pixracerpro_neural
+```
+
+You can add the module to other board configurations by modifying their `default.px4board file` configuration to include these lines:
+
+```sh
+CONFIG_LIB_TFLM=y
+CONFIG_MODULES_MC_NN_CONTROL=y
+```
+
+:::tip
+The `mc_nn_control` module takes up roughly 50KB, and many of the `default.px4board file` are already close to filling all the flash on their boards. To make room for the neural control module you can remove the include statements for other modules, such as FW, rover, VTOL and UUV.
+:::
+
+## Example Module Overview
+
+The example module replaces the entire controller structure as well as the control allocator, as shown in the diagram below:
+
+![neural_control](../../assets/advanced/neural_control.png)
+
+In the [controller diagram](../flight_stack/controller_diagrams.md) you can see the [uORB message](../middleware/uorb.md) flow.
+We hook into this flow by subscribing to messages at particular points, using our neural network to calculate outputs, and then publishing them into the next point in the flow.
+We also need to stop the module publishing the topic to be replaced, which is covered in [Neural Network Module: System Integration](nn_module_utilities.md)
+
+### Input
+
+The input can be changed to whatever you want.
+Set up the input you want to use during training and then provide the same input in PX4.
+In the Neural Control module the input is an array of 15 numbers, and consists of these values in this order:
+
+- [3] Local position error. (goal position - current position)
+- [6] The first 2 rows of a 3 dimensional rotation matrix.
+- [3] Linear velocity
+- [3] Angular velocity
+
+All the input values are collected from uORB topics and transformed into the correct representation in the `PopulateInputTensor()` function.
+PX4 uses the NED frame representation, while the Aerial Gym Simulator, in which the NN was trained, uses the ENU representation.
+Therefore two rotation matrices are created in the function and all the inputs are transformed from the NED representation to the ENU one.
+
+![ENU-NED](../../assets/advanced/ENU-NED.png)
+
+ENU and NED are just rotation representations, the translational difference is only there so both can be seen in the same figure.
+
+### Output
+
+The output consists of 4 values, the motor forces, one for each motor.
+These are transformed in the `RescaleActions()` function.
+This is done because PX4 expects normalized motor commands while the Aerial Gym Simulator uses physical values.
+So the output from the network needs to be normalized before they can be sent to the motors in PX4.
+
+The commands are published to the [ActuatorMotors](../msg_docs/ActuatorMotors.md) topic.
+The publishing is handled in `PublishOutput(float* command_actions)` function.
+
+:::tip
+If the neural control mode is too aggressive or unresponsive the [MC_NN_THRST_COEF](../advanced_config/parameter_reference.md#MC_NN_THRST_COEF) parameter can be tuned.
+Decrease it for more thrust.
+:::
+
+## Training your own Network
+
+The network is currently trained for the [X500 V2](../frames_multicopter/holybro_x500v2_pixhawk6c.md).
+But the controller is somewhat robust, so it could work directly on other platforms, but performing system identification and training a new network is recommended.
+
+Since the Aerial Gym Simulator is open-source you can download it and train your own networks as long as you have access to an NVIDIA GPU.
+If you want to train a control network optimized for your platform you can follow the instructions in the [Aerial Gym Documentation](https://ntnu-arl.github.io/aerial_gym_simulator/9_sim2real/).
+
+You should do one system identification flight for this and get an approximate inertia matrix for your platform.
+On the `sys-id` flight you need ESC telemetry, you can read more about that in [DSHOT](../peripherals/dshot.md).
+
+Then do the following steps:
+
+- Do a hover flight
+- Read of the logs what RPM is required for the drone to hover.
+- Use the weight of each motor, length of the motor arms, total weight of the platform with battery to calculate an approximate inertia matrix for the platform.
+- Insert these values into the Aerial Gym configuration and train your network.
+- Convert the network as explained in [TFLM](tflm.md).
@@ -2,7 +2,7 @@

 The neural control module ([mc_nn_control](../modules/modules_controller.md#mc-nn-control)) implements an end-to-end controller utilizing neural networks.

-The parts of the module directly concerned with generating the code for the trained neural network and integrating it into the module are covered in [TensorFlow Lite Micro (TFLM)](../advanced/tflm.md).
+The parts of the module directly concerned with generating the code for the trained neural network and integrating it into the module are covered in [TensorFlow Lite Micro (TFLM)](./tflm.md).
 This page covers the changes that were made to integrate the module into PX4, both within the module, and in larger system configuration.

 ::: tip
@@ -75,7 +75,7 @@ Which timing library is included and used is based on wether PX4 is built with N

 ## Changing the setpoint

-The module uses the [TrajectorySetpoint](../msg_docs/TrajectorySetpoint.md) message’s position fields to define its target.
+The module uses the [TrajectorySetpoint](../msg_docs/TrajectorySetpoint.md) message's position fields to define its target.
 To follow a trajectory, you can send updated setpoints.
 For an example of how to do this in a PX4 module, see the [mc_nn_testing](https://github.com/SindreMHegre/PX4-Autopilot-public/tree/main/src/modules/mc_nn_testing) module in this fork.
 Note that this is not included in upstream PX4.
@@ -0,0 +1,221 @@
+# RAPTOR: A Neural Network Module for Adaptive Quadrotor Control
+
+<Badge type="tip" text="main (planned for PX4 v1.18)" /> <Badge type="info" text="Multicopter" /> <Badge type="warning" text="Experimental" />
+
+::: warning
+This is an experimental module.
+Use at your own risk.
+:::
+
+RAPTOR is a tiny reinforcement-learning based neural network module for quadrotor control that can be used to control a wide variety of quadrotors without retuning.
+
+This topic provides an overview of the fundamental concepts, and explains how you can use the module in simulation and real hardware.
+
+## Overview
+
+![Visual Abstract](../../assets/advanced/neural_networks/raptor/visual_abstract.jpg)
+
+RAPTOR is an adaptive policy for end-to-end quadrotor control.
+It is motivated by the human ability to adapt learned behaviours to similar situations.
+For example, while humans may initially require many hours of driving experience to be able to smoothly control the car and blend into traffic, when faced with a new vehicle they do not need to re-learn how to drive — they only need to experience a few rough braking/acceleration/steering responses to adjust their previously learned behavior.
+
+Reinforcement Learning (RL) is a machine learning technique that uses trial and error to learn decision making/control behaviors, which is similar to the way that humans learn to drive.
+RL is interesting for controlling robots (and particularly UAVs) because it overcomes some fundamental limitations of classic, modular control architectures (information loss at module boundaries, requirement for expert tuning, etc).
+RL has been very successful in [high-performance quadrotor flight](https://doi.org/10.1038/s41586-023-06419-4), but previous designs have not been particularly adaptable to new frames and vehicle types.
+
+RAPTOR fills this gap and demonstrates a single, tiny neural-network control policy that can control a wide variety of quadrotors (tested on real quadrotors from 32 g to 2.4 kg).
+
+For more details please refer to this video:
+
+<lite-youtube videoid="hVzdWRFTX3k" title="RAPTOR: A Foundation Policy for Quadrotor Control"/>
+
+The method we developed for training the RAPTOR policy is called Meta-Imitation Learning:
+
+![Diagram showing the Method Overview](../../assets/advanced/neural_networks/raptor/method.jpg)
+
+You can torture test the RAPTOR policy in your browser at [https://raptor.rl.tools](https://raptor.rl.tools) or in the embedded app here:
+
+<iframe src="https://rl-tools.github.io/raptor.rl.tools?raptor=false" width="100%" height="1000" style="border: none;"></iframe>
+
+For more information please refer to the paper at [https://arxiv.org/abs/2509.11481](https://arxiv.org/abs/2509.11481).
+
+## Structure
+
+The RAPTOR control policy is an end-to-end policy that takes position, orientation, linear velocity and angular velocity as inputs and outputs motor commands (`actuator_motors`).
+To integrate it into PX4 we use the external mode registration facilities in PX4 (which also works well for internal modes as demonstrated in `mc_nn_control`).
+Because of this architecture the `mc_raptor` module is completely decoupled from all other PX4 logic.
+
+By default, the RAPTOR module expects setpoints via `trajectory_setpoint` messages.
+If no `trajectory_setpoint` messages are received or if no `trajectory_setpoint` is received within 200 ms, the current position and orientation (with zero velocity) is used as the setpoint.
+Since feeding setpoints reliably via telemetry is still a challenge, we also implement a simple option to generate internal reference trajectories (controlled through the `MC_RAPTOR_INTREF` parameter) for demonstration and benchmarking purposes.
+
+## Features
+
+- Tiny neural network (just 2084 parameters) => minimal CPU usage
+- Easily maintainable
+  - Simple CMake setup
+  - Self-contained (no interference with other modules)
+  - Single, simple and well-maintained dependency (RLtools)
+- Loading neural network parameters from SD card
+  - Minimal flash usage (for possible inclusion into default build configurations)
+  - Easy development: Train new neural network and just upload it via MAVLink FTP without requiring to re-flash the firmware
+- Tested on 10+ different real platforms (including flexible frames, brushed motors)
+- Actively developed and maintained
+
+## Usage
+
+### SITL
+
+Build PX4 SITL with Raptor, disable QGC requirement, and adjust the `IMU_GYRO_RATEMAX` to match the simulation IMU rate
+
+```sh
+make px4_sitl_raptor gz_x500
+param set NAV_DLL_ACT 0
+param set COM_DISARM_LAND -1 # When taking off in offboard the landing detector can cause mid-air disarms
+param set IMU_GYRO_RATEMAX 250 # Just for SITL. Tested with IMU_GYRO_RATEMAX=400 on real FCUs
+param set MC_RAPTOR_ENABLE 1 # Enable the mc_raptor module
+param save
+```
+
+Upload the RAPTOR checkpoint to the "SD card": Separate terminal
+
+```bash
+mavproxy.py --master udp:127.0.0.1:14540
+ftp mkdir /raptor # for the real FMU use: /fs/microsd/raptor
+ftp put src/modules/mc_raptor/blob/policy.tar /raptor/policy.tar
+```
+
+Restart (<kbd>Ctrl+C</kbd>)
+
+```sh
+make px4_sitl_raptor gz_x500
+commander takeoff
+commander status
+```
+
+Note the external mode ID of `RAPTOR` in the status report
+
+```sh
+commander mode ext{RAPTOR_MODE_ID}
+```
+
+#### Internal Reference Trajectory Generation
+
+In our experience, feeding the `trajectory_setpoint` via MAVLink (even via WiFi telemetry) is unreliable.
+But we do not want to constrain this module to only platforms that have a companion board.
+
+For this reason we have integrated a simple internal reference trajectory generator for testing and benchmarking purposes.
+It supports position (constant position and yaw setpoint) as well as configurable [Lissajous trajectories](https://en.wikipedia.org/wiki/Lissajous_curve).
+
+The Lissajous generator can, for example, generate smooth figure-eight trajectories that contain interesting accelerations for benchmarking and testing purposes.
+Please refer to the embedded configurator later in this section to explore the Lissajous parameters and view the resulting trajectories.
+
+To use the internal reference generator, select the mode: `0`: Off/activation position tracking, `1`: Lissajous
+
+```sh
+param set MC_RAPTOR_INTREF 1
+```
+
+Restart (ctrl+c)
+
+```sh
+commander takeoff
+commander mode ext{RAPTOR_MODE_ID}
+mc_raptor intref lissajous 0.5 1 0 2 1 1 10 3
+```
+
+The trajectory is relative to the position and yaw of the vehicle at the point where the RAPTOR mode is activated (or the position and yaw where the parameters are changed if it is already activated).
+
+You can adjust the parameters of the trajectory with the following tool.
+Make sure to copy the generated CLI string at the end:
+
+<iframe src="https://rl-tools.github.io/mc-raptor-trajectory-tool" width="100%" height="1700" style="border: none;"></iframe>
+
+### Real-World
+
+#### Setup
+
+The `mc_raptor` module has been mostly tested with the Holybro X500 V2 but it should also work out-of-the-box with other platforms (see the [Other Platforms](#other-platforms) section).
+
+```sh
+make px4_fmu-v6c_raptor upload
+```
+
+We recommend initially testing the RAPTOR mode using a dead man's switch.
+For this we configure the mode selection to be connected to a push button or a switch with a spring that automatically switches back.
+In the default position we configure e.g. `Stabilized Mode` and in the pressed configuration we select `External Mode 1` (since the name of the external mode is only transmitted at runtime).
+This allows to take off manually and then just trigger the RAPTOR mode for a split-second to see how it behaves.
+In our experiments it has been exceptionally stable (zero crashes) but we still think progressively activating it for longer is the safest way to build confidence.
+
+::: warning
+Make sure that your platform uses the standard PX4 quadrotor motor layout:
+
+1: front-right, 2: back-left, 3: front-left, 4: back-right
+:::
+
+##### Other Platforms
+
+To enable the `mc_raptor` module in other platforms, just add `CONFIG_MODULES_MC_RAPTOR=y` and `CONFIG_LIB_RL_TOOLS=y`
+
+```diff
+++ b/boards/px4/fmu-v6c/raptor.px4board
+@@ -35,2 +35,3 @@
+ CONFIG_DRIVERS_UAVCAN=y
+CONFIG_LIB_RL_TOOLS=y
+ CONFIG_MODULES_AIRSPEED_SELECTOR=y
+@@ -64,2 +65,3 @@
+ CONFIG_MODULES_MC_POS_CONTROL=y
+CONFIG_MODULES_MC_RAPTOR=y
+ CONFIG_MODULES_MC_RATE_CONTROL=y
+```
+
+#### Results
+
+Even though there were moderate winds (~ 5 m/s) during the test, we found good figure-eight tracking performance at velocities up to 12 m/s:
+
+![Lissajous](../../assets/advanced/neural_networks/raptor/results_figure_eight.svg)
+
+We also tested the linear velocity in a straight line and found that the RAPTOR policy can reliably fly at > 17 m/s (the wind direction was orthogonal to the line):
+
+![Linear Oscillation](../../assets/advanced/neural_networks/raptor/results_line.svg)
+
+### Troubleshooting
+
+#### Logging
+
+Use this logging configuration to log all relevant topics at maximum rate:
+
+```sh
+cat > logger_topics.txt << EOF
+raptor_status 0
+raptor_input 0
+trajectory_setpoint 0
+vehicle_local_position 0
+vehicle_angular_velocity 0
+vehicle_attitude 0
+vehicle_status 0
+actuator_motors 0
+EOF
+```
+
+Use mavproxy FTP to upload it:
+
+```sh
+mavproxy.py
+```
+
+##### Real
+
+```sh
+ftp mkdir /fs/microsd/etc
+ftp mkdir /fs/microsd/etc/logging
+ftp put logger_topics.txt /fs/microsd/etc/logging/logger_topics.txt
+```
+
+##### SITL
+
+```sh
+ftp mkdir etc
+ftp mkdir logging
+ftp put logger_topics.txt etc/logging/logger_topics.txt
+```
@@ -1,6 +1,6 @@
 # TensorFlow Lite Micro (TFLM)

-The PX4 [Multicopter Neural Network](../advanced/neural_networks.md) module ([mc_nn_control](../modules/modules_controller.md#mc-nn-control)) integrates a neural network that uses the [TensorFlow Lite Micro (TFLM)](https://github.com/tensorflow/tflite-micro) inference library.
+The PX4 [MC Neural Networks Control](../neural_networks/mc_neural_network_control.md) module ([mc_nn_control](../modules/modules_controller.md#mc-nn-control)) integrates a neural network that uses the [TensorFlow Lite Micro (TFLM)](https://github.com/tensorflow/tflite-micro) inference library.

 This is a mature inference library intended for use on embedded devices, and is hence a suitable choice for PX4.

@@ -68,7 +68,7 @@ The `_input_tensor` is also defined, it is fetched from `_control_interpreter->i

 The `_input_tensor` is filled in the `PopulateInputTensor()` function.
 `_input_tensor` works by accessing the `->data.f` member array and fill in the required inputs for your network.
-The inputs used in the control network is covered in [Neural Networks](../advanced/neural_networks.md).
+The inputs used in the control network is covered in [MC Neural Networks Control](../neural_networks/mc_neural_network_control.md).

 ### Outputs