Compare commits

..

7 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] fe7bf8472b Add static_assert to verify LOG_FILEPATH_SCANF_WIDTH is synchronized
- Add compile-time check that LOG_FILEPATH_SIZE == 256
- Ensures LOG_FILEPATH_SCANF_WIDTH stays synchronized if buffer size changes
- Provides clear error message if someone modifies LOG_FILEPATH_SIZE

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:50:04 +00:00
copilot-swe-agent[bot] ca336fb299 Use LOG_FILEPATH_SCANF_WIDTH macro for format specifier
- Define LOG_FILEPATH_SCANF_WIDTH macro for the sscanf width specifier
- Replace hardcoded '255' with the named macro in both sscanf calls
- Improves maintainability and makes the relationship between buffer size and width explicit
- Addresses final code review feedback about magic numbers

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:49:01 +00:00
copilot-swe-agent[bot] 5cdc783b05 Remove duplicate constant and use header LOG_FILEPATH_SIZE
- Remove duplicate LOG_FILEPATH_SIZE macro from cpp file
- Use MavlinkLogHandler::LOG_FILEPATH_SIZE consistently throughout
- Update local filepath buffer to use the same size as LogEntry.filepath
- Addresses code review feedback about duplication and consistency

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:47:25 +00:00
copilot-swe-agent[bot] 5037a96376 Define LOG_FILEPATH_SIZE constant for maintainability
- Add LOG_FILEPATH_SIZE constant (256) in both header and cpp
- Update LogEntry.filepath to use the constant
- Improve comments explaining width specifier relationship
- Addresses code review feedback about magic numbers

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:45:09 +00:00
copilot-swe-agent[bot] 8e0cabaeb7 Add static_assert and use consistent %255s width specifier
- Add static_assert to ensure PX4_MAX_FILEPATH >= 256 at compile time
- Use %255s consistently for both sscanf calls to prevent overflow
- Add explanatory comments for the width specifier choice
- Addresses code review feedback about potential overflow on NuttX

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:43:00 +00:00
copilot-swe-agent[bot] 338595edd1 Fix stack buffer overflow in mavlink_log_handler sscanf calls
- Increase LogEntry.filepath buffer from 60 to 256 bytes
- Add width specifiers to sscanf calls (%255s and %1023s) to prevent buffer overflow
- Prevents remote DoS vulnerability when parsing logdata.txt with excessively long filenames

Co-authored-by: dakejahl <37091262+dakejahl@users.noreply.github.com>
2025-12-15 21:37:29 +00:00
copilot-swe-agent[bot] f219c9f3b9 Initial plan 2025-12-15 21:33:02 +00:00
4238 changed files with 70650 additions and 136990 deletions
-73
View File
@@ -105,79 +105,6 @@ Checks: '*,
-readability-redundant-declaration,
-readability-static-accessed-through-instance,
-readability-static-definition-in-anonymous-namespace,
-altera-struct-pack-align,
-bugprone-easily-swappable-parameters,
-concurrency-mt-unsafe,
-cppcoreguidelines-avoid-const-or-ref-data-members,
-cppcoreguidelines-macro-usage,
-cppcoreguidelines-non-private-member-variables-in-classes,
-hicpp-uppercase-literal-suffix,
-llvm-qualified-auto,
-misc-non-private-member-variables-in-classes,
-misc-use-anonymous-namespace,
-modernize-concat-nested-namespaces,
-readability-const-return-type,
-readability-identifier-length,
-readability-isolate-declaration,
-readability-qualified-auto,
-readability-redundant-access-specifiers,
-cppcoreguidelines-avoid-do-while,
-misc-include-cleaner,
-misc-const-correctness,
-llvm-else-after-return,
-readability-function-cognitive-complexity,
-cppcoreguidelines-init-variables,
-bugprone-reserved-identifier,
-cert-dcl37-c,
-cert-dcl51-cpp,
-modernize-use-nodiscard,
-misc-confusable-identifiers,
-cert-err33-c,
-readability-redundant-inline-specifier,
-readability-uppercase-literal-suffix,
-bugprone-narrowing-conversions,
-cppcoreguidelines-narrowing-conversions,
-bugprone-switch-missing-default-case,
-cppcoreguidelines-avoid-goto,
-hicpp-avoid-goto,
-bugprone-branch-clone,
-performance-enum-size,
-readability-avoid-nested-conditional-operator,
-cppcoreguidelines-prefer-member-initializer,
-cppcoreguidelines-explicit-virtual-functions,
-readability-convert-member-functions-to-static,
-readability-make-member-function-const,
-bugprone-implicit-widening-of-multiplication-result,
-bugprone-multi-level-implicit-pointer-conversion,
-bugprone-signed-char-misuse,
-cppcoreguidelines-avoid-non-const-global-variables,
-cppcoreguidelines-use-default-member-init,
-hicpp-multiway-paths-covered,
-hicpp-named-parameter,
-misc-header-include-cycle,
-misc-no-recursion,
-performance-no-int-to-ptr,
-readability-avoid-return-with-void-value,
-readability-avoid-unconditional-preprocessor-if,
-readability-delete-null-pointer,
-readability-redundant-casting,
-readability-redundant-member-init,
-readability-reference-to-constructed-temporary,
-readability-simplify-boolean-expr,
-cert-msc32-c,
-cert-msc33-c,
-cert-msc51-cpp,
-cert-str34-c,
-cppcoreguidelines-macro-to-enum,
-modernize-macro-to-enum,
-abseil-string-find-str-contains,
-bugprone-suspicious-include,
-clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling,
-clang-analyzer-optin.core.EnumCastOutOfRange,
-modernize-type-traits,
-misc-definitions-in-headers,
-bugprone-casting-through-void,
-readability-redundant-string-init,
'
WarningsAsErrors: '*'
CheckOptions:
-28
View File
@@ -1,28 +0,0 @@
---
name: commit
description: Create a conventional commit for PX4 changes
disable-model-invocation: true
argument-hint: "[optional: description of changes]"
allowed-tools: Bash, Read, Glob, Grep
---
# PX4 Conventional Commit
Create a git commit: `type(scope): description`
**NEVER add Co-Authored-By lines. No Claude attribution in commits.**
Follow [CONTRIBUTING.md](../../CONTRIBUTING.md) for full project conventions.
## Steps
1. **Read [CONTRIBUTING.md](../../CONTRIBUTING.md)** for commit message format, types, scopes, and conventions.
2. Check branch (`git branch --show-current`). If on `main`, create a feature branch. Use `<username>/<description>` format where `<username>` comes from `gh api user --jq .login`. If unavailable, just use `<description>`.
3. Run `git status` and `git diff --staged`. If nothing staged, ask what to stage.
4. Follow the commit message convention from CONTRIBUTING.md: pick the correct **type** and **scope**, write a concise imperative description.
5. Body (if needed): explain **why**, not what.
6. Run `make format` or `./Tools/astyle/fix_code_style.sh <file>` on changed C/C++ files before committing.
7. Check if GPG signing is available: `git config --get user.signingkey`. If set, use `git commit -S -s`. Otherwise, use `git commit -s`.
8. Stage and commit. No `Co-Authored-By`.
If the user provided arguments, use them as context: $ARGUMENTS
-24
View File
@@ -1,24 +0,0 @@
---
name: pr
description: Create a pull request with conventional commit title and description
disable-model-invocation: true
argument-hint: "[optional: target branch or description]"
allowed-tools: Bash, Read, Glob, Grep
---
# PX4 Pull Request
**No Claude attribution anywhere (no Co-Authored-By, no "Generated with Claude").**
Follow [CONTRIBUTING.md](../../CONTRIBUTING.md) for full project conventions.
## Steps
1. Check branch. If on `main`, create a feature branch. Use `<username>/<description>` format where `<username>` comes from `gh api user --jq .login`. If unavailable, just use `<description>`.
2. Gather context: `git status`, `git log --oneline main..HEAD`, `git diff main...HEAD --stat`, check if remote tracking branch exists.
3. PR **title**: `type(scope): description` — under 72 chars, describes the overall change across all commits. This becomes the squash-merge commit message.
4. PR **body**: brief summary + bullet points for key changes. No filler.
5. Push with `-u` if needed, then `gh pr create`. Default base is `main` unless user says otherwise.
6. Return the PR URL.
If the user provided arguments, use them as context: $ARGUMENTS
-73
View File
@@ -1,73 +0,0 @@
---
name: rebase-onto-main
description: Rebase a branch onto main, handling squash-merged parent branches cleanly
argument-hint: "[optional: branch name, defaults to current branch]"
allowed-tools: Bash, Read, Glob, Grep, Agent
---
# Rebase Branch onto Main
Rebase the current (or specified) branch onto `main`, correctly handling the case where the branch was built on top of another branch that has since been squash-merged into `main`.
## Background
When a parent branch is squash-merged, its individual commits become a single new commit on `main` with a different hash. A normal `git rebase main` will try to replay the parent's original commits, causing messy conflicts. The fix is to **cherry-pick only the commits unique to this branch** onto a fresh branch from `main`.
## Steps
1. **Identify the branch.** Use `$ARGUMENTS` if provided, otherwise use the current branch.
2. **Fetch and update main:**
```
git fetch origin main:main
```
3. **Find the merge base** between the branch and `main`:
```
git merge-base <branch> main
```
4. **List all commits** on the branch since the merge base:
```
git log --oneline <merge-base>..<branch>
```
5. **Identify which commits are unique to this branch** vs. inherited from a parent branch. Look for:
- Squash-merged commits on `main` that correspond to a group of commits at the bottom of the branch's history (check PR titles, commit message keywords).
- The boundary commit: the first commit that belongs to *this* branch's work, not the parent's.
- If ALL commits are unique (no parent branch), just do a normal `git rebase main` and skip the rest.
6. **Create a fresh branch from `main`:**
```
git checkout -b <branch>-rebase main
```
7. **Cherry-pick only the unique commits** (oldest first):
```
git cherry-pick <first-unique-commit>^..<branch>
```
The `A^..B` range means "from the parent of A through B inclusive."
8. **Handle conflicts** if any arise during cherry-pick. Resolve and `git cherry-pick --continue`.
9. **Replace the old branch:**
```
git branch -m <branch> <branch>-old
git branch -m <branch>-rebase <branch>
```
10. **Verify** the result:
```
git log --oneline main..<branch>
```
Confirm only the expected commits are present.
11. **Ask the user** before force-pushing. When approved:
```
git push origin <branch> --force-with-lease
```
12. **Clean up** the old branch:
```
git branch -D <branch>-old
```
+78 -31
View File
@@ -3,45 +3,92 @@ description: Create a report to help us improve
title: "[Bug] "
labels: ["bug-report"]
body:
- type: markdown
attributes:
value: |
**Tips for a great bug report:**
- Describe what went wrong and what you expected
- Include a flight log link from [logs.px4.io](http://logs.px4.io/) if possible
- Mention your PX4 version, flight controller, and vehicle type if relevant
- type: textarea
attributes:
label: Describe the bug
description: A clear description of the bug and what you expected to happen.
placeholder: |
What happened and what did you expect instead?
Steps to reproduce (if applicable):
1.
2.
3.
description: A clear and concise description of the bug.
validations:
required: true
- type: textarea
attributes:
label: Flight Log / Additional Information
label: To Reproduce
description: |
**Flight log** (highly recommended for flight-related issues):
- Upload to [PX4 Flight Review](http://logs.px4.io/) and paste the link
**Additional details** (if relevant):
- PX4 version (output of `ver all` in MAVLink Shell)
- Flight controller model
- Vehicle type (multicopter, fixed-wing, VTOL, etc.)
- Screenshots or media
placeholder: |
Flight log link:
Version:
Hardware:
Steps to reproduce the behavior.
1. Drone switched on '...'
2. Uploaded mission '....' (attach QGC mission file)
3. Took off '....'
4. See error
validations:
required: false
- type: textarea
attributes:
label: Expected behavior
description: A clear and concise description of what you expected to happen.
validations:
required: false
- type: textarea
attributes:
label: Screenshot / Media
description: Add screenshot / media if you have them
- type: textarea
attributes:
label: Flight Log
description: |
*Always* provide a link to the flight log file:
- Download the flight log file from the vehicle ([tutorial](https://docs.px4.io/main/en/getting_started/flight_reporting.html)).
- Upload the log to the [PX4 Flight Review](http://logs.px4.io/)
- Share the link to the log (Copy and paste the URL of the log)
placeholder: |
# PASTE HERE THE LINK TO THE LOG
validations:
required: false
- type: markdown
attributes:
value: |
## Setup
- type: textarea
attributes:
label: Software Version
description: |
Which version of PX4 are you using?
placeholder: |
# If you don't know the version, paste the output of `ver all` in the MAVLink Shell of QGC
validations:
required: false
- type: input
attributes:
label: Flight controller
description: Specify your flight controller model (what type is it, where was it bought from, ...).
validations:
required: false
- type: dropdown
attributes:
label: Vehicle type
options:
- Multicopter
- Helicopter
- Fixed Wing
- Hybrid VTOL
- Airship/Balloon
- Rover
- Boat
- Submarine
- Other
- type: textarea
attributes:
label: How are the different components wired up (including port information)
description: Details about how all is wired.
- type: textarea
attributes:
label: Additional context
description: Add any other context about the problem here.
+1 -1
View File
@@ -1,4 +1,4 @@
blank_issues_enabled: true
blank_issues_enabled: false
contact_links:
- name: Support Question
url: https://docs.px4.io/main/en/contribute/support.html#forums-and-chat
@@ -1,50 +0,0 @@
---
applyTo: "docs/en/**"
---
# Review Guidelines docs/en Tree
## File System & Structure
- **Naming:** Use `lowercase_with_underscores` for all filenames. No spaces.
- **Hierarchy:** Markdown files must reside exactly in a first-level category folder.
- Valid: `docs/en/category/file.md`
- Invalid: `docs/en/category/subcategory/file.md`
- **Text Files:** Any `.txt` or `.text` files must start with an underscore (e.g., `_notes.txt`).
- **Assets:** All images/non-docs must be in `/docs/assets/`. Deep nesting is permitted here.
- **Formats:** Prefer **SVG** for diagrams and **PNG** for screenshots. Flag JPG files.
## Markdown & Style
- **Headings:** Use Title Case ("First Letter Capitalisation").
- The Page Title must be the only H1 (`#`). All others must be `##` or lower.
- Do not apply bold or italic styling inside a heading.
- **Formatting:**
- **Bold:** Only for UI elements (buttons, menu items).
- **Inline Code:** Use backticks for file paths, parameters, and CLI commands (e.g., `prettier`).
Backticks are optional for hyperlinked CLI commands and tool names.
- **Italics (Emphasis):** Use for application names (e.g., *QGroundControl*).
Emphasis is optional for hyperlinked applications.
- **Structure:** End every line at the end of a sentence (Semantic Line Breaks).
## Linking & Navigation
- **Standard Links:** Use standard inline syntax: `[link text](../category/filename.md)`.
Note relative link.
- **Table Links:** To keep tables readable, use reference-style links.
- Definition: `[Link Name]: https://example.com` (placed below the table).
- Usage: `[Link Name]` within the table cell.
- **Images:** All image links must include a descriptive, accessible alt-text in the brackets: `![Detailed description of the image content](../../assets/path/to/image.png)`.
Note that all images should be relative references to images stored in the assets folder, which should be two folders below the any markdown file (as they are stored in a "category" subfolder)
- **Standard Links:** Use standard inline syntax: `[link text](../category/filename.md)`. Note the use of relative links.
- **Table Links:** To keep tables easier to edit, prefer reference-style links.
- Definition: `[Link Name]: https://example.com` (placed below the table).
- Usage: `[Link Name]` within the table cell.
- **Images:** All image links must include a descriptive, accessible alt-text: `![Detailed description of the image content](../../assets/path/to/image.png)`.
- **Note:** All images must be relative references to the `/docs/assets/` folder. Since documents are nested in a category folder, this is usually two levels up (`../../assets/`).
## Quality Control
- **Prettier Check:** Ensure Prettier rules have been applied. If there is evidence of inconsistent indentation or spacing, request the author run `npx prettier --write .` before merging.
- **Language:** Enforce **UK English** spelling and grammar.
+28 -57
View File
@@ -2,37 +2,6 @@
# - If you want to keep the tests running in GitHub Actions you need to uncomment the "runs-on: ubuntu-latest" lines
# and comment the "runs-on: [runs-on,runner=..." lines.
# - If you would like to duplicate this setup try setting up "RunsOn" on your own AWS account try https://runs-on.com
#
# ===================================================================================
# RELEASE UPLOAD LOGIC
# ===================================================================================
# This workflow handles building firmware and uploading to S3 + GitHub Releases.
#
# S3 Bucket Structure (s3://px4-travis/Firmware/):
# - master/ <- Latest main branch build (for QGC compatibility)
# - stable/ <- Latest stable release, controlled by 'stable' branch
# - beta/ <- Latest pre-release, controlled by 'beta' branch
# - vX.Y.Z/ <- Archived stable release
# - vX.Y.Z-beta1/ <- Archived pre-release
#
# Trigger Behavior:
# - Tag v1.16.1 -> Upload to: v1.16.1/ only (versioned archive)
# - Tag v1.17.0-beta1 -> Upload to: v1.17.0-beta1/ only (versioned archive)
# - Branch main -> Upload to: master/ (for QGC compatibility)
# - Branch stable -> Upload to: stable/ (QGC stable firmware)
# - Branch beta -> Upload to: beta/ (QGC beta firmware)
# - Branch release/** -> Build only, no S3 upload (CI validation)
# - Pull requests -> Build only, no S3 upload (CI validation)
#
# GitHub Releases:
# - All version tags create a draft GitHub Release
# - Pre-releases (alpha/beta/rc suffixes) are automatically marked as such
#
# IMPORTANT: Version tags do NOT upload to stable/ or beta/. Only the
# corresponding branch pushes control those directories. This prevents
# pre-release tags from accidentally overwriting stable firmware (#26340)
# and avoids race conditions between tag and branch builds.
# ===================================================================================
name: Build all targets
@@ -60,7 +29,6 @@ concurrency:
permissions:
contents: write
actions: read
packages: read
jobs:
group_targets:
@@ -126,9 +94,6 @@ jobs:
fail-fast: false
container:
image: ${{ matrix.container }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v4
@@ -194,13 +159,6 @@ jobs:
path: ~/.ccache
key: ${{ steps.cc_restore.outputs.cache-primary-key }}
# ===========================================================================
# ARTIFACT UPLOAD JOB
# ===========================================================================
# Uploads build artifacts to S3 and creates GitHub Releases.
# Runs for version tags (v*), main, stable, and beta branch pushes.
# See header comments for full upload logic documentation.
# ===========================================================================
artifacts:
name: Upload Artifacts
# runs-on: ubuntu-latest
@@ -219,31 +177,31 @@ jobs:
- name: Choose Upload Location
id: upload-location
run: |
# Determine upload location based on branch or tag with the following considerations:
# Destination: AWS S3 bucket px4-travis in folder Firmware/
# - If branch is main -> upload to master/
# - Older versions of QGC are hardocded to look for master/
# - If branch is stable or beta -> upload to stable/ or beta/
# - If a tag vX.Y.Z -> upload to vX.Y.Z/
# - Also update stable/ to point to the same version
#. - Older versions of QGC are hardocded to look for stable/
# - If a pull request -> do not upload
set -euo pipefail
ref="${GITHUB_REF}"
branch=${{ needs.group_targets.outputs.branchname }}
location="$branch"
is_prerelease="false"
# Main branch uploads to "master" for QGC backward compatibility
if [[ "$branch" == "main" ]]; then
location="master"
fi
# Version tags: upload to versioned directory (e.g., v1.16.1/)
if [[ "$ref" == refs/tags/v[0-9]* ]]; then
tag="${ref#refs/tags/}"
location="$tag"
# Pre-release tags contain -alpha, -beta, or -rc suffix
if [[ "$tag" =~ -(alpha|beta|rc) ]]; then
is_prerelease="true"
fi
fi
echo "uploadlocation=$location" >> $GITHUB_OUTPUT
echo "is_prerelease=$is_prerelease" >> $GITHUB_OUTPUT
- name: Uploading Artifacts to S3 [${{ steps.upload-location.outputs.uploadlocation }}]
uses: jakejarvis/s3-sync-action@master
@@ -257,15 +215,28 @@ jobs:
SOURCE_DIR: artifacts/
DEST_DIR: Firmware/${{ steps.upload-location.outputs.uploadlocation }}/
# Create a draft GitHub Release for all version tags
# Pre-releases are automatically marked as such
# if we are uploading artifacts to a versioned folder
# we should also update the stable folder in the s3 bucket
- name: Uploading Artifacts to S3 [stable]
uses: jakejarvis/s3-sync-action@master
if: startsWith(github.ref, 'refs/tags/v')
with:
args: --acl public-read
env:
AWS_S3_BUCKET: 'px4-travis'
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: 'us-west-1'
SOURCE_DIR: artifacts/
DEST_DIR: Firmware/stable/
# if build is a release triggered by a versioned tag then create a github release
# and upload the build artifacts. A draft release is created so that the release
# can be reviewed before publishing
- name: Upload Artifacts to GitHub Release
uses: softprops/action-gh-release@v2
if: startsWith(github.ref, 'refs/tags/v')
with:
draft: true
prerelease: ${{ steps.upload-location.outputs.is_prerelease == 'true' }}
files: |
artifacts/*.px4
artifacts/*.deb
files: artifacts/*.px4
name: ${{ steps.upload-location.outputs.uploadlocation }}
+8 -9
View File
@@ -19,10 +19,6 @@ concurrency:
jobs:
build:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
fail-fast: false
matrix:
@@ -39,17 +35,20 @@ jobs:
"px4_sitl_allyes",
"module_documentation",
]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Building [${{ matrix.check }}]
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
make ${{ matrix.check }}
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
options: -v ${{ github.workspace }}:/workspace
run: |
cd /workspace
git config --global --add safe.directory /workspace
make ${{ matrix.check }}
- name: Uploading Coverage to Codecov.io
if: contains(matrix.check, 'coverage')
+11 -50
View File
@@ -1,4 +1,4 @@
name: Static Analysis
name: Clang Tidy
on:
push:
@@ -11,59 +11,20 @@ on:
- '**'
paths-ignore:
- 'docs/**'
permissions:
contents: read
jobs:
clang_tidy:
name: Clang-Tidy
runs-on: [runs-on, runner=16cpu-linux-x64, "run-id=${{ github.run_id }}", "extras=s3-cache"]
container:
image: px4io/px4-dev:v1.17.0-beta1
build:
runs-on: ubuntu-latest
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- name: Restore Compiler Cache
id: cc_restore
uses: actions/cache/restore@v4
- name: Testing (clang-tidy)
uses: addnab/docker-run-action@v3
with:
path: ~/.ccache
key: ccache-clang-tidy-${{ github.head_ref || github.ref_name }}
restore-keys: |
ccache-clang-tidy-${{ github.head_ref || github.ref_name }}-
ccache-clang-tidy-main-
ccache-clang-tidy-
- name: Configure Compiler Cache
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 120M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Run Clang-Tidy Analysis
run: make -j16 clang-tidy
- name: Compiler Cache Stats
if: always()
run: ccache -s
- name: Save Compiler Cache
if: always()
uses: actions/cache/save@v4
with:
path: ~/.ccache
key: ${{ steps.cc_restore.outputs.cache-primary-key }}
image: px4io/px4-dev-clang:2021-09-08
options: -v ${{ github.workspace }}:/workspace
run: |
cd /workspace
git config --global --add safe.directory /workspace
make clang-tidy
-148
View File
@@ -1,148 +0,0 @@
name: Commit Quality
on:
pull_request:
types: [opened, edited, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: write
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
IS_FORK: ${{ github.event.pull_request.head.repo.full_name != github.repository }}
jobs:
pr-title:
name: PR Title
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
- name: Check PR title
id: check
run: |
python3 Tools/ci/check_pr_title.py "${{ github.event.pull_request.title }}" --markdown-file comment.md && rc=0 || rc=$?
echo "exit_code=$rc" >> "$GITHUB_OUTPUT"
- name: Post or clear comment
if: env.IS_FORK == 'false'
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ "${{ steps.check.outputs.exit_code }}" != "0" ]; then
python3 Tools/ci/pr_comment.py --marker pr-title --pr "$PR_NUMBER" --result fail < comment.md
else
python3 Tools/ci/pr_comment.py --marker pr-title --pr "$PR_NUMBER" --result pass
fi
- name: Result
if: steps.check.outputs.exit_code != '0'
run: |
echo "::error::PR title does not follow conventional commits format. See the PR comment for details."
exit 1
commit-messages:
name: Commit Messages
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
- name: Check commit messages
id: check
env:
GH_TOKEN: ${{ github.token }}
run: |
gh api \
"repos/${{ github.repository }}/pulls/${PR_NUMBER}/commits?per_page=100" \
| python3 Tools/ci/check_commit_messages.py --markdown-file comment.md && rc=0 || rc=$?
echo "exit_code=$rc" >> "$GITHUB_OUTPUT"
# Check for warnings (non-empty markdown on exit 0)
if [ "$rc" -eq 0 ] && [ -s comment.md ]; then
echo "has_warnings=true" >> "$GITHUB_OUTPUT"
else
echo "has_warnings=false" >> "$GITHUB_OUTPUT"
fi
- name: Post or clear comment
if: env.IS_FORK == 'false'
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ "${{ steps.check.outputs.exit_code }}" != "0" ]; then
python3 Tools/ci/pr_comment.py --marker commit-msgs --pr "$PR_NUMBER" --result fail < comment.md
elif [ "${{ steps.check.outputs.has_warnings }}" == "true" ]; then
python3 Tools/ci/pr_comment.py --marker commit-msgs --pr "$PR_NUMBER" --result warn < comment.md
else
python3 Tools/ci/pr_comment.py --marker commit-msgs --pr "$PR_NUMBER" --result pass
fi
- name: Result
if: steps.check.outputs.exit_code != '0'
run: |
echo "::error::Commit message errors found. See the PR comment for details."
exit 1
pr-body:
name: PR Description
runs-on: ubuntu-latest
steps:
- name: Checkout
if: env.IS_FORK == 'false'
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
- name: Check PR body
id: check
env:
PR_BODY: ${{ github.event.pull_request.body }}
run: |
message=""
if [ -z "$PR_BODY" ]; then
message="PR description is empty. Please add a summary of the changes."
echo "::warning::PR description is empty."
else
cleaned=$(echo "$PR_BODY" | sed 's/<!--.*-->//g' | tr -d '[:space:]')
if [ -z "$cleaned" ]; then
message="PR description contains only template comments. Please fill in the details."
echo "::warning::PR description contains only template comments."
fi
fi
echo "message=$message" >> "$GITHUB_OUTPUT"
- name: Post or clear comment
if: env.IS_FORK == 'false'
env:
GH_TOKEN: ${{ github.token }}
run: |
if [ -n "${{ steps.check.outputs.message }}" ]; then
printf '%s\n' \
"## PR Description (advisory)" \
"" \
"This is **not blocking**, but your PR description appears to be empty or incomplete." \
"" \
"${{ steps.check.outputs.message }}" \
"" \
"A good PR description helps reviewers understand what changed and why." \
"" \
"---" \
"*This comment will be automatically removed once the issue is resolved.*" \
| python3 Tools/ci/pr_comment.py --marker pr-body --pr "$PR_NUMBER" --result warn
else
python3 Tools/ci/pr_comment.py --marker pr-body --pr "$PR_NUMBER" --result pass
fi
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
shell: cmake -P {0}
run: |
string(TIMESTAMP current_date "%Y-%m-%d-%H;%M;%S" UTC)
file(APPEND "$ENV{GITHUB_OUTPUT}" "timestamp=${current_date}\n")
message("::set-output name=timestamp::${current_date}")
- name: ccache cache files
uses: actions/cache@v4
with:
+2 -2
View File
@@ -130,8 +130,8 @@ jobs:
load: false
push: ${{ startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry) }}
provenance: false
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
cache-from: type=gha,version=1,scope=${{ matrix.arch }}
cache-to: type=gha,version=1,mode=max,scope=${{ matrix.arch }}
deploy:
name: Deploy To Registry
-416
View File
@@ -1,416 +0,0 @@
# Docs - Orchestrator
#
# Trigger paths:
# push (main, release/**) → metadata-regen → build-site → deploy-aws
# pull_request → detect-changes → pr-metadata-regen → link-check → build-site (if docs/source changed)
# workflow_dispatch → metadata-regen → build-site → deploy-aws
#
# Container jobs (pr-metadata-regen, metadata-regen) run in px4-dev image and
# require safe.directory + fetch-depth: 0 for git operations.
name: Docs - Orchestrator
on:
push:
branches:
- "main"
- "release/**"
paths:
- "docs/**"
- "src/**"
- "msg/**"
- "ROMFS/**"
- "Tools/module_config/**"
- ".github/workflows/docs-orchestrator.yml"
pull_request:
paths:
- "docs/**"
- ".github/workflows/docs-orchestrator.yml"
workflow_dispatch:
concurrency:
group: docs-orchestrator-${{ github.ref }}
cancel-in-progress: true
jobs:
# =============================================================================
# Detect Changes (PR only)
# =============================================================================
detect-changes:
name: "T1: Detect Changes"
if: github.event_name == 'pull_request'
permissions:
contents: read
runs-on: ubuntu-latest
outputs:
source_changed: ${{ steps.changes.outputs.source }}
docs_changed: ${{ steps.changes.outputs.docs }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: changes
with:
filters: |
source:
- 'src/**'
- 'msg/**'
- 'ROMFS/**'
- 'Tools/module_config/**'
docs:
- 'docs/**'
# =============================================================================
# PR Metadata Regen (conditional - only when PR touches source files)
# =============================================================================
pr-metadata-regen:
name: "T2: PR Metadata"
needs: [detect-changes]
if: github.event_name == 'pull_request' && needs.detect-changes.outputs.source_changed == 'true'
permissions:
contents: read
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
container:
image: px4io/px4-dev:v1.17.0-beta1
steps:
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Cache Restore - ccache
id: cache-ccache
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
restore-keys: |
ccache-docs-metadata-
- name: Setup ccache
run: |
mkdir -p ~/.ccache
echo "max_size = 1G" > ~/.ccache/ccache.conf
- name: Build px4_sitl_default
run: |
make px4_sitl_default
env:
CCACHE_DIR: ~/.ccache
- name: Cache Save - ccache
uses: actions/cache/save@v4
if: always()
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
- name: Generate and sync metadata
run: Tools/ci/metadata_sync.sh --generate --sync parameters airframes modules msg_docs failsafe_web
env:
CCACHE_DIR: ~/.ccache
- name: Upload metadata artifact
uses: actions/upload-artifact@v4
with:
name: pr-metadata
path: docs/
retention-days: 1
# =============================================================================
# Push Metadata Regen (main/release branches)
# =============================================================================
metadata-regen:
name: "T2: Metadata Sync"
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
permissions:
contents: write
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
container:
image: px4io/px4-dev:v1.17.0-beta1
steps:
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
token: ${{ secrets.PX4BUILTBOT_PERSONAL_ACCESS_TOKEN }}
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Cache Restore - ccache
id: cache-ccache
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
restore-keys: |
ccache-docs-metadata-
- name: Setup ccache
run: |
mkdir -p ~/.ccache
echo "max_size = 1G" > ~/.ccache/ccache.conf
- name: Build px4_sitl_default
run: |
make px4_sitl_default
env:
CCACHE_DIR: ~/.ccache
- name: Cache Save - ccache
uses: actions/cache/save@v4
if: always()
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
- name: Generate and sync metadata
run: Tools/ci/metadata_sync.sh --generate --sync parameters airframes modules msg_docs failsafe_web
env:
CCACHE_DIR: ~/.ccache
- name: Install Node.js and Yarn
run: |
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs
corepack enable
- name: Format markdown with Prettier
run: |
cd docs
yarn install --frozen-lockfile
yarn prettier --write "en/**/*.md"
- name: Commit and push changes
run: |
git config --global user.name "${{ secrets.PX4BUILDBOT_USER }}"
git config --global user.email "${{ secrets.PX4BUILDBOT_EMAIL }}"
git add docs/
if git diff --staged --quiet; then
echo "No changes to commit"
else
git commit -m "docs: auto-sync metadata [skip ci]
Co-Authored-By: PX4 BuildBot <${{ secrets.PX4BUILDBOT_EMAIL }}>"
git push
fi
# =============================================================================
# Link Check
# =============================================================================
link-check:
name: "T2: Link Check"
needs: [detect-changes, pr-metadata-regen]
if: always() && (github.event_name == 'pull_request')
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Download metadata artifact
if: needs.pr-metadata-regen.result == 'success'
uses: actions/download-artifact@v4
with:
name: pr-metadata
path: docs/
- name: Get changed doc files
id: changed-files
uses: tj-actions/changed-files@v46.0.5
with:
json: true
write_output_files: true
output_dir: ./logs
base_sha: ${{ github.event.pull_request.base.sha }}
sha: ${{ github.event.pull_request.head.sha }}
files: |
docs/en/**/*.md
- name: Save changed files list
run: |
mv ./logs/all_changed_files.json ./logs/prFiles.json
echo "Changed files:"
cat ./logs/prFiles.json
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20
- name: Run filtered link checker (changed files)
run: |
npm -g install markdown_link_checker_sc@0.0.138
if [ "$(jq length ./logs/prFiles.json)" -gt 0 ]; then
markdown_link_checker_sc \
-r "$GITHUB_WORKSPACE" \
-d docs \
-e en \
-f ./logs/prFiles.json \
-i assets \
-u docs.px4.io/main/ \
> ./logs/filtered-link-check-results.md || true
fi
if [ ! -s ./logs/filtered-link-check-results.md ]; then
echo "No broken links found in changed files." > ./logs/filtered-link-check-results.md
fi
cat ./logs/filtered-link-check-results.md
- name: Run full link checker
run: |
markdown_link_checker_sc \
-r "$GITHUB_WORKSPACE" \
-d docs \
-e en \
-i assets \
-u docs.px4.io/main/ \
> ./logs/link-check-results.md || true
cat ./logs/link-check-results.md
- name: Post PR comment with link check results
if: github.event.pull_request.head.repo.full_name == github.repository
uses: marocchino/sticky-pull-request-comment@v2
with:
header: flaws
path: ./logs/filtered-link-check-results.md
- name: Upload link check results
uses: actions/upload-artifact@v4
with:
name: link-check-results
path: logs/
retention-days: 7
# =============================================================================
# Build Site
# =============================================================================
build-site:
name: "T3: Build Site"
needs: [detect-changes, pr-metadata-regen, metadata-regen, link-check]
if: >-
always() &&
(needs.metadata-regen.result == 'success' || needs.metadata-regen.result == 'skipped') &&
(needs.link-check.result == 'success' || needs.link-check.result == 'skipped') &&
(github.event_name != 'pull_request' || needs.detect-changes.outputs.docs_changed == 'true' || needs.detect-changes.outputs.source_changed == 'true')
permissions:
contents: read
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
outputs:
branchname: ${{ steps.set-branch.outputs.branchname }}
releaseversion: ${{ steps.set-version.outputs.releaseversion }}
steps:
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Download metadata artifact (PR)
if: github.event_name == 'pull_request' && needs.pr-metadata-regen.result == 'success'
uses: actions/download-artifact@v4
with:
name: pr-metadata
path: docs/
- id: set-branch
run: echo "branchname=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
- id: set-version
run: |
branch="${{ steps.set-branch.outputs.branchname }}"
if [[ "$branch" == "main" ]]; then
version="main"
elif [[ "$branch" =~ ^release/ ]]; then
version="v${branch#release/}"
elif [[ "${{ github.event_name }}" == "pull_request" ]]; then
version="main"
else
echo "::error::Unsupported branch for docs deploy: $branch (expected main or release/*)"
exit 1
fi
echo "releaseversion=$version" >> $GITHUB_OUTPUT
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: ./docs/yarn.lock
- name: Install dependencies
run: yarn install --frozen-lockfile --cwd ./docs
- name: Build with VitePress
working-directory: ./docs
env:
BRANCH_NAME: ${{ steps.set-version.outputs.releaseversion }}
run: |
npm run docs:build_ubuntu
touch .vitepress/dist/.nojekyll
npm run docs:sitemap
- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: px4_docs_build
path: docs/.vitepress/dist/
retention-days: 1
# =============================================================================
# Deploy to AWS (push + workflow_dispatch)
# =============================================================================
deploy-aws:
name: "T4: Deploy"
needs: [metadata-regen, build-site]
if: >-
always() &&
needs.metadata-regen.result == 'success' &&
needs.build-site.result == 'success' &&
(github.event_name == 'push' || github.event_name == 'workflow_dispatch')
permissions:
id-token: write
runs-on: ubuntu-latest
steps:
- name: Download Artifact
uses: actions/download-artifact@v4
with:
name: px4_docs_build
path: ~/_book
- name: Configure AWS from OIDC
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: us-west-2
- name: Sanity check AWS credentials
run: aws sts get-caller-identity
- name: Upload HTML with short cache
run: |
aws s3 sync ~/_book/ s3://px4-docs/${{ needs.build-site.outputs.releaseversion }}/ \
--delete \
--exclude "*" --include "*.html" \
--cache-control "public, max-age=60"
- name: Upload assets with long cache
run: |
aws s3 sync ~/_book/ s3://px4-docs/${{ needs.build-site.outputs.releaseversion }}/ \
--delete \
--exclude "*.html" \
--cache-control "public, max-age=86400, immutable"
+3 -3
View File
@@ -34,13 +34,13 @@ jobs:
upload_sources: false
upload_translations: false
download_translations: true
commit_message: 'docs(i18n): PX4 guide translations (Crowdin) - ${{ matrix.lc }}'
commit_message: New Crowdin translations - ${{ matrix.lc }}
localization_branch_name: l10n_crowdin_docs_translations_${{ matrix.lc }}
crowdin_branch_name: main
create_pull_request: true
pull_request_base_branch_name: 'main'
pull_request_title: 'docs(i18n): PX4 guide translations (Crowdin) - ${{ matrix.lc }}'
pull_request_body: 'docs(i18n): PX4 guide Crowdin translations by [Crowdin GH Action](https://github.com/crowdin/github-action) for ${{ matrix.lc }}'
pull_request_title: New PX4 guide translations (Crowdin) - ${{ matrix.lc }}
pull_request_body: 'New PX4 guide Crowdin translations by [Crowdin GH Action](https://github.com/crowdin/github-action) for ${{ matrix.lc }}'
pull_request_labels: 'Documentation 📑'
pull_request_reviewers: hamishwillee
download_language: ${{ matrix.lc }}
+116
View File
@@ -0,0 +1,116 @@
name: Docs - Deploy PX4 User Guide to AWS
on:
push:
branches:
- "main"
- "release/**"
paths:
- "docs/en/**"
- "docs/zh/**"
- "docs/uk/**"
- "docs/ko/**"
pull_request:
paths:
- "docs/en/**"
- "docs/zh/**"
- "docs/uk/**"
- "docs/ko/**"
workflow_dispatch:
permissions:
contents: read
actions: read
id-token: write # for AWS OIDC
concurrency:
group: docs-deploy
cancel-in-progress: false
jobs:
build:
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
outputs:
branchname: ${{ steps.set-branch.outputs.branchname }}
releaseversion: ${{ steps.set-version.outputs.releaseversion }}
steps:
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
- id: set-branch
run: echo "branchname=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
- id: set-version
run: |
branch="${{ steps.set-branch.outputs.branchname }}"
if [[ "$branch" == "main" ]]; then
version="main"
else
version="v${branch#release/}"
fi
echo "releaseversion=$version" >> $GITHUB_OUTPUT
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: ./docs/yarn.lock
- name: Install dependencies
run: yarn install --frozen-lockfile --cwd ./docs
- name: Build with VitePress
working-directory: ./docs
env:
BRANCH_NAME: ${{ steps.set-version.outputs.releaseversion }}
run: |
npm run docs:build_ubuntu
touch .vitepress/dist/.nojekyll
npm run docs:sitemap
- name: Upload artifact
if: ${{ github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.merged) || github.event_name == 'workflow_dispatch' }}
uses: actions/upload-artifact@v4
with:
name: px4_docs_build
path: docs/.vitepress/dist/
retention-days: 1
deploy:
if: ${{ github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.merged) || github.event_name == 'workflow_dispatch' }}
needs: build
runs-on: ubuntu-latest
steps:
- name: Download Artifact
uses: actions/download-artifact@v4
with:
name: px4_docs_build
path: ~/_book
- name: Configure AWS from OIDC
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: us-west-2
- name: Sanity check AWS credentials
run: aws sts get-caller-identity
- name: Upload HTML with short cache
run: |
aws s3 sync ~/_book/ s3://px4-docs/${{ needs.build.outputs.releaseversion }}/ \
--delete \
--exclude "*" --include "*.html" \
--cache-control "public, max-age=60"
- name: Upload assets with long cache
run: |
aws s3 sync ~/_book/ s3://px4-docs/${{ needs.build.outputs.releaseversion }}/ \
--delete \
--exclude "*.html" \
--cache-control "public, max-age=86400, immutable"
+85
View File
@@ -0,0 +1,85 @@
name: Docs - Check for flaws in PX4 Guide Source
# So far:
# Modifications of translations files
# Broken internal links
on:
pull_request_target:
types: [opened, edited, synchronize]
paths:
- 'docs/en/**'
jobs:
check_flaws:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Install Node.js
uses: actions/setup-node@v3
with:
node-version: '18'
- name: Create logs directory
run: |
mkdir logs
- name: Get changed english doc files
id: get_changed_markdown_english
uses: tj-actions/changed-files@v46.0.5
with:
json: true
base_sha: "${{ github.event.pull_request.base.sha }}"
sha: "${{ github.event.pull_request.head.sha }}"
# Below are used to output files to a directory. May use in flaw checker.
# write_output_files: true
# output_dir: "./logs"
files: |
docs/en/**/*.md
- name: Save JSON file containing files to link check
run: |
echo "$ALL_CHANGED_FILES"
# echo "$ALL_CHANGED_FILES" > ./logs/prFiles.json
echo "$ALL_CHANGED_FILES" | sed 's/\\//g' | jq '.' > ./logs/prFiles.json
env:
ALL_CHANGED_FILES: ${{ steps.get_changed_markdown_english.outputs.all_changed_files }}
- name: Run link checker
id: link-check
run: |
npm -g install markdown_link_checker_sc@0.0.138
markdown_link_checker_sc \
-r "$GITHUB_WORKSPACE" \
-d docs \
-e en \
-f ./logs/prFiles.json \
-i assets \
-u docs.px4.io/main/ \
> ./logs/errorsFilteredByPrPages.md
mkdir -p ./pr
cp ./logs/errorsFilteredByPrPages.md ./pr/errorsFilteredByPrPages.md
- name: Read errorsFilteredByPrPages.md file
id: read-errors-by-page
uses: juliangruber/read-file-action@v1
with:
path: ./logs/errorsFilteredByPrPages.md
- name: Echo Errors by Page
run: echo "$ERRORS"
env:
ERRORS: ${{ steps.read-errors-by-page.outputs.content }}
- name: Save PR number
run: echo "$PR_NUMBER" > ./pr/pr_number
env:
PR_NUMBER: ${{ github.event.number }}
- uses: actions/upload-artifact@v4
with:
name: pr_number
path: pr/
+111
View File
@@ -0,0 +1,111 @@
name: Docs - Comment Workflow
on:
workflow_run:
workflows: ["Docs - Check for flaws in PX4 Guide Source"]
types:
- completed
jobs:
comment:
permissions:
pull-requests: write # for marocchino/sticky-pull-request-comment
name: Comments
runs-on: ubuntu-latest
steps:
- name: 'Download PR artifact'
uses: actions/github-script@v6
with:
script: |
let allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id,
});
let matchArtifact = allArtifacts.data.artifacts.filter((artifact) => {
return artifact.name == "pr_number"
})[0];
let download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
let fs = require('fs');
fs.writeFileSync(`${process.env.GITHUB_WORKSPACE}/pr_number.zip`, Buffer.from(download.data));
- name: 'Unzip artifact'
run: unzip pr_number.zip
# Doesn't work across workflows
#- name: Get artifacts from flaw checker workflow
# uses: actions/download-artifact@v3
# with:
# name: logs_and_errors
# #path: ./logs
- name: Read errorsFilteredByPrPages.md file
id: read-errors-by-page
uses: juliangruber/read-file-action@v1
with:
path: ./errorsFilteredByPrPages.md
- name: Read PR number
id: read-error-pr-number
uses: juliangruber/read-file-action@v1
with:
path: ./pr_number
- name: File detail info
run: |
echo "$ERRORS"
echo "$PRNUM"
env:
ERRORS: ${{ steps.read-errors-by-page.outputs.content }}
PRNUM: ${{ steps.read-error-pr-number.outputs.content }}
- name: Create or update comment
id: comment_to_pr
uses: marocchino/sticky-pull-request-comment@v2
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
recreate: true
number: ${{ steps.read-error-pr-number.outputs.content }}
header: flaws
message: ${{ steps.read-errors-by-page.outputs.content || 'No flaws found' }}
#- name: Dump GitHub context
# env:
# GITHUB_CONTEXT: ${{ toJSON(github) }}
# run: echo "$GITHUB_CONTEXT"
# Would like to do this, but it doesn't work (for me).
# Moving to time-based, or triggering on workflow
#- name: Wait for artifacts upload to succeed
# uses: lewagon/wait-on-check-action@v1.3.1
# with:
# ref: ${{ github.ref }}
# check-name: 'Archive production artifacts'
# repo-token: ${{ secrets.GITHUB_TOKEN }}
# wait-interval: 80
# Not needed for now - trying to trigger off the workflow
#- name: Sleep for 80 seconds
# run: sleep 80s
# shell: bash
#- name: Find Comment
# uses: peter-evans/find-comment@v2
# id: fc
# with:
# issue-number: ${{ steps.read-error-pr-number.outputs.content }}
# comment-author: 'github-actions[bot]'
# body-includes: Flaws (may be none)
#- name: Create or update comment
# uses: peter-evans/create-or-update-comment@v3
# with:
# comment-id: ${{ steps.fc.outputs.comment-id }}
# issue-number: ${{ steps.read-error-pr-number.outputs.content }}
# body: |
# Flaws (may be none)
# ${{ steps.read-errors-by-page.outputs.content }}
# edit-mode: replace
@@ -15,21 +15,21 @@ concurrency:
jobs:
unit_tests:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: main test
- name: main test
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
options: -v ${{ github.workspace }}:/workspace
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
cd /workspace
git config --global --add safe.directory /workspace
make tests TESTFILTER=EKF
- name: Check if there is a functional change
run: git diff --exit-code
working-directory: src/modules/ekf2/test/change_indication
- name: Check if there is a functional change
run: git diff --exit-code
working-directory: src/modules/ekf2/test/change_indication
@@ -8,47 +8,40 @@ on:
jobs:
unit_tests:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
env:
GIT_COMMITTER_EMAIL: bot@px4.io
GIT_COMMITTER_NAME: PX4BuildBot
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: main test
- name: main test
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
options: -v ${{ github.workspace }}:/workspace
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
cd /workspace
git config --global --add safe.directory /workspace
make tests TESTFILTER=EKF
- name: Check if there exists diff and save result in variable
id: diff-check
working-directory: src/modules/ekf2/test/change_indication
run: |
if git diff --quiet; then
echo "CHANGE_INDICATED=false" >> $GITHUB_OUTPUT
else
echo "CHANGE_INDICATED=true" >> $GITHUB_OUTPUT
fi
- name: Check if there exists diff and save result in variable
id: diff-check
run: echo "CHANGE_INDICATED=$(git diff --exit-code --output=/dev/null || echo $?)" >> $GITHUB_OUTPUT
working-directory: src/modules/ekf2/test/change_indication
- name: auto-commit any changes to change indication
if: steps.diff-check.outputs.CHANGE_INDICATED == 'true'
uses: stefanzweifel/git-auto-commit-action@v4
with:
file_pattern: 'src/modules/ekf2/test/change_indication/*.csv'
commit_user_name: ${{ env.GIT_COMMITTER_NAME }}
commit_user_email: ${{ env.GIT_COMMITTER_EMAIL }}
commit_message: |
[AUTO COMMIT] update change indication
- name: auto-commit any changes to change indication
uses: stefanzweifel/git-auto-commit-action@v4
with:
file_pattern: 'src/modules/ekf2/test/change_indication/*.csv'
commit_user_name: ${GIT_COMMITTER_NAME}
commit_user_email: ${GIT_COMMITTER_EMAIL}
commit_message: |
'[AUTO COMMIT] update change indication'
See .github/workflows/ekf_update_change_indicator.yml for more details
See .github/workflopws/ekf_update_change_indicator.yml for more details
- name: if there is a functional change, fail check
if: steps.diff-check.outputs.CHANGE_INDICATED == 'true'
run: exit 1
- name: if there is a functional change, fail check
if: ${{ steps.diff-check.outputs.CHANGE_INDICATED }}
run: exit 1
+16 -18
View File
@@ -19,27 +19,25 @@ concurrency:
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
config:
- {vehicle: "iris", mission: "MC_mission_box"}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build SITL and Run Tests (inside old ROS container)
- name: Build SITL and Run Tests
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev-ros-melodic:2021-09-08
options: -v ${{ github.workspace }}:/workspace
run: |
docker run --rm \
-v "${GITHUB_WORKSPACE}:/workspace" \
-w /workspace \
px4io/px4-dev-ros-melodic:2021-09-08 \
bash -c '
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh \
mavros_posix_test_mission.test \
mission:=MC_mission_box \
vehicle:=iris
'
cd /workspace
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh mavros_posix_test_mission.test mission:=${{matrix.config.mission}} vehicle:=${{matrix.config.vehicle}}
+18 -17
View File
@@ -19,26 +19,27 @@ concurrency:
jobs:
build:
runs-on: ubuntu-latest
env:
ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
strategy:
fail-fast: false
matrix:
config:
- {test_file: "mavros_posix_tests_offboard_posctl.test", vehicle: "iris"}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build SITL and Run Tests (inside old ROS container)
- name: Build PX4 and Run Tests
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev-ros-melodic:2021-09-08
options: -v ${{ github.workspace }}:/workspace
run: |
docker run --rm \
-v "${GITHUB_WORKSPACE}:/workspace" \
-w /workspace \
px4io/px4-dev-ros-melodic:2021-09-08 \
bash -c '
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh \
mavros_posix_tests_offboard_posctl.test \
vehicle:=iris
'
cd /workspace
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh ${{matrix.config.test_file}} vehicle:=${{matrix.config.vehicle}}
+14 -15
View File
@@ -19,28 +19,27 @@ concurrency:
jobs:
build:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
matrix:
config:
- px4_fmu-v5_default
config: [
px4_fmu-v5_default,
]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build PX4 and Run Test [${{ matrix.config }}]
- name: Build PX4 and Run Test [${{ matrix.config }}]
uses: addnab/docker-run-action@v3
with:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
options: -v ${{ github.workspace }}:/workspace
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
export PX4_EXTRA_NUTTX_CONFIG='CONFIG_NSH_LOGIN_PASSWORD="test";CONFIG_NSH_CONSOLE_LOGIN=y'
cd /workspace
git config --global --add safe.directory /workspace
export PX4_EXTRA_NUTTX_CONFIG="CONFIG_NSH_LOGIN_PASSWORD=\"test\";CONFIG_NSH_CONSOLE_LOGIN=y"
echo "PX4_EXTRA_NUTTX_CONFIG: $PX4_EXTRA_NUTTX_CONFIG"
make ${{ matrix.config }} nuttx_context
echo "Check that the config option is set"
grep CONFIG_NSH_LOGIN_PASSWORD build/${{ matrix.config }}/NuttX/nuttx/.config
+1 -12
View File
@@ -89,18 +89,7 @@ jobs:
. /opt/ros/galactic/setup.bash
mkdir -p /opt/px4_ws/src
cd /opt/px4_ws/src
BRANCH="${GITHUB_HEAD_REF:-$GITHUB_REF_NAME}"
REPO_URL="https://github.com/Auterion/px4-ros2-interface-lib.git"
if git ls-remote --heads "$REPO_URL" "$BRANCH" | grep -q "$BRANCH"; then
echo "Cloning px4-ros2-interface-lib with matching branch: $BRANCH"
git clone --recursive --branch "$BRANCH" "$REPO_URL"
else
echo "Branch '$BRANCH' not found in px4-ros2-interface-lib, using default (main)"
git clone --recursive "$REPO_URL"
fi
# Ignore python packages due to compilation issue (can be enabled when updating ROS)
touch px4-ros2-interface-lib/px4_ros2_py/COLCON_IGNORE || true
touch px4-ros2-interface-lib/examples/python/COLCON_IGNORE || true
git clone --recursive https://github.com/Auterion/px4-ros2-interface-lib.git
cd ..
# Copy messages to ROS workspace
"${PX4_DIR}/Tools/copy_to_ros_ws.sh" "$(pwd)"
+2 -4
View File
@@ -33,10 +33,8 @@ jobs:
matrix:
config:
- {model: "iris", latitude: "59.617693", longitude: "-151.145316", altitude: "48", build_type: "RelWithDebInfo" } # Alaska
# VTOL/tailsitter disabled: persistent flaky CI failures (timeouts, erratic
# transitions). Re-enable once the test infrastructure is stabilized.
# - {model: "tailsitter" , latitude: "29.660316", longitude: "-82.316658", altitude: "30", build_type: "RelWithDebInfo" } # Florida
# - {model: "standard_vtol", latitude: "47.397742", longitude: "8.545594", altitude: "488", build_type: "Coverage" } # Zurich
- {model: "tailsitter" , latitude: "29.660316", longitude: "-82.316658", altitude: "30", build_type: "RelWithDebInfo" } # Florida
- {model: "standard_vtol", latitude: "47.397742", longitude: "8.545594", altitude: "488", build_type: "Coverage" } # Zurich
steps:
- uses: actions/checkout@v4
+4 -4
View File
@@ -15,7 +15,7 @@ jobs:
stale-issue-label: 'stale'
stale-pr-label: 'stale'
remove-stale-when-updated: true
stale-issue-message: ''
close-issue-message: 'This issue has been automatically closed due to 120 days of inactivity. If you believe this is still relevant, please feel free to reopen it or create a new issue.'
stale-pr-message: ''
close-pr-message: 'This pull request has been automatically closed due to 120 days of inactivity. If you would like to continue, please feel free to reopen it or submit a new PR.'
stale-issue-message: 'This issue has been marked as stale due to 90 days of inactivity. If no further activity occurs, it will be automatically closed in 30 days. Please leave a comment, add a reaction, make an update, or remove the stale label if youd like to keep it open.'
close-issue-message: 'This issue has been closed due to prolonged inactivity after being marked as stale. If you believe this was closed in error or the topic is still relevant, please feel free to reopen it or create a new issue.'
stale-pr-message: 'This PR was identified as stale and it will be closed in 30 days unless any activity is detected.'
close-pr-message: 'This pull request has been closed after being marked as stale with no further activity. Thank you for the time and effort you put into this contribution. If youd like to continue the discussion or update the work, please feel free to reopen it or submit a new PR.'
@@ -1,43 +0,0 @@
name: Sync release branch to px4-ros2-interface-lib
on:
create:
workflow_dispatch:
inputs:
branch:
description: 'Release branch name (e.g. release/1.18)'
required: true
type: string
permissions: {}
jobs:
notify-interface-lib:
if: >-
github.repository == 'PX4/PX4-Autopilot' &&
(
(github.event_name == 'create' && github.ref_type == 'branch' && startsWith(github.ref_name, 'release/')) ||
github.event_name == 'workflow_dispatch'
)
runs-on: ubuntu-latest
steps:
- name: Determine branch name
id: params
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
BRANCH="${{ inputs.branch }}"
else
BRANCH="${{ github.ref_name }}"
fi
echo "branch=$BRANCH" >> "$GITHUB_OUTPUT"
echo "Dispatching for branch: $BRANCH"
- name: Dispatch release branch creation
run: |
BRANCH="${{ steps.params.outputs.branch }}"
curl -s -f -X POST \
-H "Authorization: token ${{ secrets.PX4BUILTBOT_PERSONAL_ACCESS_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/Auterion/px4-ros2-interface-lib/dispatches \
-d "{\"event_type\":\"px4_release_branch\",\"client_payload\":{\"branch\":\"$BRANCH\"}}"
echo "Dispatched px4_release_branch event for $BRANCH"
-3
View File
@@ -109,6 +109,3 @@ src/systemcmds/topic_listener/listener_generated.cpp
# colcon
log/
keys/
# metadata
_emscripten_sdk/
-12
View File
@@ -103,15 +103,3 @@
[submodule "src/drivers/ins/sbgecom/sbgECom"]
path = src/drivers/ins/sbgecom/sbgECom
url = https://github.com/PX4/sbgECom.git
[submodule "src/modules/mc_raptor/blob"]
path = src/modules/mc_raptor/blob
url = https://github.com/rl-tools/px4-blob
[submodule "src/lib/rl_tools/rl_tools"]
path = src/lib/rl_tools/rl_tools
url = https://github.com/rl-tools/rl-tools.git
[submodule "libmodal-json"]
path = boards/modalai/voxl2/src/lib/mpa/libmodal-json
url = https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-json.git
[submodule "libmodal-pipe"]
path = boards/modalai/voxl2/src/lib/mpa/libmodal-pipe
url = https://gitlab.com/voxl-public/voxl-sdk/core-libs/libmodal-pipe.git
-15
View File
@@ -6,16 +6,6 @@ CONFIG:
buildType: RelWithDebInfo
settings:
CONFIG: px4_sitl_default
px4_sitl_raptor:
short: px4_sitl_raptor
buildType: RelWithDebInfo
settings:
CONFIG: px4_sitl_raptor
px4_sitl_raptor_debug:
short: px4_sitl_raptor_debug
buildType: Debug
settings:
CONFIG: px4_sitl_raptor
px4_sitl_spacecraft:
short: px4_sitl_spacecraft
buildType: RelWithDebInfo
@@ -336,11 +326,6 @@ CONFIG:
buildType: MinSizeRel
settings:
CONFIG: cuav_x25-evo_default
cuav_x25-super_default:
short: cuav_x25-super
buildType: MinSizeRel
settings:
CONFIG: cuav_x25-super_default
cubepilot_cubeorange_test:
short: cubepilot_cubeorange
buildType: MinSizeRel
-27
View File
@@ -1,27 +0,0 @@
cff-version: 1.2.0
title: "PX4 Autopilot"
message: "If you use PX4 in your research, please cite it using this metadata."
type: software
authors:
- family-names: Meier
given-names: Lorenz
- name: "The PX4 Contributors"
repository-code: "https://github.com/PX4/PX4-Autopilot"
url: "https://px4.io"
abstract: >-
PX4 is an open-source autopilot stack for drones and
unmanned vehicles. It supports multirotors, fixed-wing,
VTOL, rovers, and many more platforms. PX4 runs on both
RTOS and POSIX-compatible operating systems.
keywords:
- autopilot
- drone
- uav
- flight-controller
- robotics
- ros2
license: BSD-3-Clause
identifiers:
- type: doi
value: "10.5281/zenodo.595432"
description: "Zenodo concept DOI (resolves to latest version)"
+1 -1
View File
@@ -267,7 +267,7 @@ endif()
set(package-contact "px4users@googlegroups.com")
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_C_STANDARD 11)
set(CMAKE_C_STANDARD_REQUIRED ON)
+21 -147
View File
@@ -1,170 +1,44 @@
# Contributing to PX4-Autopilot
# Contributing to PX4 Firmware
We follow the [GitHub flow](https://guides.github.com/introduction/flow/) development model.
We follow the [Github flow](https://guides.github.com/introduction/flow/) development model.
## Fork the project, then clone your repo
### Fork the project, then clone your repo
First [fork and clone](https://help.github.com/articles/fork-a-repo) the project.
First [fork and clone](https://help.github.com/articles/fork-a-repo) the project project.
## Create a feature branch
### Create a feature branch
Always branch off `main` for new features.
*Always* branch off main for new features.
```
git checkout -b mydescriptivebranchname
```
## Edit and build the code
### Edit and build the code
The [developer guide](https://docs.px4.io/main/en/development/development.html) explains how to set up the development environment on Mac OS, Linux or Windows.
The [developer guide](https://docs.px4.io/main/en/development/development.html) explains how to set up the development environment on Mac OS, Linux or Windows. Please take note of our [coding style](https://docs.px4.io/main/en/contribute/code.html) when editing files.
### Coding standards
### Commit your changes
All C/C++ code must follow the [PX4 coding style](https://docs.px4.io/main/en/contribute/code.html). Formatting is enforced by [astyle](http://astyle.sourceforge.net/) in CI (`make check_format`). Code quality checks run via [clang-tidy](https://clang.llvm.org/extra/clang-tidy/). Pull requests that fail either check will not be merged.
Always write descriptive commit messages and add a fixes or relates note to them with an [issue number](https://github.com/px4/Firmware/issues) (Github will link these then conveniently)
Python code is checked with [mypy](https://mypy-lang.org/) and [flake8](https://flake8.pycqa.org/).
## Commit message convention
PX4 uses [conventional commits](https://www.conventionalcommits.org/) for all commit messages and PR titles.
### Format
**Example:**
```
type(scope): short description of the change
Change how the attitude controller works
- Fixes rate feed forward
- Allows a local body rate override
Fixes issue #123
```
| Part | Rule |
|------|------|
| **type** | Category of change (see types table below) |
| **scope** | The module, driver, board, or area of PX4 affected |
| **`!`** (optional) | Append before `:` to mark a breaking change |
| **description** | What the change does, at least 5 characters, written in imperative form |
### Test your changes
### Types
Since we care about safety, we will regularly ask you for test results. Best is to do a test flight (or bench test where it applies) and upload the logfile from it (on the microSD card in the logs directory) to Google Drive or Dropbox and share the link.
| Type | Description |
|------|-------------|
| `feat` | A new feature |
| `fix` | A bug fix |
| `docs` | Documentation only changes |
| `style` | Formatting, whitespace, no code change |
| `refactor` | Code change that neither fixes a bug nor adds a feature |
| `perf` | Performance improvement |
| `test` | Adding or correcting tests |
| `build` | Build system or external dependencies |
| `ci` | CI configuration files and scripts |
| `chore` | Other changes that don't modify src or test files |
| `revert` | Reverts a previous commit |
### Push your changes
### Scopes
The scope identifies which part of PX4 is affected. Common scopes:
| Scope | Area |
|-------|------|
| `ekf2` | Extended Kalman Filter (state estimation) |
| `mavlink` | MAVLink messaging protocol |
| `commander` | Commander and mode management |
| `navigator` | Mission, RTL, Land, and other navigation modes |
| `sensors` | Sensor drivers and processing |
| `drivers` | Hardware drivers |
| `boards/px4_fmu-v6x` | Board-specific changes (use the board name) |
| `mc_att_control` | Multicopter attitude control |
| `mc_pos_control` | Multicopter position control |
| `fw_att_control` | Fixed-wing attitude control |
| `vtol` | VTOL-specific logic |
| `actuators` | Mixer and actuator output |
| `battery` | Battery monitoring and estimation |
| `logger` | On-board logging |
| `param` | Parameter system |
| `simulation` | SITL, Gazebo, SIH |
| `ci` | Continuous integration and workflows |
| `docs` | Documentation |
| `build` | CMake, toolchain, build system |
| `uorb` | Inter-module messaging |
For changes spanning multiple subsystems, use the primary one affected. Look at the directory path of the files you changed to find the right scope: `src/modules/ekf2/` uses `ekf2`, `src/drivers/imu/` uses `drivers/imu`, `.github/workflows/` uses `ci`.
### Breaking changes
Append `!` before the colon to indicate a breaking change:
```
feat(ekf2)!: remove deprecated height fusion API
```
### Good commit messages
```
feat(ekf2): add height fusion timeout
fix(mavlink): correct BATTERY_STATUS_V2 parsing
refactor(navigator): simplify RTL altitude logic
ci(workflows): migrate to reusable workflows
docs(ekf2): update tuning guide
feat(boards/px4_fmu-v6x)!: remove deprecated driver API
perf(mc_rate_control): reduce loop latency
```
### Commits to avoid
These will be flagged by CI and should be squashed or reworded before merging:
```
fix # too vague, no type or scope
update # too vague, no type or scope
ekf2: fix something # missing type prefix
apply suggestions from code review # squash into parent commit
do make format # squash into parent commit
WIP: trying something # not ready for main
oops # not descriptive
```
### PR titles
The PR title follows the same `type(scope): description` format. This is enforced by CI and is especially important because the PR title becomes the commit message when a PR is squash-merged.
### Merge policy
Commits should be atomic and independently revertable. Squash at reviewer discretion for obvious cases (multiple WIP commits, messy review-response history). When your commits are clean and logical, they will be preserved as individual commits on `main`.
### Cleaning up commits
If CI flags your commit messages, you can fix them with an interactive rebase:
```bash
# Squash all commits into one:
git rebase -i HEAD~N # replace N with the number of commits
# mark all commits except the first as 'squash' or 'fixup'
# reword the remaining commit to follow the format
git push --force-with-lease
# Or reword specific commits:
git rebase -i HEAD~N
# mark the bad commits as 'reword'
git push --force-with-lease
```
## Test your changes
PX4 is safety-critical software. All contributions must include adequate testing where practical:
- **New features** must include unit tests and/or integration tests that exercise the new functionality, where practical. Hardware-dependent changes that cannot be tested in SITL should include bench test or flight test evidence.
- **Bug fixes** must include a regression test where practical. When automated testing is not feasible (hardware-specific issues, race conditions, etc.), provide a link to a flight log demonstrating the fix and the reproduction steps for the original bug.
- **Reviewers** will verify that tests or test evidence exist before approving a pull request.
### Types of tests
| Test type | When to use | How to run |
|-----------|-------------|------------|
| **Unit tests** (gtest) | Module-level logic, math, parsing | `make tests` |
| **SITL integration tests** (MAVSDK) | Flight behavior, failsafes, missions | `test/mavsdk_tests/` |
| **Bench tests / flight logs** | Hardware-dependent changes | Upload logs to [Flight Review](https://logs.px4.io) |
Since we care about safety, we will regularly ask you for test results. Best is to do a test flight (or bench test where it applies) and upload the log file from it (on the microSD card in the logs directory) to Google Drive or Dropbox and share the link.
## Push your changes
Push changes to your repo and send a [pull request](https://github.com/PX4/PX4-Autopilot/compare/).
Push changes to your repo and send a [pull request](https://github.com/PX4/Firmware/compare/).
Make sure to provide some testing feedback and if possible the link to a flight log file. Upload flight log files to [Flight Review](http://logs.px4.io) and link the resulting report.
+15
View File
@@ -0,0 +1,15 @@
## This file should be placed in the root directory of your project.
## Then modify the CMakeLists.txt file in the root directory of your
## project to incorporate the testing dashboard.
##
## # The following are required to submit to the CDash dashboard:
## ENABLE_TESTING()
## INCLUDE(CTest)
set(CTEST_PROJECT_NAME "PX4 Firmware")
set(CTEST_NIGHTLY_START_TIME "00:00:00 EST")
set(CTEST_DROP_METHOD "http")
set(CTEST_DROP_SITE "my.cdash.org")
set(CTEST_DROP_LOCATION "/submit.php?project=PX4+Firmware")
set(CTEST_DROP_SITE_CDASH TRUE)
Vendored
-1
View File
@@ -101,7 +101,6 @@ pipeline {
echo $0;
git clone https://github.com/emscripten-core/emsdk.git _emscripten_sdk;
cd _emscripten_sdk;
git checkout 4.0.15;
./emsdk install latest;
./emsdk activate latest;
cd ..;
+4 -42
View File
@@ -226,22 +226,9 @@ CONFIG_TARGETS_DEFAULT := $(patsubst %_default,%,$(filter %_default,$(ALL_CONFIG
$(CONFIG_TARGETS_DEFAULT):
@$(call cmake-build,$@_default$(BUILD_DIR_SUFFIX))
# Multi-processor boards: build all processor targets together
# VOXL2 apps processor (default) depends on SLPI DSP being built first
modalai_voxl2_default: modalai_voxl2_slpi
modalai_voxl2: modalai_voxl2_slpi
modalai_voxl2_deb: modalai_voxl2_slpi
all_config_targets: $(ALL_CONFIG_TARGETS)
all_default_targets: $(CONFIG_TARGETS_DEFAULT)
# DEB package targets: builds _default config, then runs cpack.
# Multi-processor boards (e.g. VOXL2) chain companion builds automatically
# via existing cmake prerequisites.
%_deb:
@$(call cmake-build,$(subst _deb,_default,$@)$(BUILD_DIR_SUFFIX))
@cd "$(SRC_DIR)/build/$(subst _deb,_default,$@)" && cpack -G DEB
updateconfig:
@./Tools/kconfig/updateconfig.py
@@ -345,7 +332,6 @@ bootloaders_update: \
cuav_7-nano_bootloader \
cuav_fmu-v6x_bootloader \
cuav_x25-evo_bootloader \
cuav_x25-super_bootloader \
cubepilot_cubeorange_bootloader \
cubepilot_cubeorangeplus_bootloader \
hkust_nxt-dual_bootloader \
@@ -426,7 +412,7 @@ tests:
$(call cmake-build,px4_sitl_test)
# work around lcov bug #316; remove once lcov is fixed (see https://github.com/linux-test-project/lcov/issues/316)
LCOBUG = --ignore-errors mismatch,negative
LCOBUG = --ignore-errors mismatch
tests_coverage:
@$(MAKE) clean
@$(MAKE) --no-print-directory tests PX4_CMAKE_BUILD_TYPE=Coverage
@@ -506,29 +492,13 @@ px4_sitl_default-clang:
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && cmake "$(SRC_DIR)" $(CMAKE_ARGS) -G"$(PX4_CMAKE_GENERATOR)" -DCONFIG=px4_sitl_default -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
@$(PX4_MAKE) -C "$(SRC_DIR)"/build/px4_sitl_default-clang
# Paths to exclude from clang-tidy (auto-generated from .gitmodules + manual additions):
# - All submodules (external code we consume, not edit)
# - Test code (allowed looser style)
# - Example code (educational, not production)
# - Vendored third-party code (e.g., CMSIS_5)
# - NuttX-only drivers excluded at CMake level (mcp_common); I2C-dependent libs excluded here (smbus)
# - GPIO excluded here (NuttX platform headers)
# - Emscripten failsafe web build: source path + Unity build path (failsafe_test.dir)
# because CMake Unity Builds merge sources into a generated .cxx under build/
#
# To add manual exclusions, append to CLANG_TIDY_EXCLUDE_EXTRA below.
# Submodules are automatically excluded - no action needed when adding new ones.
CLANG_TIDY_SUBMODULES := $(shell git config --file .gitmodules --get-regexp path | awk '{print $$2}' | tr '\n' '|' | sed 's/|$$//')
CLANG_TIDY_EXCLUDE_EXTRA := src/systemcmds/tests|src/examples|src/modules/gyro_fft/CMSIS_5|src/lib/drivers/smbus|src/drivers/gpio|src/modules/commander/failsafe/emscripten|failsafe_test\.dir|\.pb\.cc
CLANG_TIDY_EXCLUDE := $(CLANG_TIDY_SUBMODULES)|$(CLANG_TIDY_EXCLUDE_EXTRA)
clang-tidy: px4_sitl_default-clang
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && "$(SRC_DIR)"/Tools/run-clang-tidy.py -header-filter=".*\.hpp" -j$(j_clang_tidy) -exclude="$(CLANG_TIDY_EXCLUDE)" -p .
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && "$(SRC_DIR)"/Tools/run-clang-tidy.py -header-filter=".*\.hpp" -j$(j_clang_tidy) -p .
# to automatically fix a single check at a time, eg modernize-redundant-void-arg
# % run-clang-tidy-4.0.py -fix -j4 -checks=-\*,modernize-redundant-void-arg -p .
clang-tidy-fix: px4_sitl_default-clang
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && "$(SRC_DIR)"/Tools/run-clang-tidy.py -header-filter=".*\.hpp" -j$(j_clang_tidy) -exclude="$(CLANG_TIDY_EXCLUDE)" -fix -p .
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && "$(SRC_DIR)"/Tools/run-clang-tidy.py -header-filter=".*\.hpp" -j$(j_clang_tidy) -fix -p .
# TODO: Fix cppcheck errors then try --enable=warning,performance,portability,style,unusedFunction or --enable=all
cppcheck: px4_sitl_default
@@ -548,8 +518,7 @@ validate_module_configs:
-not -path "$(SRC_DIR)/src/modules/zenoh/zenoh-pico/*" \
-not -path "$(SRC_DIR)/src/lib/events/libevents/*" \
-not -path "$(SRC_DIR)/src/lib/cdrstream/*" \
-not -path "$(SRC_DIR)/src/lib/crypto/libtommath/*" \
-not -path "$(SRC_DIR)/src/lib/tensorflow_lite_micro/*" -print0 | \
-not -path "$(SRC_DIR)/src/lib/crypto/libtommath/*" -print0 | \
xargs -0 "$(SRC_DIR)"/Tools/validate_yaml.py --schema-file "$(SRC_DIR)"/validation/module_schema.yaml
# Cleanup
@@ -629,10 +598,3 @@ failsafe_web:
run_failsafe_web_server: failsafe_web
@cd build/px4_sitl_default_failsafe_web && \
python3 -m http.server
# Generate reference documentation for uORB messages
.PHONY: msg_docs
msg_docs:
$(call colorecho,'Generating uORB message reference docs')
@mkdir -p build/msg_docs
@./Tools/msg/generate_msg_docs.py -d build/msg_docs
+37 -85
View File
@@ -1,110 +1,62 @@
<p align="center">
<a href="https://px4.io">
<img src="docs/assets/site/px4_logo.svg" alt="PX4 Autopilot" width="240">
</a>
</p>
# PX4 Drone Autopilot
<p align="center">
<em>The autopilot stack the industry builds on.</em>
</p>
[![Releases](https://img.shields.io/github/release/PX4/PX4-Autopilot.svg)](https://github.com/PX4/PX4-Autopilot/releases) [![DOI](https://zenodo.org/badge/22634/PX4/PX4-Autopilot.svg)](https://zenodo.org/badge/latestdoi/22634/PX4/PX4-Autopilot)
<p align="center">
<a href="https://github.com/PX4/PX4-Autopilot/releases"><img src="https://img.shields.io/github/release/PX4/PX4-Autopilot.svg" alt="Releases"></a>
<a href="https://www.bestpractices.dev/projects/6520"><img src="https://www.bestpractices.dev/projects/6520/badge" alt="OpenSSF Best Practices"></a>
<a href="https://zenodo.org/badge/latestdoi/22634/PX4/PX4-Autopilot"><img src="https://zenodo.org/badge/22634/PX4/PX4-Autopilot.svg" alt="DOI"></a>
<a href="https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml"><img src="https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml/badge.svg?branch=main" alt="Build Targets"></a>
<a href="https://discord.gg/dronecode"><img src="https://discordapp.com/api/guilds/1022170275984457759/widget.png?style=shield" alt="Discord"></a>
</p>
[![Build Targets](https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml/badge.svg?branch=main)](https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml) [![SITL Tests](https://github.com/PX4/PX4-Autopilot/workflows/SITL%20Tests/badge.svg?branch=master)](https://github.com/PX4/PX4-Autopilot/actions?query=workflow%3A%22SITL+Tests%22)
---
[![Discord Shield](https://discordapp.com/api/guilds/1022170275984457759/widget.png?style=shield)](https://discord.gg/dronecode)
## About
This repository holds the [PX4](http://px4.io) flight control solution for drones, with the main applications located in the [src/modules](https://github.com/PX4/PX4-Autopilot/tree/main/src/modules) directory. It also contains the PX4 Drone Middleware Platform, which provides drivers and middleware to run drones.
PX4 is an open-source autopilot stack for drones and unmanned vehicles. It supports multirotors, fixed-wing, VTOL, rovers, and many more experimental platforms from racing quads to industrial survey aircraft. It runs on [NuttX](https://nuttx.apache.org/), Linux, and macOS. Licensed under [BSD 3-Clause](LICENSE).
PX4 is highly portable, OS-independent and supports Linux, NuttX and MacOS out of the box.
## Why PX4
* Official Website: http://px4.io (License: BSD 3-clause, [LICENSE](https://github.com/PX4/PX4-Autopilot/blob/main/LICENSE))
* [Supported airframes](https://docs.px4.io/main/en/airframes/airframe_reference.html) ([portfolio](https://px4.io/ecosystem/commercial-systems/)):
* [Multicopters](https://docs.px4.io/main/en/frames_multicopter/)
* [Fixed wing](https://docs.px4.io/main/en/frames_plane/)
* [VTOL](https://docs.px4.io/main/en/frames_vtol/)
* [Autogyro](https://docs.px4.io/main/en/frames_autogyro/)
* [Rover](https://docs.px4.io/main/en/frames_rover/)
* many more experimental types (Blimps, Boats, Submarines, High Altitude Balloons, Spacecraft, etc)
* Releases: [Downloads](https://github.com/PX4/PX4-Autopilot/releases)
**Modular architecture.** PX4 is built around [uORB](https://docs.px4.io/main/en/middleware/uorb.html), a [DDS](https://docs.px4.io/main/en/middleware/uxrce_dds.html)-compatible publish/subscribe middleware. Modules are fully parallelized and thread safe. You can build custom configurations and trim what you don't need.
## Releases
**Wide hardware support.** PX4 runs on a wide range of [autopilot boards](https://docs.px4.io/main/en/flight_controller/) and supports an extensive set of sensors, telemetry radios, and actuators through the [Pixhawk](https://pixhawk.org/) ecosystem.
Release notes and supporting information for PX4 releases can be found on the [Developer Guide](https://docs.px4.io/main/en/releases/).
**Developer friendly.** First-class support for [MAVLink](https://mavlink.io/) and [DDS / ROS 2](https://docs.px4.io/main/en/ros2/) integration. Comprehensive [SITL simulation](https://docs.px4.io/main/en/simulation/), hardware-in-the-loop testing, and [log analysis](https://docs.px4.io/main/en/log/flight_log_analysis.html) tools. An active developer community on [Discord](https://discord.gg/dronecode) and the [weekly dev call](https://docs.px4.io/main/en/contribute/).
## Building a PX4 based drone, rover, boat or robot
**Vendor neutral governance.** PX4 is hosted under the [Dronecode Foundation](https://www.dronecode.org/), part of the Linux Foundation. Business-friendly BSD-3 license. No single vendor controls the roadmap.
The [PX4 User Guide](https://docs.px4.io/main/en/) explains how to assemble [supported vehicles](https://docs.px4.io/main/en/airframes/airframe_reference.html) and fly drones with PX4. See the [forum and chat](https://docs.px4.io/main/en/#getting-help) if you need help!
## Supported Vehicles
<table>
<tr>
<td align="center">
<a href="https://docs.px4.io/main/en/frames_multicopter/">
<img src="docs/assets/airframes/types/QuadRotorX.svg" width="50" alt="Multicopter"><br>
<sub>Multicopter</sub>
</a>
</td>
<td align="center">
<a href="https://docs.px4.io/main/en/frames_plane/">
<img src="docs/assets/airframes/types/Plane.svg" width="50" alt="Fixed Wing"><br>
<sub>Fixed Wing</sub>
</a>
</td>
<td align="center">
<a href="https://docs.px4.io/main/en/frames_vtol/">
<img src="docs/assets/airframes/types/VTOLPlane.svg" width="50" alt="VTOL"><br>
<sub>VTOL</sub>
</a>
</td>
<td align="center">
<a href="https://docs.px4.io/main/en/frames_rover/">
<img src="docs/assets/airframes/types/Rover.svg" width="50" alt="Rover"><br>
<sub>Rover</sub>
</a>
</td>
</tr>
</table>
## Changing Code and Contributing
<sub>…and many more: helicopters, autogyros, airships, submarines, boats, and other experimental platforms. These frames have basic support but are not part of the regular flight-test program. See the <a href="https://docs.px4.io/main/en/airframes/airframe_reference.html">full airframe reference</a>.</sub>
This [Developer Guide](https://docs.px4.io/main/en/development/development.html) is for software developers who want to modify the flight stack and middleware (e.g. to add new flight modes), hardware integrators who want to support new flight controller boards and peripherals, and anyone who wants to get PX4 working on a new (unsupported) airframe/vehicle.
## Quick Start
Developers should read the [Guide for Contributions](https://docs.px4.io/main/en/contribute/).
See the [forum and chat](https://docs.px4.io/main/en/#getting-help) if you need help!
```bash
git clone https://github.com/PX4/PX4-Autopilot.git --recursive
cd PX4-Autopilot
make px4_sitl
```
> [!NOTE]
> See the [Development Guide](https://docs.px4.io/main/en/development/development.html) for toolchain setup and build options.
## Weekly Dev Call
## Documentation & Resources
The PX4 Dev Team syncs up on a [weekly dev call](https://docs.px4.io/main/en/contribute/).
| Resource | Description |
| --- | --- |
| [User Guide](https://docs.px4.io/main/en/) | Build, configure, and fly with PX4 |
| [Developer Guide](https://docs.px4.io/main/en/development/development.html) | Modify the flight stack, add peripherals, port to new hardware |
| [Airframe Reference](https://docs.px4.io/main/en/airframes/airframe_reference.html) | Full list of supported frames |
| [Autopilot Hardware](https://docs.px4.io/main/en/flight_controller/) | Compatible flight controllers |
| [Release Notes](https://docs.px4.io/main/en/releases/) | What's new in each release |
| [Contribution Guide](https://docs.px4.io/main/en/contribute/) | How to contribute to PX4 |
> **Note** The dev call is open to all interested developers (not just the core dev team). This is a great opportunity to meet the team and contribute to the ongoing development of the platform. It includes a QA session for newcomers. All regular calls are listed in the [Dronecode calendar](https://www.dronecode.org/calendar/).
## Community
- **Weekly Dev Call** — open to all developers ([Dronecode calendar](https://www.dronecode.org/calendar/))
- **Discord** — [Join the Dronecode server](https://discord.gg/dronecode)
- **Discussion Forum** — [PX4 Discuss](https://discuss.px4.io/)
- **Maintainers** — see [`MAINTAINERS.md`](MAINTAINERS.md)
- **Contributor Stats** — [LFX Insights](https://insights.lfx.linuxfoundation.org/foundation/dronecode)
## Maintenance Team
## Contributing
See the latest list of maintainers on [MAINTAINERS](MAINTAINERS.md) file at the root of the project.
We welcome contributions of all kinds — bug reports, documentation, new features, and code reviews. Please read the [Contribution Guide](https://docs.px4.io/main/en/contribute/) to get started.
For the latest stats on contributors please see the latest stats for the Dronecode ecosystem in our project dashboard under [LFX Insights](https://insights.lfx.linuxfoundation.org/foundation/dronecode). For information on how to update your profile and affiliations please see the following support link on how to [Complete Your LFX Profile](https://docs.linuxfoundation.org/lfx/my-profile/complete-your-lfx-profile). Dronecode publishes a yearly snapshot of contributions and achievements on its [website under the Reports section](https://dronecode.org).
## Governance
## Supported Hardware
The PX4 Autopilot project is hosted by the [Dronecode Foundation](https://www.dronecode.org/), a [Linux Foundation](https://www.linuxfoundation.org/) Collaborative Project. Dronecode holds all PX4 trademarks and serves as the project's legal guardian, ensuring vendor-neutral stewardship — no single company owns the name or controls the roadmap. The source code is licensed under the [BSD 3-Clause](LICENSE) license, so you are free to use, modify, and distribute it in your own projects.
For the most up to date information, please visit [PX4 User Guide > Autopilot Hardware](https://docs.px4.io/main/en/flight_controller/).
<p align="center">
<a href="https://www.dronecode.org/">
<img src="docs/assets/site/dronecode_logo.svg" alt="Dronecode Logo" width="180">
</a>
</p>
## Project Governance
The PX4 Autopilot project including all of its trademarks is hosted under [Dronecode](https://www.dronecode.org/), part of the Linux Foundation.
<a href="https://www.dronecode.org/" style="padding:20px" ><img src="https://dronecode.org/wp-content/uploads/sites/24/2020/08/dronecode_logo_default-1.png" alt="Dronecode Logo" width="110px"/></a>
<div style="padding:10px">&nbsp;</div>
@@ -34,7 +34,6 @@ param set-default PWM_MAIN_FUNC2 102
param set-default PWM_MAIN_FUNC3 103
param set-default PWM_MAIN_FUNC4 104
param set-default SENS_GPS0_DELAY 0
param set-default SENS_GPS1_DELAY 0
param set-default EKF2_GPS_DELAY 0
param set SIH_VEHICLE_TYPE 0
@@ -44,8 +44,7 @@ param set-default PWM_MAIN_FUNC2 202
param set-default PWM_MAIN_FUNC3 203
param set-default PWM_MAIN_FUNC4 101
param set-default SENS_GPS0_DELAY 0
param set-default SENS_GPS1_DELAY 0
param set-default EKF2_GPS_DELAY 0
# Rate controllers
param set-default FW_RR_P 0.0500
@@ -11,8 +11,7 @@
PX4_SIMULATOR=${PX4_SIMULATOR:=sihsim}
PX4_SIM_MODEL=${PX4_SIM_MODEL:=xvert}
param set-default SENS_GPS0_DELAY 0
param set-default SENS_GPS1_DELAY 0
param set-default EKF2_GPS_DELAY 0
param set-default EKF2_FUSE_BETA 0 # side slip fusion is currently not supported for tailsitters
param set-default SENS_EN_GPSSIM 1
@@ -27,8 +27,7 @@ param set-default SENS_EN_BAROSIM 1
param set-default SENS_EN_MAGSIM 1
param set-default SENS_EN_ARSPDSIM 1
param set-default SENS_GPS0_DELAY 0
param set-default SENS_GPS1_DELAY 0
param set-default EKF2_GPS_DELAY 0
param set-default VT_TYPE 2
param set-default MPC_MAN_Y_MAX 60
@@ -44,5 +44,4 @@ param set-default PWM_MAIN_FUNC4 104
param set-default PWM_MAIN_FUNC5 105
param set-default PWM_MAIN_FUNC6 106
param set-default SENS_GPS0_DELAY 0
param set-default SENS_GPS1_DELAY 0
param set-default EKF2_GPS_DELAY 0
@@ -44,6 +44,8 @@ param set-default FW_T_SINK_MIN 3
param set-default FW_W_EN 1
param set-default FD_ESCS_EN 0
param set-default MIS_TAKEOFF_ALT 30
param set-default NAV_ACC_RAD 15
@@ -101,6 +101,6 @@ param set-default NAV_ACC_RAD 5
param set-default NAV_DLL_ACT 2
param set-default VT_FWD_THRUST_EN 4
param set-default VT_PITCH_MIN -5
param set-default VT_F_TRANS_THR 1
param set-default VT_F_TRANS_THR 0.3
param set-default VT_TYPE 2
param set-default FD_ESCS_EN 0
@@ -20,8 +20,8 @@ param set-default COM_DISARM_LAND 0.5
# EKF2 parameters
param set-default EKF2_DRAG_CTRL 1
param set-default EKF2_IMU_POS_X 0.02
param set-default SENS_GPS0_OFFX 0.055
param set-default SENS_GPS0_OFFZ -0.15
param set-default EKF2_GPS_POS_X 0.055
param set-default EKF2_GPS_POS_Z -0.15
param set-default EKF2_MIN_RNG 0.03
param set-default EKF2_OF_CTRL 1
param set-default EKF2_OF_POS_X 0.055
@@ -19,6 +19,5 @@ param set-default MNT_MAN_PITCH 2
param set-default MNT_MAN_YAW 3
param set-default MNT_RANGE_ROLL 180
param set-default MNT_MAX_PITCH 45
param set-default MNT_MIN_PITCH -135
param set-default MNT_RANGE_PITCH 180
param set-default MNT_RANGE_YAW 720
@@ -26,6 +26,7 @@ param set-default SENS_EN_GPSSIM 1
param set-default SENS_EN_BAROSIM 1
param set-default SENS_EN_MAGSIM 1
param set-default COM_ARM_CHK_ESCS 0 # We don't have ESCs
param set-default FD_ESCS_EN 0 # We don't have ESCs - but maybe we need this later?
# Set proper failsafes
param set-default COM_ACT_FAIL_ACT 0
@@ -41,21 +42,23 @@ param set-default FD_FAIL_R 0
param set-default CA_ROTOR_COUNT 8
param set-default CA_R_REV 255
param set-default CA_ROTOR0_AX 1
param set-default CA_ROTOR0_AY -1
param set-default CA_ROTOR0_AX -1
param set-default CA_ROTOR0_AY 1
param set-default CA_ROTOR0_AZ 0
param set-default CA_ROTOR0_KM 0
param set-default CA_ROTOR0_PX 0.14
param set-default CA_ROTOR0_PY 0.10
param set-default CA_ROTOR0_PZ 0.06
#param set-default CA_ROTOR0_PZ 0.0
param set-default CA_ROTOR1_AX 1
param set-default CA_ROTOR1_AY 1
param set-default CA_ROTOR1_AX -1
param set-default CA_ROTOR1_AY -1
param set-default CA_ROTOR1_AZ 0
param set-default CA_ROTOR1_KM 0
param set-default CA_ROTOR1_PX 0.14
param set-default CA_ROTOR1_PY -0.10
param set-default CA_ROTOR1_PZ 0.06
#param set-default CA_ROTOR1_PZ 0.0
param set-default CA_ROTOR2_AX 1
param set-default CA_ROTOR2_AY 1
@@ -64,6 +67,7 @@ param set-default CA_ROTOR2_KM 0
param set-default CA_ROTOR2_PX -0.14
param set-default CA_ROTOR2_PY 0.10
param set-default CA_ROTOR2_PZ 0.06
#param set-default CA_ROTOR2_PZ 0.0
param set-default CA_ROTOR3_AX 1
param set-default CA_ROTOR3_AY -1
@@ -75,7 +79,7 @@ param set-default CA_ROTOR3_PZ 0.06
param set-default CA_ROTOR4_AX 0
param set-default CA_ROTOR4_AY 0
param set-default CA_ROTOR4_AZ -1
param set-default CA_ROTOR4_AZ 1
param set-default CA_ROTOR4_KM 0
param set-default CA_ROTOR4_PX 0.12
param set-default CA_ROTOR4_PY 0.22
@@ -99,7 +103,7 @@ param set-default CA_ROTOR6_PZ 0
param set-default CA_ROTOR7_AX 0
param set-default CA_ROTOR7_AY 0
param set-default CA_ROTOR7_AZ -1
param set-default CA_ROTOR7_AZ 1
param set-default CA_ROTOR7_KM 0
param set-default CA_ROTOR7_PX -0.12
param set-default CA_ROTOR7_PY -0.22
@@ -28,6 +28,7 @@ param set-default SIM_GZ_EN 1
param set-default SENS_EN_MAGSIM 1
param set-default COM_ARM_CHK_ESCS 0 # We don't have ESCs
param set-default FD_ESCS_EN 0
param set-default CA_AIRFRAME 14
param set-default MAV_TYPE 45
@@ -1,166 +0,0 @@
#!/bin/sh
#
# @name KTH-ATMOS
#
# @type Free-Flyer
# @class Spacecraft
#
# @output Motor1 back left thruster, +x thrust
# @output Motor2 front left thruster, -x thrust
# @output Motor3 back right thruster, +x thrust
# @output Motor4 front right thruster, -x thrust
# @output Motor5 front left thruster, +y thrust
# @output Motor6 front right thruster, -y thrust
# @output Motor7 back left thruster, +y thrust
# @output Motor8 back right thruster, -y thrust
#
# @maintainer discower-io
# @url https://atmos.discower.io
#
. ${R}etc/init.d/rc.sc_defaults
PX4_SIMULATOR=${PX4_SIMULATOR:=gz}
PX4_GZ_WORLD=${PX4_GZ_WORLD:=default}
PX4_SIM_MODEL=${PX4_SIM_MODEL:=atmos_dual}
param set-default SIM_GZ_EN 1
param set-default SENS_EN_MAGSIM 1
param set-default COM_ARM_CHK_ESCS 0 # We don't have ESCs
param set-default CA_AIRFRAME 14
param set-default MAV_TYPE 45
param set-default CA_ROTOR_COUNT 8
param set-default CA_R_REV 0
# Auto to be provided by Custom Airframe
param set-default CA_METHOD 0
# Set proper failsafes
param set-default COM_ACT_FAIL_ACT 0
param set-default COM_LOW_BAT_ACT 0
param set-default NAV_DLL_ACT 0
param set-default GF_ACTION 1
param set-default NAV_RCL_ACT 1
# disable attitude failure detection
param set-default FD_FAIL_P 0
param set-default FD_FAIL_R 0
param set-default CA_ROTOR0_PX -0.12
param set-default CA_ROTOR0_PY -0.12
param set-default CA_ROTOR0_PZ 0.0
param set-default CA_ROTOR0_CT 1.4
param set-default CA_ROTOR0_AX 1.0
param set-default CA_ROTOR0_AY 0.0
param set-default CA_ROTOR0_AZ 0.0
param set-default CA_ROTOR1_PX 0.12
param set-default CA_ROTOR1_PY -0.12
param set-default CA_ROTOR1_PZ 0.0
param set-default CA_ROTOR1_CT 1.4
param set-default CA_ROTOR1_AX -1.0
param set-default CA_ROTOR1_AY 0.0
param set-default CA_ROTOR1_AZ 0.0
param set-default CA_ROTOR2_PX -0.12
param set-default CA_ROTOR2_PY 0.12
param set-default CA_ROTOR2_PZ 0.0
param set-default CA_ROTOR2_CT 1.4
param set-default CA_ROTOR2_AX 1.0
param set-default CA_ROTOR2_AY 0.0
param set-default CA_ROTOR2_AZ 0.0
param set-default CA_ROTOR3_PX 0.12
param set-default CA_ROTOR3_PY 0.12
param set-default CA_ROTOR3_PZ 0.0
param set-default CA_ROTOR3_CT 1.4
param set-default CA_ROTOR3_AX -1.0
param set-default CA_ROTOR3_AY 0.0
param set-default CA_ROTOR3_AZ 0.0
param set-default CA_ROTOR4_PX 0.12
param set-default CA_ROTOR4_PY -0.12
param set-default CA_ROTOR4_PZ 0.0
param set-default CA_ROTOR4_CT 1.4
param set-default CA_ROTOR4_AX 0.0
param set-default CA_ROTOR4_AY 1.0
param set-default CA_ROTOR4_AZ 0.0
param set-default CA_ROTOR5_PX 0.12
param set-default CA_ROTOR5_PY 0.12
param set-default CA_ROTOR5_PZ 0.0
param set-default CA_ROTOR5_CT 1.4
param set-default CA_ROTOR5_AX 0.0
param set-default CA_ROTOR5_AY -1.0
param set-default CA_ROTOR5_AZ 0.0
param set-default CA_ROTOR6_PX -0.12
param set-default CA_ROTOR6_PY -0.12
param set-default CA_ROTOR6_PZ 0.0
param set-default CA_ROTOR6_CT 1.4
param set-default CA_ROTOR6_AX 0.0
param set-default CA_ROTOR6_AY 1.0
param set-default CA_ROTOR6_AZ 0.0
param set-default CA_ROTOR7_PX -0.12
param set-default CA_ROTOR7_PY 0.12
param set-default CA_ROTOR7_PZ 0.0
param set-default CA_ROTOR7_CT 1.4
param set-default CA_ROTOR7_AX 0.0
param set-default CA_ROTOR7_AY -1.0
param set-default CA_ROTOR7_AZ 0.0
param set-default SIM_GZ_EC_FUNC1 101
param set-default SIM_GZ_EC_FUNC2 102
param set-default SIM_GZ_EC_FUNC3 103
param set-default SIM_GZ_EC_FUNC4 104
param set-default SIM_GZ_EC_FUNC5 105
param set-default SIM_GZ_EC_FUNC6 106
param set-default SIM_GZ_EC_FUNC7 107
param set-default SIM_GZ_EC_FUNC8 108
param set-default SIM_GZ_EC_FUNC9 301
param set-default SIM_GZ_EC_FUNC10 302
param set-default SIM_GZ_EC_FUNC11 303
param set-default SIM_GZ_EC_FUNC12 304
param set-default SIM_GZ_EC_MIN1 0
param set-default SIM_GZ_EC_MIN2 0
param set-default SIM_GZ_EC_MIN3 0
param set-default SIM_GZ_EC_MIN4 0
param set-default SIM_GZ_EC_MIN5 0
param set-default SIM_GZ_EC_MIN6 0
param set-default SIM_GZ_EC_MIN7 0
param set-default SIM_GZ_EC_MIN8 0
param set-default SIM_GZ_EC_MIN9 1100
param set-default SIM_GZ_EC_MIN10 1100
param set-default SIM_GZ_EC_MIN11 1100
param set-default SIM_GZ_EC_MIN12 1100
param set-default SIM_GZ_EC_MAX1 10000
param set-default SIM_GZ_EC_MAX2 10000
param set-default SIM_GZ_EC_MAX3 10000
param set-default SIM_GZ_EC_MAX4 10000
param set-default SIM_GZ_EC_MAX5 10000
param set-default SIM_GZ_EC_MAX6 10000
param set-default SIM_GZ_EC_MAX7 10000
param set-default SIM_GZ_EC_MAX8 10000
param set-default SIM_GZ_EC_MAX9 1900
param set-default SIM_GZ_EC_MAX10 1900
param set-default SIM_GZ_EC_MAX11 1900
param set-default SIM_GZ_EC_MAX12 1900
# Controller Tunings
param set SC_YAWRATE_P 3.335
param set SC_YAWRATE_I 0.87
param set SC_YAWRATE_D 0.15
param set SC_YR_INT_LIM 0.2
param set SC_YAW_P 3.0
param set SPC_POS_P 0.20
param set SPC_VEL_P 6.55
param set SPC_VEL_I 0.0
param set SPC_VEL_D 0.0
param set SPC_VEL_MAX 12.0
@@ -118,7 +118,6 @@ px4_add_romfs_files(
60002_gz_uuv_bluerov2_heavy
70000_gz_atmos
70001_gz_atmos_dual
# [22000, 22999] Reserve for custom models
)
@@ -2,8 +2,7 @@
# shellcheck disable=SC2154
# EKF2 specifics
param set-default SENS_GPS0_DELAY 10
param set-default SENS_GPS1_DELAY 10
param set-default EKF2_GPS_DELAY 10
param set-default EKF2_MULTI_IMU 3
param set-default SENS_IMU_MODE 0
+12 -13
View File
@@ -119,14 +119,22 @@ else
param set SYS_AUTOCONFIG 1
fi
# To trigger a parameter reset during boot SYS_AUTOCONFIG was set to 1 before
if param greater SYS_AUTOCONFIG 0
if param compare SYS_AUTOCONFIG 1
then
# Reset parameters except airframe, parameter version, sensor calibration, total flight time, flight UUID
param reset_all SYS_AUTOSTART SYS_PARAM_VER CAL_* LND_FLIGHT* TC_* COM_FLIGHT*
# Reset params except Airframe, RC calibration, sensor calibration, flight modes, total flight time, and next flight UUID.
param reset_all SYS_AUTOSTART RC* CAL_* COM_FLTMODE* LND_FLIGHT* TC_* COM_FLIGHT*
set AUTOCNF yes
fi
# Allow overriding parameters via env variables: export PX4_PARAM_{name}={value}
env | while IFS='=' read -r line; do
value=${line#*=}
name=${line%%=*}
case $name in
"PX4_PARAM_"*) param set "${name#PX4_PARAM_}" "$value" ;;
esac
done
# multi-instance setup
# shellcheck disable=SC2154
param set MAV_SYS_ID $((px4_instance+1))
@@ -230,15 +238,6 @@ then
exit 1
fi
# Allow overriding parameters via env variables: export PX4_PARAM_{name}={value}
env | while IFS='=' read -r line; do
value=${line#*=}
name=${line%%=*}
case $name in
"PX4_PARAM_"*) param set "${name#PX4_PARAM_}" "$value" ;;
esac
done
dataman start
# only start the simulator if not in replay mode, as both control the lockstep time
@@ -123,9 +123,3 @@ if(CONFIG_MODULES_TEMPERATURE_COMPENSATION)
rc.thermal_cal
)
endif()
if(CONFIG_DRIVERS_VTXTABLE)
px4_add_romfs_files(
rc.vtxtable
)
endif()
@@ -18,8 +18,6 @@
#
# @board px4_fmu-v2 exclude
# @board bitcraze_crazyflie exclude
# @board px4_fmu-v6x exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.fw_defaults
@@ -16,8 +16,6 @@
#
# @board px4_fmu-v2 exclude
# @board bitcraze_crazyflie exclude
# @board px4_fmu-v6x exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.fw_defaults
@@ -7,8 +7,6 @@
#
# @board px4_fmu-v2 exclude
# @board bitcraze_crazyflie exclude
# @board px4_fmu-v6x exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.mc_defaults
@@ -19,8 +17,8 @@ param set-default COM_DISARM_LAND 0.5
# EKF2 parameters
param set-default EKF2_DRAG_CTRL 1
param set-default EKF2_IMU_POS_X 0.02
param set-default SENS_GPS0_OFFX 0.055
param set-default SENS_GPS0_OFFZ -0.15
param set-default EKF2_GPS_POS_X 0.055
param set-default EKF2_GPS_POS_Z -0.15
param set-default EKF2_MIN_RNG 0.03
param set-default EKF2_OF_CTRL 1
param set-default EKF2_OF_POS_X 0.055
@@ -77,6 +75,9 @@ param set-default NAV_ACC_RAD 2
param set-default RTL_DESCEND_ALT 5
param set-default RTL_RETURN_ALT 5
# Logging Parameters
param set-default SDLOG_PROFILE 131
# Sensors Parameters
param set-default SENS_CM8JL65_CFG 104
param set-default SENS_FLOW_MAXHGT 25
@@ -19,8 +19,8 @@ param set-default COM_DISARM_LAND 0.5
# EKF2 parameters
param set-default EKF2_DRAG_CTRL 1
param set-default EKF2_IMU_POS_X 0.02
param set-default SENS_GPS0_OFFX 0.055
param set-default SENS_GPS0_OFFZ -0.15
param set-default EKF2_GPS_POS_X 0.055
param set-default EKF2_GPS_POS_Z -0.15
param set-default EKF2_MIN_RNG 0.03
param set-default EKF2_OF_CTRL 1
param set-default EKF2_OF_POS_X 0.055
@@ -78,6 +78,9 @@ param set-default NAV_ACC_RAD 2
param set-default RTL_DESCEND_ALT 5
param set-default RTL_RETURN_ALT 5
# Logging Parameters
param set-default SDLOG_PROFILE 131
# Sensors Parameters
param set-default SENS_CM8JL65_CFG 202
param set-default SENS_FLOW_MAXHGT 25
@@ -29,6 +29,9 @@ param set-default MPC_MAN_TILT_MAX 60
param set-default THR_MDL_FAC 0.3
# enable high-rate logging profile (helps with tuning)
param set-default SDLOG_PROFILE 19
param set-default IMU_DGYRO_CUTOFF 50
param set-default IMU_GYRO_CUTOFF 90
@@ -2,7 +2,7 @@
#
# @name HolyBro QAV250
#
# @url https://docs.px4.io/main/en/frames_multicopter/holybro_qav250_pixhawk4_mini
# @url https://docs.px4.io/main/en/frames_multicopter/holybro_qav250_pixhawk4_mini.html
#
# @type Quadrotor x
# @class Copter
@@ -11,8 +11,6 @@
#
# @board px4_fmu-v2 exclude
# @board bitcraze_crazyflie exclude
# @board px4_fmu-v6x exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.mc_defaults
@@ -9,8 +9,6 @@
#
# @board px4_fmu-v2 exclude
# @board bitcraze_crazyflie exclude
# @board px4_fmu-v6x exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.mc_defaults
@@ -47,9 +47,8 @@ param set-default EKF2_BCOEF_Y 25.5
param set-default EKF2_DRAG_CTRL 1
param set-default SENS_GPS0_DELAY 100
param set-default SENS_GPS1_DELAY 100
param set-default SENS_GPS0_OFFX 0.06
param set-default EKF2_GPS_DELAY 100
param set-default EKF2_GPS_POS_X 0.06
param set-default EKF2_GPS_V_NOISE 0.5
param set-default EKF2_IMU_POS_X 0.06
@@ -2,7 +2,7 @@
#
# @name Aion Robotics R1 UGV
#
# @url https://docs.px4.io/main/en/complete_vehicles_rover/aion_r1
# @url https://docs.px4.io/main/en/complete_vehicles_rover/aion_r1.html
#
# @type Rover
# @class Rover
@@ -22,9 +22,6 @@
. ${R}etc/init.d/rc.uuv_defaults
# Overwrite DDS AG IP to `192.168.0.1`
param set-default UXRCE_DDS_AG_IP -1062731775
# param set-default MAV_1_CONFIG 102
param set-default BAT1_A_PER_V 37.8798
@@ -12,9 +12,7 @@
# @board px4_fmu-v4pro exclude
# @board px4_fmu-v5 exclude
# @board px4_fmu-v5x exclude
# @board px4_fmu-v6x exclude
# @board bitcraze_crazyflie exclude
# @board ark_fmu-v6x exclude
#
. ${R}etc/init.d/rc.mc_defaults
@@ -20,9 +20,6 @@
. ${R}etc/init.d/rc.sc_defaults
# Overwrite DDS AG IP to `192.168.0.1`
param set-default UXRCE_DDS_AG_IP -1062731775
param set-default CA_AIRFRAME 14
param set-default MAV_TYPE 45
-6
View File
@@ -41,9 +41,3 @@ if param compare -s MC_NN_EN 1
then
mc_nn_control start
fi
if param compare -s MC_RAPTOR_ENABLE 1
then
mc_raptor start
fi
@@ -8,6 +8,9 @@ set VEHICLE_TYPE spacecraft
# MAV_TYPE_SPACECRAFT_ORBITTER
param set-default MAV_TYPE 45
# Set micro-dds-client to use ethernet and IP-address 192.168.0.1
param set-default UXRCE_DDS_AG_IP -1062731775
# Disable preflight disarm to not interfere with external launching
param set-default COM_DISARM_PRFLT -1
param set-default CBRK_SUPPLY_CHK 894281
-6
View File
@@ -237,12 +237,6 @@ then
tla2528 start -X
fi
# Start TMP102 temperature sensor
if param compare -s SENS_EN_TMP102 1
then
tmp102 start -X
fi
# probe for optional external I2C devices
if param compare SENS_EXT_I2C_PRB 1
then
@@ -10,6 +10,9 @@ set VEHICLE_TYPE uuv
# MAV_TYPE_SUBMARINE 12
param set-default MAV_TYPE 12
# Set micro-dds-client to use ethernet and IP-address 192.168.0.1
param set-default UXRCE_DDS_AG_IP -1062731775
# Disable preflight disarm to not interfere with external launching
param set-default COM_DISARM_PRFLT -1
param set-default CBRK_SUPPLY_CHK 894281
-8
View File
@@ -1,8 +0,0 @@
#!/bin/sh
#
# VTX table loading script.
#
# NOTE: Script variables are declared/initialized/unset in the rcS script.
#
vtxtable load
+24 -34
View File
@@ -188,11 +188,11 @@ else
netman update -i eth0
fi
# To trigger a parameter reset during boot SYS_AUTOCONFIG was set to 1 before
# To trigger a parameter reset during boot SYS_AUTCONFIG was set to 1 before
if param greater SYS_AUTOCONFIG 0
then
# Reset parameters except airframe, parameter version, sensor calibration, total flight time, flight UUID
param reset_all SYS_AUTOSTART SYS_PARAM_VER CAL_* LND_FLIGHT* TC_* COM_FLIGHT*
# Reset parameters except airframe, parameter version, RC calibration, sensor calibration, flight modes, total flight time, flight UUID
param reset_all SYS_AUTOSTART SYS_PARAM_VER RC* CAL_* COM_FLTMODE* LND_FLIGHT* TC_* COM_FLIGHT*
fi
#
@@ -479,19 +479,6 @@ else
pwm_out start
fi
#
# Optional UAVCAN/DroneCAN or Cyphal
#
if param greater -s UAVCAN_ENABLE 0
then
uavcan start
else
if param greater -s CYPHAL_ENABLE 0
then
cyphal start
fi
fi
#
# Configure vehicle type specific parameters.
# Note: rc.vehicle_setup is the entry point for all vehicle type specific setup.
@@ -520,11 +507,6 @@ else
#
. ${R}etc/init.d/rc.serial
if param greater -s ZENOH_ENABLE 0
then
zenoh start
fi
# Must be started after the serial config is read
rc_input start $RC_INPUT_ARGS
@@ -630,19 +612,6 @@ else
fi
unset RC_LOGGING
#
# Start the VTX services.
#
if ! param compare VTX_SER_CFG 0
then
set RC_VTXTABLE ${R}etc/init.d/rc.vtxtable
if [ -f ${RC_VTXTABLE} ]
then
. ${RC_VTXTABLE}
fi
unset RC_VTXTABLE
fi
#
# Set additional parameters and env variables for selected AUTOSTART.
#
@@ -659,6 +628,27 @@ else
fi
unset BOARD_BOOTLOADER_UPGRADE
#
# Check if UAVCAN is enabled, default to it for ESCs.
#
if param greater -s UAVCAN_ENABLE 0
then
# Start core UAVCAN module.
if ! uavcan start
then
tune_control play error
fi
else
if param greater -s CYPHAL_ENABLE 0
then
cyphal start
fi
fi
if param greater -s ZENOH_ENABLE 0
then
zenoh start
fi
#
# End of autostart.
#
+12 -28
View File
@@ -2,40 +2,24 @@
## Supported Versions
The following versions receive security updates:
The following is a list of versions the development team is currently supporting.
| Version | Supported |
| ------- | ------------------ |
| 1.16.x | :white_check_mark: |
| < 1.16 | :x: |
| 1.4.x | :white_check_mark: |
| 1.3.3 | :white_check_mark: |
| < 1.3 | :x: |
## Reporting a Vulnerability
We receive security vulnerability reports through GitHub Security Advisories.
We currently only receive security vulnerability reports through GitHub.
To begin a report, go to the [PX4/PX4-Autopilot](https://github.com/PX4/PX4-Autopilot) repository
and click on the **Security** tab. If you are on mobile, click the **...** dropdown menu, then click **Security**.
To begin a report, please go to the top-level repository, for example, PX4/PX4-Autopilot,
and click on the Security tab. If you are on mobile, click the ... dropdown menu, and then click Security.
Click **Report a Vulnerability** to open the advisory form. Fill in the advisory details form.
Make sure your title is descriptive and the description contains all relevant details needed
to verify the issue. We welcome logs, screenshots, photos, and videos.
Click Report a Vulnerability to open the advisory form. Fill in the advisory details form.
Make sure your title is descriptive, and the development team can find all of the relevant details needed
to verify on the description box. We recommend you add as much data as possible. We welcome logs,
screenshots, photos, and videos, anything that can help us verify and identify the issues being reported.
At the bottom of the form, click **Submit report**.
## Response Process
1. **Acknowledgment**: The maintainer team will acknowledge your report within **7 days**.
2. **Triage**: We will assess severity and impact and communicate next steps.
3. **Disclosure**: We coordinate disclosure with the reporter. We follow responsible disclosure practices and will credit reporters in the advisory unless they request anonymity.
If you do not receive acknowledgment within 7 days, please follow up by emailing the [release managers](MAINTAINERS.md).
## Secure Development Practices
The PX4 development team applies the following practices to reduce security risk:
- **Code review**: All changes require peer review before merging.
- **Static analysis**: [clang-tidy](https://clang.llvm.org/extra/clang-tidy/) runs on every pull request with warnings treated as errors.
- **Fuzzing**: A daily fuzzing pipeline using [Google fuzztest](https://github.com/google/fuzztest) tests MAVLink message handling and GNSS driver protocol parsing.
- **Input validation**: All external inputs (MAVLink messages, RC signals, sensor data) are validated against expected ranges before use.
- **Compiler hardening**: Builds use `-Wall -Werror`, stack protectors, and other hardening flags where supported by the target platform.
At the bottom of the form, click Submit report. The maintainer team will be notified and will get back to you ASAP.
@@ -21,13 +21,11 @@ exec find boards msg src platforms test \
-path src/lib/crypto/monocypher -prune -o \
-path src/lib/events/libevents -prune -o \
-path src/lib/parameters/uthash -prune -o \
-path src/lib/rl_tools/rl_tools -prune -o \
-path src/lib/wind_estimator/python/generated -prune -o \
-path src/modules/ekf2/EKF/python/ekf_derivation/generated -prune -o \
-path src/modules/ekf2/EKF/yaw_estimator/derivation/generated -prune -o \
-path src/modules/gyro_fft/CMSIS_5 -prune -o \
-path src/modules/mavlink/mavlink -prune -o \
-path src/modules/mc_raptor/blob -prune -o \
-path test/fuzztest -prune -o \
-path test/mavsdk_tests/catch2 -prune -o \
-path src/lib/crypto/monocypher -prune -o \
@@ -39,8 +37,6 @@ exec find boards msg src platforms test \
-path src/lib/cdrstream/rosidl -prune -o \
-path src/modules/zenoh/zenoh-pico -prune -o \
-path boards/modalai/voxl2/libfc-sensor-api -prune -o \
-path boards/modalai/voxl2/src/lib/mpa/libmodal-json -prune -o \
-path boards/modalai/voxl2/src/lib/mpa/libmodal-pipe -prune -o \
-path src/drivers/actuators/vertiq_io/iq-module-communication-cpp -prune -o \
-path src/lib/tensorflow_lite_micro/tflite_micro -prune -o \
-path src/drivers/ins/sbgecom/sbgECom -prune -o \
+28 -3
View File
@@ -17,12 +17,37 @@ if [[ -f $1"/.git" || -d $1"/.git" ]]; then
SUBMODULE_STATUS=$(git submodule summary "$1")
STATUSRETVAL=$(echo $SUBMODULE_STATUS | grep -A20 -i "$1")
if ! [[ -z "$STATUSRETVAL" ]]; then
echo -e "\033[33mWarning: $1 submodule has uncommitted changes:\033[0m"
echo -e "\033[31mChecked $1 submodule, ACTION REQUIRED:\033[0m"
echo ""
echo -e "Different commits:"
echo -e "$SUBMODULE_STATUS"
echo ""
echo -e "To update submodules to the expected version, run:"
echo -e " \033[94mgit submodule sync --recursive && git submodule update --init --recursive\033[0m"
echo ""
echo -e " *******************************************************************************"
echo -e " * \033[31mIF YOU DID NOT CHANGE THIS FILE (OR YOU DON'T KNOW WHAT A SUBMODULE IS):\033[0m *"
echo -e " * \033[31mHit 'u' and <ENTER> to update ALL submodules and resolve this.\033[0m *"
echo -e " * (performs \033[94mgit submodule sync --recursive\033[0m *"
echo -e " * and \033[94mgit submodule update --init --recursive\033[0m ) *"
echo -e " *******************************************************************************"
echo ""
echo ""
echo -e " Only for EXPERTS:"
echo -e " $1 submodule is not in the recommended version."
echo -e " Hit 'y' and <ENTER> to continue the build with this version. Hit <ENTER> to resolve manually."
echo -e " Use \033[94mgit add $1 && git commit -m 'Updated $1'\033[0m to choose this version (careful!)"
echo ""
read user_cmd
if [ "$user_cmd" == "y" ]; then
echo "Continuing build with manually overridden submodule.."
elif [ "$user_cmd" == "u" ]; then
git submodule sync --recursive -- $1
git submodule update --init --recursive -- $1 || true
git submodule update --init --recursive --force -- $1
echo "Submodule fixed, continuing build.."
else
echo "Build aborted."
exit 1
fi
fi
else
git submodule --quiet sync --recursive --quiet -- $1
-331
View File
@@ -1,331 +0,0 @@
#!/usr/bin/env python3
"""Validate commit messages in a PR against conventional commits format.
Reads a JSON array of GitHub commit objects from stdin (as returned by the
GitHub API's /pulls/{n}/commits endpoint) and checks each message for
blocking errors and advisory warnings.
With --markdown, outputs a formatted PR comment body instead of plain text.
"""
import json
import sys
from conventional_commits import (
EXEMPT_PREFIXES,
parse_header,
)
# Blocking: prefixes that indicate unsquashed fixup commits
FIXUP_PREFIXES = ('fixup!', 'squash!', 'amend!')
# Blocking: single-word throwaway messages (case-insensitive exact match)
THROWAWAY_WORDS = frozenset({
'fix', 'fixed', 'fixes',
'update', 'updated', 'updates',
'test', 'tests', 'testing',
'tmp', 'temp',
'oops', 'wip',
'debug', 'cleanup',
})
# Blocking: debug session leftovers
DEBUG_KEYWORDS = ('tmate',)
# Warning: review-response messages (case-insensitive substring match)
REVIEW_RESPONSE_PATTERNS = (
'address review',
'apply suggestions from code review',
'code review',
)
# Warning: formatter-only commits
FORMATTER_PATTERNS = (
'do make format',
'make format',
'run formatter',
'apply format',
)
MIN_MESSAGE_LENGTH = 5
def check_commit(message: str) -> tuple[list[str], list[str]]:
"""Return (errors, warnings) for a single commit message."""
errors: list[str] = []
warnings: list[str] = []
first_line = message.split('\n', 1)[0].strip()
lower = first_line.lower()
# --- Blocking checks ---
for prefix in FIXUP_PREFIXES:
if lower.startswith(prefix):
errors.append(f'Unsquashed commit: starts with "{prefix}"')
if lower == 'wip' or lower.startswith('wip ') or lower.startswith('wip:'):
errors.append('WIP commit should not be merged')
if len(first_line) < MIN_MESSAGE_LENGTH:
errors.append(f'Message too short ({len(first_line)} chars, minimum {MIN_MESSAGE_LENGTH})')
if first_line.strip() and first_line.strip().lower() in THROWAWAY_WORDS:
errors.append(f'Single-word throwaway message: "{first_line.strip()}"')
for kw in DEBUG_KEYWORDS:
if kw in lower:
errors.append(f'Debug session leftover: contains "{kw}"')
# --- Warning checks ---
for pattern in REVIEW_RESPONSE_PATTERNS:
if pattern in lower:
warnings.append('Review-response commit')
break
for pattern in FORMATTER_PATTERNS:
if pattern in lower:
warnings.append('Formatter-only commit')
break
if not parse_header(first_line):
# Exempt merge commits
for prefix in EXEMPT_PREFIXES:
if first_line.startswith(prefix):
break
else:
warnings.append(
'Missing conventional commit format '
'(e.g. "feat(ekf2): add something")'
)
return errors, warnings
def suggest_commit(message: str) -> str | None:
"""Suggest how to fix a bad commit message."""
first_line = message.split('\n', 1)[0].strip()
lower = first_line.lower()
for prefix in FIXUP_PREFIXES:
if lower.startswith(prefix):
return 'Squash this into the commit it fixes'
if lower == 'wip' or lower.startswith('wip ') or lower.startswith('wip:'):
return 'Reword with a descriptive message (e.g. "feat(scope): what changed")'
if len(first_line) < MIN_MESSAGE_LENGTH:
return 'Reword with a descriptive message (e.g. "feat(ekf2): what changed")'
if first_line.strip().lower() in THROWAWAY_WORDS:
return 'Reword with a descriptive message (e.g. "fix(scope): what changed")'
return None
def format_plain(data: list) -> tuple[bool, bool]:
"""Print plain text output. Returns (has_blocking, has_warnings)."""
has_blocking = False
has_warnings = False
for commit in data:
sha = commit.get('sha', '?')[:10]
message = commit.get('commit', {}).get('message', '')
first_line = message.split('\n', 1)[0].strip()
errors, warnings = check_commit(message)
if errors or warnings:
print(f"\n {sha} {first_line}")
for err in errors:
print(f" ERROR: {err}")
has_blocking = True
for warn in warnings:
print(f" WARNING: {warn}")
has_warnings = True
if has_blocking:
print(
"\n"
"ERROR = must fix before merging (CI will block the PR)\n"
"WARNING = advisory, not blocking, but recommended to fix\n"
"\n"
"See the contributing guide for details:\n"
" https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention\n",
)
elif has_warnings:
print(
"\n"
"WARNING = advisory, not blocking, but recommended to fix\n"
"\n"
"See the contributing guide for details:\n"
" https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention\n",
)
return has_blocking, has_warnings
def format_markdown_blocking(data: list) -> str:
"""Format a blocking error markdown comment."""
error_groups: dict[str, list[str]] = {}
unique_commits: list[tuple[str, str, list[str], str]] = []
for commit in data:
sha = commit.get('sha', '?')[:10]
message = commit.get('commit', {}).get('message', '')
first_line = message.split('\n', 1)[0].strip()
errors, _ = check_commit(message)
if not errors:
continue
suggestion = suggest_commit(message) or ''
unique_commits.append((sha, first_line, errors, suggestion))
for err in errors:
error_groups.setdefault(err, []).append(sha)
lines = [
"## \u274c Commit messages need attention before merging",
"",
]
has_large_group = any(len(shas) > 3 for shas in error_groups.values())
if has_large_group:
lines.extend([
"**Issues found:**",
"",
])
for err_msg, shas in error_groups.items():
if len(shas) > 3:
lines.append(f"- **{len(shas)} commits**: {err_msg} "
f"(`{shas[0]}`, `{shas[1]}`, ... `{shas[-1]}`)")
else:
sha_list = ', '.join(f'`{s}`' for s in shas)
lines.append(f"- {err_msg}: {sha_list}")
distinct_messages = {msg for _, msg, _, _ in unique_commits}
if len(distinct_messages) <= 5:
lines.extend(["", "**Affected commits:**", ""])
for sha, msg, errors, suggestion in unique_commits:
safe_msg = msg.replace('|', '\\|')
lines.append(f"- `{sha}` {safe_msg}")
else:
lines.extend([
"| Commit | Message | Issue | Suggested fix |",
"|--------|---------|-------|---------------|",
])
for sha, msg, errors, suggestion in unique_commits:
issues = '; '.join(errors)
safe_msg = msg.replace('|', '\\|')
lines.append(f"| `{sha}` | {safe_msg} | {issues} | {suggestion} |")
lines.extend([
"",
"See [CONTRIBUTING.md](https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention) "
"for how to clean up commits.",
"",
"---",
"*This comment will be automatically removed once the issues are resolved.*",
])
return '\n'.join(lines)
def format_markdown_advisory(data: list) -> str:
"""Format an advisory warning markdown comment."""
lines = [
"## \U0001f4a1 Commit messages could be improved",
"",
"Not blocking, but these commit messages could use some cleanup.",
"",
"| Commit | Message | Suggestion |",
"|--------|---------|------------|",
]
for commit in data:
sha = commit.get('sha', '?')[:10]
message = commit.get('commit', {}).get('message', '')
first_line = message.split('\n', 1)[0].strip()
_, warnings = check_commit(message)
if not warnings:
continue
suggestion = '; '.join(warnings)
safe_msg = first_line.replace('|', '\\|')
lines.append(f"| `{sha}` | {safe_msg} | {suggestion} |")
lines.extend([
"",
"See the [commit message convention](https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention) "
"for details.",
"",
"---",
"*This comment will be automatically removed once the issues are resolved.*",
])
return '\n'.join(lines)
def main() -> None:
markdown_stdout = '--markdown' in sys.argv
markdown_file = None
for i, a in enumerate(sys.argv):
if a == '--markdown-file' and i + 1 < len(sys.argv):
markdown_file = sys.argv[i + 1]
elif a.startswith('--markdown-file='):
markdown_file = a.split('=', 1)[1]
try:
data = json.load(sys.stdin)
except json.JSONDecodeError as exc:
print(f"Failed to parse JSON input: {exc}", file=sys.stderr)
sys.exit(2)
if not isinstance(data, list):
print("Expected a JSON array of commit objects.", file=sys.stderr)
sys.exit(2)
# Always compute blocking/warning state
has_blocking = False
has_warnings = False
for commit in data:
message = commit.get('commit', {}).get('message', '')
errors, warnings = check_commit(message)
if errors:
has_blocking = True
if warnings:
has_warnings = True
# Generate markdown if needed
md = None
if has_blocking:
md = format_markdown_blocking(data)
elif has_warnings:
md = format_markdown_advisory(data)
if md:
if markdown_stdout:
print(md)
if markdown_file:
with open(markdown_file, 'w') as f:
f.write(md + '\n')
elif markdown_file:
with open(markdown_file, 'w') as f:
pass
# Plain text output to stderr for CI logs (always, unless --markdown only)
if not markdown_stdout:
has_blocking, _ = format_plain(data)
sys.exit(1 if has_blocking else 0)
if __name__ == '__main__':
main()
-163
View File
@@ -1,163 +0,0 @@
#!/usr/bin/env python3
"""Validate that a PR title follows conventional commits format.
Format: type(scope): description
Can output plain text for CI logs or markdown for PR comments.
"""
import re
import sys
from conventional_commits import (
CONVENTIONAL_TYPES,
EXEMPT_PREFIXES,
parse_header,
suggest_scope,
suggest_type,
)
def suggest_title(title: str) -> str | None:
"""Try to suggest a corrected title in conventional commits format."""
stripped = title.strip()
# Remove common bracket prefixes like [docs], [CI], etc.
bracket_match = re.match(r'^\[([^\]]+)\]\s*(.+)', stripped)
if bracket_match:
prefix = bracket_match.group(1).strip().lower()
rest = bracket_match.group(2).strip()
rest = re.sub(r'^[\-:]\s*', '', rest).strip()
if len(rest) >= 5:
# Try to map bracket content to a type
commit_type = prefix if prefix in CONVENTIONAL_TYPES else suggest_type(rest)
scope = suggest_scope(rest)
if scope:
return f"{commit_type}({scope}): {rest}"
# Already has old-style "subsystem: description" format - convert it
colon_match = re.match(r'^([a-zA-Z][a-zA-Z0-9_/\-\. ]*): (.+)$', stripped)
if colon_match:
old_subsystem = colon_match.group(1).strip()
desc = colon_match.group(2).strip()
if len(desc) >= 5:
commit_type = suggest_type(desc)
# Use the old subsystem as scope (clean it up)
scope = old_subsystem.lower().replace(' ', '_')
return f"{commit_type}({scope}): {desc}"
# No format at all - try to guess both type and scope
commit_type = suggest_type(stripped)
scope = suggest_scope(stripped)
if scope:
desc = stripped[0].lower() + stripped[1:] if stripped else stripped
return f"{commit_type}({scope}): {desc}"
return None
def check_title(title: str) -> bool:
title = title.strip()
if not title:
print("PR title is empty.", file=sys.stderr)
return False
for prefix in EXEMPT_PREFIXES:
if title.startswith(prefix):
return True
if parse_header(title):
return True
types_str = ', '.join(f'`{t}`' for t in CONVENTIONAL_TYPES.keys())
print(
f"PR title does not match conventional commits format.\n"
f"\n"
f" Title: {title}\n"
f"\n"
f"Expected format: type(scope): description\n"
f"\n"
f"Valid types: {types_str}\n"
f"\n"
f"Good examples:\n"
f" feat(ekf2): add height fusion timeout\n"
f" fix(mavlink): correct BATTERY_STATUS_V2 parsing\n"
f" ci(workflows): migrate to reusable workflows\n"
f" feat(boards/px4_fmu-v6x)!: remove deprecated driver API\n"
f"\n"
f"Bad examples:\n"
f" fix stuff\n"
f" Update file\n"
f" ekf2: fix something (missing type prefix)\n"
f"\n"
f"See the contributing guide for details:\n"
f" https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention\n",
file=sys.stderr,
)
return False
def format_markdown(title: str) -> str:
"""Format a markdown PR comment body for a bad title."""
lines = [
"## \u274c PR title needs conventional commit format",
"",
"Expected format: `type(scope): description` "
"([conventional commits](https://www.conventionalcommits.org/)).",
"",
"**Your title:**",
f"> {title}",
"",
]
suggestion = suggest_title(title)
if suggestion:
lines.extend([
"**Suggested fix:**",
f"> {suggestion}",
"",
])
lines.extend([
"**To fix this:** click the ✏️ next to the PR title at the top "
"of this page and update it.",
"",
"See [CONTRIBUTING.md](https://github.com/PX4/PX4-Autopilot/blob/main/CONTRIBUTING.md#commit-message-convention) "
"for details.",
"",
"---",
"*This comment will be automatically removed once the issue is resolved.*",
])
return '\n'.join(lines)
def main() -> None:
import argparse
parser = argparse.ArgumentParser(description='Check PR title format')
parser.add_argument('title', help='The PR title to validate')
parser.add_argument('--markdown', action='store_true',
help='Output markdown to stdout on failure')
parser.add_argument('--markdown-file', metavar='FILE',
help='Write markdown to FILE on failure')
args = parser.parse_args()
passed = check_title(args.title)
if not passed:
md = format_markdown(args.title)
if args.markdown:
print(md)
if args.markdown_file:
with open(args.markdown_file, 'w') as f:
f.write(md + '\n')
elif args.markdown_file:
with open(args.markdown_file, 'w') as f:
pass
sys.exit(0 if passed else 1)
if __name__ == '__main__':
main()
-146
View File
@@ -1,146 +0,0 @@
"""Shared constants and helpers for conventional commit validation.
Format: type(scope): description
Optional breaking change marker: type(scope)!: description
"""
import re
CONVENTIONAL_TYPES = {
'feat': 'A new feature',
'fix': 'A bug fix',
'docs': 'Documentation only changes',
'style': 'Formatting, whitespace, no code change',
'refactor': 'Code change that neither fixes a bug nor adds a feature',
'perf': 'Performance improvement',
'test': 'Adding or correcting tests',
'build': 'Build system or external dependencies',
'ci': 'CI configuration files and scripts',
'chore': 'Other changes that don\'t modify src or test files',
'revert': 'Reverts a previous commit',
}
# type(scope)[!]: description
# - type: one of CONVENTIONAL_TYPES keys
# - scope: required, alphanumeric with _/-/.
# - !: optional breaking change marker
# - description: at least 5 chars
HEADER_PATTERN = re.compile(
r'^(' + '|'.join(CONVENTIONAL_TYPES.keys()) + r')'
r'\(([a-zA-Z0-9_/\-\.]+)\)'
r'(!)?'
r': (.{5,})$'
)
EXEMPT_PREFIXES = ('Merge ',)
# Common PX4 subsystem scopes for suggestions
KNOWN_SCOPES = [
'ekf2', 'mavlink', 'commander', 'navigator', 'sensors',
'mc_att_control', 'mc_pos_control', 'mc_rate_control',
'fw_att_control', 'fw_pos_control', 'fw_rate_control',
'vtol', 'actuators', 'battery', 'param', 'logger',
'uorb', 'drivers', 'boards', 'simulation', 'sitl',
'gps', 'rc', 'safety', 'can', 'serial',
'ci', 'docs', 'build', 'cmake', 'tools',
'mixer', 'land_detector', 'airspeed', 'gyroscope',
'accelerometer', 'magnetometer', 'barometer',
]
# Keyword patterns to suggest scopes from description text
KEYWORD_SCOPES = [
(r'\b(ekf|estimator|height|fusion|imu|baro)\b', 'ekf2'),
(r'\b(mavlink|MAVLink|MAVLINK|command_int|heartbeat)\b', 'mavlink'),
(r'\b(uorb|orb|pub|sub|topic)\b', 'uorb'),
(r'\b(board|fmu|nuttx|stm32)\b', 'boards'),
(r'\b(mixer|actuator|motor|servo|pwm|dshot)\b', 'actuators'),
(r'\b(battery|power)\b', 'battery'),
(r'\b(param|parameter)\b', 'param'),
(r'\b(log|logger|sdlog)\b', 'logger'),
(r'\b(sensor|accel|gyro)\b', 'sensors'),
(r'\b(land|takeoff|rtl|mission|navigator|geofence)\b', 'navigator'),
(r'\b(position|velocity|attitude|rate)\s*(control|ctrl)\b', 'mc_att_control'),
(r'\b(mc|multicopter|quad)\b', 'mc_att_control'),
(r'\b(fw|fixedwing|fixed.wing|plane)\b', 'fw_att_control'),
(r'\b(vtol|transition)\b', 'vtol'),
(r'\b(ci|workflow|github.action|pipeline)\b', 'ci'),
(r'\b(doc|docs|documentation|readme)\b', 'docs'),
(r'\b(cmake|make|toolchain|compiler)\b', 'build'),
(r'\b(sitl|simulation|gazebo|jmavsim|sih)\b', 'simulation'),
(r'\b(can|uavcan|cyphal|dronecan)\b', 'can'),
(r'\b(serial|uart|spi|i2c)\b', 'serial'),
(r'\b(safety|failsafe|arm|disarm|kill)\b', 'safety'),
(r'\b(rc|radio|sbus|crsf|elrs|dsm)\b', 'rc'),
(r'\b(gps|gnss|rtk|ubx)\b', 'gps'),
(r'\b(optical.flow|flow|rangefinder|lidar|distance)\b', 'sensors'),
(r'\b(orbit|follow|offboard)\b', 'commander'),
(r'\b(driver)\b', 'drivers'),
]
# Verb patterns to suggest conventional commit type
VERB_TYPE_MAP = [
(r'^fix(e[ds])?[\s:]', 'fix'),
(r'^bug[\s:]', 'fix'),
(r'^add(s|ed|ing)?[\s:]', 'feat'),
(r'^implement', 'feat'),
(r'^introduce', 'feat'),
(r'^support', 'feat'),
(r'^enable', 'feat'),
(r'^update[ds]?[\s:]', 'feat'),
(r'^improv(e[ds]?|ing)', 'perf'),
(r'^optimi[zs](e[ds]?|ing)', 'perf'),
(r'^refactor', 'refactor'),
(r'^clean\s*up', 'refactor'),
(r'^restructure', 'refactor'),
(r'^simplif(y|ied)', 'refactor'),
(r'^remov(e[ds]?|ing)', 'refactor'),
(r'^delet(e[ds]?|ing)', 'refactor'),
(r'^deprecat', 'refactor'),
(r'^replac(e[ds]?|ing)', 'refactor'),
(r'^renam(e[ds]?|ing)', 'refactor'),
(r'^migrat', 'refactor'),
(r'^revert', 'revert'),
(r'^doc(s|ument)', 'docs'),
(r'^test', 'test'),
(r'^format', 'style'),
(r'^lint', 'style'),
(r'^whitespace', 'style'),
(r'^build', 'build'),
(r'^ci[\s:]', 'ci'),
]
def parse_header(text: str) -> dict | None:
"""Parse a conventional commit header into components.
Returns dict with keys {type, scope, breaking, subject} or None if
the text doesn't match conventional commits format.
"""
text = text.strip()
m = HEADER_PATTERN.match(text)
if not m:
return None
return {
'type': m.group(1),
'scope': m.group(2),
'breaking': m.group(3) == '!',
'subject': m.group(4),
}
def suggest_type(text: str) -> str:
"""Infer a conventional commit type from description text."""
lower = text.strip().lower()
for pattern, commit_type in VERB_TYPE_MAP:
if re.search(pattern, lower):
return commit_type
return 'feat'
def suggest_scope(text: str) -> str | None:
"""Infer a scope from keywords in the text."""
lower = text.strip().lower()
for pattern, scope in KEYWORD_SCOPES:
if re.search(pattern, lower, re.IGNORECASE):
return scope
return None
+7 -88
View File
@@ -36,20 +36,11 @@ if args.filter:
target_filter.append(target)
default_container = 'ghcr.io/px4/px4-dev:v1.16.0-rc1-258-g0369abd556'
voxl2_container = 'ghcr.io/px4/px4-dev-voxl2:v1.5'
build_configs = []
grouped_targets = {}
excluded_boards = ['px4_ros2', 'espressif_esp32'] # TODO: fix and enable
excluded_boards = ['modalai_voxl2', 'px4_ros2', 'espressif_esp32'] # TODO: fix and enable
excluded_manufacturers = ['atlflight']
excluded_platforms = []
# Container overrides for platforms/boards that need a non-default container
platform_container_overrides = {
'qurt': voxl2_container,
}
board_container_overrides = {
'modalai_voxl2': voxl2_container,
}
excluded_platforms = ['qurt']
excluded_labels = [
'stackcheck',
'nolockstep', 'replay', 'test',
@@ -97,20 +88,7 @@ def process_target(px4board_file, target_name):
if platform not in excluded_platforms:
container = default_container
# Extract board name (manufacturer_board) from target name
board_name = '_'.join(target_name.split('_')[:2])
# Apply container overrides for specific platforms or boards
if platform in platform_container_overrides:
container = platform_container_overrides[platform]
if board_name in board_container_overrides:
container = board_container_overrides[board_name]
# Boards with container overrides get their own group
if board_name in board_container_overrides or platform in platform_container_overrides:
group = 'voxl2'
elif platform == 'posix':
if platform == 'posix':
group = 'base'
if toolchain:
if toolchain.startswith('aarch64'):
@@ -148,18 +126,18 @@ grouped_targets['base']['manufacturers'] = {}
grouped_targets['base']['manufacturers']['px4'] = []
grouped_targets['base']['manufacturers']['px4'] += metadata_targets
for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), key=lambda e: e.name):
for manufacturer in os.scandir(os.path.join(source_dir, '../boards')):
if not manufacturer.is_dir():
continue
if manufacturer.name in excluded_manufacturers:
if verbose: print(f'excluding manufacturer {manufacturer.name}')
continue
for board in sorted(os.scandir(manufacturer.path), key=lambda e: e.name):
for board in os.scandir(manufacturer.path):
if not board.is_dir():
continue
for files in sorted(os.scandir(board.path), key=lambda e: e.name):
for files in os.scandir(board.path):
if files.is_file() and files.name.endswith('.px4board'):
board_name = manufacturer.name + '_' + board.name
@@ -189,65 +167,6 @@ for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), ke
if target is not None:
build_configs.append(target)
# Remove companion targets from CI groups (parent target builds them via Make prerequisite)
for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), key=lambda e: e.name):
if not manufacturer.is_dir():
continue
for board in sorted(os.scandir(manufacturer.path), key=lambda e: e.name):
if not board.is_dir():
continue
companion_file = os.path.join(board.path, 'companion_targets')
if os.path.exists(companion_file):
with open(companion_file) as f:
companions = {l.strip() for l in f if l.strip() and not l.startswith('#')}
for arch in grouped_targets:
for man in grouped_targets[arch]['manufacturers']:
grouped_targets[arch]['manufacturers'][man] = [
t for t in grouped_targets[arch]['manufacturers'][man]
if t not in companions
]
# Append _deb targets for boards that have cmake/package.cmake
for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), key=lambda e: e.name):
if not manufacturer.is_dir():
continue
if manufacturer.name in excluded_manufacturers:
continue
for board in sorted(os.scandir(manufacturer.path), key=lambda e: e.name):
if not board.is_dir():
continue
board_name = manufacturer.name + '_' + board.name
if board_name in excluded_boards:
continue
package_cmake = os.path.join(board.path, 'cmake', 'package.cmake')
if os.path.exists(package_cmake):
deb_target = board_name + '_deb'
if target_filter and not any(deb_target.startswith(f) for f in target_filter):
continue
# Determine the container and group for this board
container = default_container
if board_name in board_container_overrides:
container = board_container_overrides[board_name]
target_entry = {'target': deb_target, 'container': container}
if args.group:
# Find the group where this board's _default target already lives
default_target = board_name + '_default'
group = None
for g in grouped_targets:
targets_in_group = grouped_targets[g].get('manufacturers', {}).get(manufacturer.name, [])
if default_target in targets_in_group:
group = g
break
if group is None:
group = 'base'
target_entry['arch'] = group
if group not in grouped_targets:
grouped_targets[group] = {'container': container, 'manufacturers': {}}
if manufacturer.name not in grouped_targets[group]['manufacturers']:
grouped_targets[group]['manufacturers'][manufacturer.name] = []
grouped_targets[group]['manufacturers'][manufacturer.name].append(deb_target)
build_configs.append(target_entry)
if(verbose):
import pprint
print("============================")
@@ -284,7 +203,7 @@ if (args.group):
if(verbose):
print(f'=:Architectures: [{grouped_targets.keys()}]')
for arch in grouped_targets:
runner = 'x64' if arch in ('nuttx', 'voxl2') else 'arm64'
runner = 'x64' if arch == 'nuttx' else 'arm64'
if(verbose):
print(f'=:Processing: [{arch}]')
temp_group = []
-431
View File
@@ -1,431 +0,0 @@
#!/usr/bin/env bash
#
# metadata_sync.sh - Unified metadata generation and synchronization for PX4 docs
#
# Usage:
# Tools/ci/metadata_sync.sh [OPTIONS] [TYPES...]
#
# Types:
# parameters - Parameter reference (docs/en/advanced_config/parameter_reference.md)
# airframes - Airframe reference (docs/en/airframes/airframe_reference.md)
# modules - Module documentation (docs/en/modules/*.md)
# msg_docs - uORB message docs (docs/en/msg_docs/*.md + docs/en/middleware/dds_topics.md)
# uorb_graphs - uORB graph JSONs (docs/public/middleware/*.json)
# failsafe_web - Failsafe simulator (docs/public/config/failsafe/*.{js,wasm,json})
# all - All of the above (default)
#
# Options:
# --generate Build the make targets to generate fresh metadata
# --sync Copy generated files to docs/
# --verbose Show detailed output
# --help Show this help
#
# Exit codes:
# 0 - Success (files synced or already up-to-date)
# 1 - Error (build failed, missing files, etc.)
#
# Examples:
# # Full regeneration and sync (orchestrator use case)
# Tools/ci/metadata_sync.sh --generate --sync all
#
# # Just sync specific type (assumes already built)
# Tools/ci/metadata_sync.sh --sync parameters
#
# # Generate only, don't copy
# Tools/ci/metadata_sync.sh --generate uorb_graphs
#
set -euo pipefail
shopt -s nullglob
# ═══════════════════════════════════════════════════════════════════════════════
# Configuration
# ═══════════════════════════════════════════════════════════════════════════════
EMSCRIPTEN_VERSION="3.1.64"
EMSDK_DIR="${EMSDK_DIR:-_emscripten_sdk}"
# All available metadata types
ALL_TYPES=(parameters airframes modules msg_docs uorb_graphs failsafe_web)
# ═══════════════════════════════════════════════════════════════════════════════
# Logging
# ═══════════════════════════════════════════════════════════════════════════════
VERBOSE=false
log() {
echo "[metadata_sync] $*"
}
log_verbose() {
if [[ "$VERBOSE" == "true" ]]; then
echo "[metadata_sync] $*"
fi
}
die() {
echo "[metadata_sync] ERROR: $*" >&2
exit 1
}
# ═══════════════════════════════════════════════════════════════════════════════
# Help
# ═══════════════════════════════════════════════════════════════════════════════
show_help() {
head -n 35 "$0" | tail -n +2 | sed 's/^# \?//'
exit 0
}
# ═══════════════════════════════════════════════════════════════════════════════
# Emscripten Setup
# ═══════════════════════════════════════════════════════════════════════════════
ensure_emscripten() {
if command -v emcc >/dev/null 2>&1; then
log_verbose "Emscripten already available: $(emcc --version | head -1)"
return 0
fi
log "Setting up Emscripten ${EMSCRIPTEN_VERSION}..."
if [[ ! -d "$EMSDK_DIR" ]]; then
log_verbose "Cloning emsdk to $EMSDK_DIR"
if [[ "$VERBOSE" == "true" ]]; then
git clone https://github.com/emscripten-core/emsdk.git "$EMSDK_DIR"
else
git clone https://github.com/emscripten-core/emsdk.git "$EMSDK_DIR" >/dev/null 2>&1
fi
fi
pushd "$EMSDK_DIR" >/dev/null
if [[ "$VERBOSE" == "true" ]]; then
./emsdk install "$EMSCRIPTEN_VERSION"
./emsdk activate "$EMSCRIPTEN_VERSION"
else
./emsdk install "$EMSCRIPTEN_VERSION" >/dev/null 2>&1
./emsdk activate "$EMSCRIPTEN_VERSION" >/dev/null 2>&1
fi
popd >/dev/null
# shellcheck source=/dev/null
source "${EMSDK_DIR}/emsdk_env.sh" >/dev/null 2>&1
log_verbose "Emscripten ready: $(emcc --version | head -1)"
}
# ═══════════════════════════════════════════════════════════════════════════════
# Generation Functions
# ═══════════════════════════════════════════════════════════════════════════════
generate_parameters() {
log "Generating parameters metadata..."
if [[ "$VERBOSE" == "true" ]]; then
make parameters_metadata
else
make parameters_metadata >/dev/null
fi
}
generate_airframes() {
log "Generating airframes metadata..."
if [[ "$VERBOSE" == "true" ]]; then
make airframe_metadata
else
make airframe_metadata >/dev/null
fi
}
generate_modules() {
log "Generating modules documentation..."
if [[ "$VERBOSE" == "true" ]]; then
make module_documentation
else
make module_documentation >/dev/null
fi
}
generate_msg_docs() {
log "Generating message documentation..."
if [[ "$VERBOSE" == "true" ]]; then
make msg_docs
else
make msg_docs >/dev/null
fi
}
generate_uorb_graphs() {
log "Generating uORB graphs..."
if [[ "$VERBOSE" == "true" ]]; then
make uorb_graphs
else
make uorb_graphs >/dev/null
fi
}
generate_failsafe_web() {
ensure_emscripten
log "Generating failsafe web..."
if [[ "$VERBOSE" == "true" ]]; then
make failsafe_web
else
make failsafe_web >/dev/null
fi
}
# ═══════════════════════════════════════════════════════════════════════════════
# Sync Functions
# ═══════════════════════════════════════════════════════════════════════════════
sync_parameters() {
local src="build/px4_sitl_default/docs/parameters.md"
local dest="docs/en/advanced_config/parameter_reference.md"
log "Syncing parameters..."
if [[ ! -f "$src" ]]; then
die "Source file not found: $src (did you run --generate?)"
fi
mkdir -p "$(dirname "$dest")"
cp "$src" "$dest"
log_verbose " $src -> $dest"
}
sync_airframes() {
local src="build/px4_sitl_default/docs/airframes.md"
local dest="docs/en/airframes/airframe_reference.md"
log "Syncing airframes..."
if [[ ! -f "$src" ]]; then
die "Source file not found: $src (did you run --generate?)"
fi
mkdir -p "$(dirname "$dest")"
cp "$src" "$dest"
log_verbose " $src -> $dest"
}
sync_modules() {
local src_dir="build/px4_sitl_default/docs/modules"
local dest_dir="docs/en/modules"
log "Syncing modules..."
if [[ ! -d "$src_dir" ]]; then
die "Source directory not found: $src_dir (did you run --generate?)"
fi
local src_files=("$src_dir"/*.md)
if [[ ${#src_files[@]} -eq 0 ]]; then
die "No .md files found in $src_dir"
fi
mkdir -p "$dest_dir"
for src in "${src_files[@]}"; do
local name
name=$(basename "$src")
cp "$src" "$dest_dir/$name"
log_verbose " $src -> $dest_dir/$name"
done
}
sync_msg_docs() {
local src_dir="build/msg_docs"
local dest_dir="docs/en/msg_docs"
local middleware_dir="docs/en/middleware"
log "Syncing message docs..."
if [[ ! -d "$src_dir" ]]; then
die "Source directory not found: $src_dir (did you run --generate?)"
fi
local src_files=("$src_dir"/*.md)
if [[ ${#src_files[@]} -eq 0 ]]; then
die "No .md files found in $src_dir"
fi
mkdir -p "$dest_dir"
mkdir -p "$middleware_dir"
for src in "${src_files[@]}"; do
local name
name=$(basename "$src")
# dds_topics.md goes to middleware dir
if [[ "$name" == "dds_topics.md" ]]; then
cp "$src" "$middleware_dir/$name"
log_verbose " $src -> $middleware_dir/$name"
else
cp "$src" "$dest_dir/$name"
log_verbose " $src -> $dest_dir/$name"
fi
done
}
sync_uorb_graphs() {
local src_dir="Tools/uorb_graph"
local dest_dir="docs/public/middleware"
log "Syncing uORB graphs..."
local src_files=("$src_dir"/*.json)
if [[ ${#src_files[@]} -eq 0 ]]; then
die "No .json files found in $src_dir (did you run --generate?)"
fi
mkdir -p "$dest_dir"
for src in "${src_files[@]}"; do
local name
name=$(basename "$src")
cp "$src" "$dest_dir/$name"
log_verbose " $src -> $dest_dir/$name"
done
}
sync_failsafe_web() {
local src_dir="build/px4_sitl_default_failsafe_web"
local dest_dir="docs/public/config/failsafe"
log "Syncing failsafe web..."
if [[ ! -d "$src_dir" ]]; then
die "Source directory not found: $src_dir (did you run --generate?)"
fi
# Gather js, wasm, json files
local src_files=()
for ext in js wasm json; do
src_files+=("$src_dir"/*."$ext")
done
if [[ ${#src_files[@]} -eq 0 ]]; then
die "No .js/.wasm/.json files found in $src_dir"
fi
mkdir -p "$dest_dir"
for src in "${src_files[@]}"; do
local name
name=$(basename "$src")
cp "$src" "$dest_dir/$name"
log_verbose " $src -> $dest_dir/$name"
done
}
# ═══════════════════════════════════════════════════════════════════════════════
# Main Logic
# ═══════════════════════════════════════════════════════════════════════════════
DO_GENERATE=false
DO_SYNC=false
SELECTED_TYPES=()
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--generate)
DO_GENERATE=true
shift
;;
--sync)
DO_SYNC=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
--help|-h)
show_help
;;
-*)
die "Unknown option: $1"
;;
*)
# It's a type
SELECTED_TYPES+=("$1")
shift
;;
esac
done
# Default to all types if none specified
if [[ ${#SELECTED_TYPES[@]} -eq 0 ]]; then
SELECTED_TYPES=("all")
fi
# Expand "all" to all types
local expanded_types=()
for t in "${SELECTED_TYPES[@]}"; do
if [[ "$t" == "all" ]]; then
expanded_types+=("${ALL_TYPES[@]}")
else
expanded_types+=("$t")
fi
done
SELECTED_TYPES=("${expanded_types[@]}")
# Validate types
for t in "${SELECTED_TYPES[@]}"; do
local valid=false
for valid_type in "${ALL_TYPES[@]}"; do
if [[ "$t" == "$valid_type" ]]; then
valid=true
break
fi
done
if [[ "$valid" == "false" ]]; then
die "Unknown type: $t (valid: ${ALL_TYPES[*]})"
fi
done
# Must specify at least one action
if [[ "$DO_GENERATE" == "false" && "$DO_SYNC" == "false" ]]; then
die "Must specify at least one of: --generate, --sync"
fi
}
main() {
parse_args "$@"
log "Selected types: ${SELECTED_TYPES[*]}"
[[ "$DO_GENERATE" == "true" ]] && log "Actions: generate"
[[ "$DO_SYNC" == "true" ]] && log "Actions: sync"
# Remove duplicates from SELECTED_TYPES
local -A seen
local unique_types=()
for t in "${SELECTED_TYPES[@]}"; do
if [[ -z "${seen[$t]:-}" ]]; then
seen[$t]=1
unique_types+=("$t")
fi
done
SELECTED_TYPES=("${unique_types[@]}")
# Generate phase
if [[ "$DO_GENERATE" == "true" ]]; then
log "=== Generation Phase ==="
for t in "${SELECTED_TYPES[@]}"; do
"generate_$t"
done
fi
# Sync phase
if [[ "$DO_SYNC" == "true" ]]; then
log "=== Sync Phase ==="
for t in "${SELECTED_TYPES[@]}"; do
"sync_$t"
done
fi
log "Done."
exit 0
}
main "$@"
+22 -35
View File
@@ -1,48 +1,37 @@
#!/bin/bash
mkdir artifacts
cp **/**/*.px4 artifacts/ 2>/dev/null || true
cp **/**/*.elf artifacts/ 2>/dev/null || true
cp **/**/*.deb artifacts/ 2>/dev/null || true
cp **/**/*.px4 artifacts/
cp **/**/*.elf artifacts/
for build_dir_path in build/*/ ; do
build_dir_path=${build_dir_path::${#build_dir_path}-1}
build_dir=${build_dir_path#*/}
mkdir -p artifacts/$build_dir
mkdir artifacts/$build_dir
find artifacts/ -maxdepth 1 -type f -name "*$build_dir*"
# Airframe (NuttX: build root, SITL: docs/ subdirectory)
airframes_src=""
if [ -f "$build_dir_path/airframes.xml" ]; then
airframes_src="$build_dir_path/airframes.xml"
elif [ -f "$build_dir_path/docs/airframes.xml" ]; then
airframes_src="$build_dir_path/docs/airframes.xml"
fi
if [ -n "$airframes_src" ]; then
cp "$airframes_src" "artifacts/$build_dir/"
fi
# Airframe
cp $build_dir_path/airframes.xml artifacts/$build_dir/
# Parameters
cp $build_dir_path/parameters.xml artifacts/$build_dir/ 2>/dev/null || true
cp $build_dir_path/parameters.json artifacts/$build_dir/ 2>/dev/null || true
cp $build_dir_path/parameters.json.xz artifacts/$build_dir/ 2>/dev/null || true
cp $build_dir_path/parameters.xml artifacts/$build_dir/
cp $build_dir_path/parameters.json artifacts/$build_dir/
cp $build_dir_path/parameters.json.xz artifacts/$build_dir/
# Actuators
cp $build_dir_path/actuators.json artifacts/$build_dir/ 2>/dev/null || true
cp $build_dir_path/actuators.json.xz artifacts/$build_dir/ 2>/dev/null || true
cp $build_dir_path/actuators.json artifacts/$build_dir/
cp $build_dir_path/actuators.json.xz artifacts/$build_dir/
# Events
mkdir -p artifacts/$build_dir/events/
cp $build_dir_path/events/all_events.json.xz artifacts/$build_dir/events/ 2>/dev/null || true
cp $build_dir_path/events/all_events.json.xz artifacts/$build_dir/
# ROS 2 msgs
cp $build_dir_path/events/all_events.json.xz artifacts/$build_dir/
# Module Docs
ls -la artifacts/$build_dir
echo "----------"
done
if [ -d artifacts/px4_sitl_default ]; then
# general metadata (used by Flight Review and other downstream consumers)
mkdir -p artifacts/_general/
# general metadata
mkdir artifacts/_general/
cp artifacts/px4_sitl_default/airframes.xml artifacts/_general/
# Airframe
if [ -f artifacts/px4_sitl_default/airframes.xml ]; then
cp artifacts/px4_sitl_default/airframes.xml artifacts/_general/
else
echo "Error: expected 'artifacts/px4_sitl_default/airframes.xml' not found." >&2
exit 1
fi
cp artifacts/px4_sitl_default/airframes.xml artifacts/_general/
# Parameters
cp artifacts/px4_sitl_default/parameters.xml artifacts/_general/
cp artifacts/px4_sitl_default/parameters.json artifacts/_general/
@@ -51,11 +40,9 @@ if [ -d artifacts/px4_sitl_default ]; then
cp artifacts/px4_sitl_default/actuators.json artifacts/_general/
cp artifacts/px4_sitl_default/actuators.json.xz artifacts/_general/
# Events
if [ -f artifacts/px4_sitl_default/events/all_events.json.xz ]; then
cp artifacts/px4_sitl_default/events/all_events.json.xz artifacts/_general/
else
echo "Error: expected 'artifacts/px4_sitl_default/events/all_events.json.xz' not found." >&2
exit 1
fi
cp artifacts/px4_sitl_default/events/all_events.json.xz artifacts/_general/
# ROS 2 msgs
cp artifacts/px4_sitl_default/events/all_events.json.xz artifacts/_general/
# Module Docs
ls -la artifacts/_general/
fi
-95
View File
@@ -1,95 +0,0 @@
#!/usr/bin/env python3
"""Post, update, or delete a PR comment with deduplication.
Uses hidden HTML markers to find existing comments and avoid duplicates.
Reads comment body from stdin when posting or updating.
Usage:
echo "comment body" | python3 pr_comment.py --marker pr-title --pr 123 --result fail
python3 pr_comment.py --marker pr-title --pr 123 --result pass
Results:
fail - post/update comment with body from stdin
warn - post/update comment with body from stdin
pass - delete existing comment if any
Requires GH_TOKEN and GITHUB_REPOSITORY environment variables.
"""
import argparse
import json
import os
import subprocess
import sys
def gh_api(endpoint: str, method: str = 'GET', body: dict | None = None) -> str:
"""Call the GitHub API via gh cli."""
cmd = ['gh', 'api', endpoint, '-X', method]
if body:
for key, value in body.items():
cmd.extend(['-f', f'{key}={value}'])
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0 and method != 'DELETE':
print(f"gh api error: {result.stderr}", file=sys.stderr)
return result.stdout
def find_comment(repo: str, pr: int, marker: str) -> str | None:
"""Find an existing comment by its hidden marker. Returns comment ID or None."""
response = gh_api(f"repos/{repo}/issues/{pr}/comments?per_page=100")
try:
comments = json.loads(response)
except json.JSONDecodeError:
return None
if not isinstance(comments, list):
return None
for comment in comments:
if isinstance(comment, dict) and comment.get('body', '').startswith(marker):
return str(comment['id'])
return None
def main() -> None:
parser = argparse.ArgumentParser(description='Manage PR quality comments')
parser.add_argument('--marker', required=True,
help='Marker name (e.g. pr-title, commit-msgs, pr-body)')
parser.add_argument('--pr', required=True, type=int,
help='Pull request number')
parser.add_argument('--result', required=True, choices=['pass', 'fail', 'warn'],
help='Check result: pass deletes comment, fail/warn posts it')
args = parser.parse_args()
repo = os.environ.get('GITHUB_REPOSITORY', '')
if not repo:
print("GITHUB_REPOSITORY not set", file=sys.stderr)
sys.exit(2)
marker = f"<!-- commit-quality-{args.marker} -->"
existing_id = find_comment(repo, args.pr, marker)
if args.result == 'pass':
if existing_id:
gh_api(f"repos/{repo}/issues/comments/{existing_id}", method='DELETE')
return
# Read comment body from stdin
body_content = sys.stdin.read().strip()
if not body_content:
print("No comment body provided on stdin", file=sys.stderr)
sys.exit(2)
full_body = f"{marker}\n{body_content}"
if existing_id:
gh_api(f"repos/{repo}/issues/comments/{existing_id}", method='PATCH',
body={'body': full_body})
else:
gh_api(f"repos/{repo}/issues/{args.pr}/comments", method='POST',
body={'body': full_body})
if __name__ == '__main__':
main()
-163
View File
@@ -1,163 +0,0 @@
#!/usr/bin/env bash
#
# test_metadata_sync.sh - Test metadata_sync.sh locally using Docker
#
# Usage:
# Tools/ci/test_metadata_sync.sh [OPTIONS] [TYPES...]
#
# Options:
# --shell Drop into interactive shell instead of running sync
# --verbose Pass --verbose to metadata_sync.sh
# --skip-build Skip SITL build (use existing build artifacts)
# --help Show this help
#
# Types:
# Same as metadata_sync.sh: parameters, airframes, modules, msg_docs, uorb_graphs, failsafe_web, all
#
# Examples:
# # Test full regeneration
# Tools/ci/test_metadata_sync.sh all
#
# # Test just parameters (faster)
# Tools/ci/test_metadata_sync.sh parameters
#
# # Drop into shell for debugging
# Tools/ci/test_metadata_sync.sh --shell
#
# # Skip build if you already have artifacts
# Tools/ci/test_metadata_sync.sh --skip-build --verbose all
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
DOCKER_IMAGE="px4io/px4-dev:v1.17.0-alpha1"
CONTAINER_NAME="px4-metadata-test-$$"
SHELL_MODE=false
VERBOSE=""
SKIP_BUILD=false
TYPES=()
show_help() {
head -n 28 "$0" | tail -n +2 | sed 's/^# \?//'
exit 0
}
cleanup() {
echo "[test] Cleaning up container..."
docker rm -f "$CONTAINER_NAME" 2>/dev/null || true
}
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--shell)
SHELL_MODE=true
shift
;;
--verbose)
VERBOSE="--verbose"
shift
;;
--skip-build)
SKIP_BUILD=true
shift
;;
--help|-h)
show_help
;;
-*)
echo "Unknown option: $1" >&2
exit 1
;;
*)
TYPES+=("$1")
shift
;;
esac
done
# Default to all types
if [[ ${#TYPES[@]} -eq 0 ]]; then
TYPES=("all")
fi
}
main() {
parse_args "$@"
cd "$REPO_ROOT"
echo "[test] Using Docker image: $DOCKER_IMAGE"
echo "[test] Repository root: $REPO_ROOT"
# Pull image if not present
if ! docker image inspect "$DOCKER_IMAGE" >/dev/null 2>&1; then
echo "[test] Pulling Docker image..."
docker pull "$DOCKER_IMAGE"
fi
trap cleanup EXIT
# Handle git worktrees: the .git file points to the main repo's .git directory
# We need to mount that directory too so git works inside the container
local git_mounts=()
if [[ -f "$REPO_ROOT/.git" ]]; then
# It's a worktree - read the gitdir path and mount it
local gitdir
gitdir=$(grep '^gitdir:' "$REPO_ROOT/.git" | cut -d' ' -f2)
if [[ -n "$gitdir" ]]; then
# Mount the gitdir at the same path so the .git file reference works
git_mounts+=("-v" "$gitdir:$gitdir:ro")
# Also need the main .git directory (parent of worktrees/)
local main_git_dir
main_git_dir=$(dirname "$(dirname "$gitdir")")
git_mounts+=("-v" "$main_git_dir:$main_git_dir:ro")
echo "[test] Detected git worktree, mounting git directories"
fi
fi
if [[ "$SHELL_MODE" == "true" ]]; then
echo "[test] Starting interactive shell..."
echo "[test] Run: Tools/ci/metadata_sync.sh --generate --sync all"
docker run -it --rm \
--name "$CONTAINER_NAME" \
-v "$REPO_ROOT:/src" \
"${git_mounts[@]}" \
-w /src \
"$DOCKER_IMAGE" \
/bin/bash
else
echo "[test] Running metadata sync for: ${TYPES[*]}"
# Build the command
local cmd=""
if [[ "$SKIP_BUILD" == "false" ]]; then
cmd="Tools/ci/metadata_sync.sh --generate --sync $VERBOSE ${TYPES[*]}"
else
cmd="Tools/ci/metadata_sync.sh --sync $VERBOSE ${TYPES[*]}"
fi
echo "[test] Command: $cmd"
docker run --rm \
--name "$CONTAINER_NAME" \
-v "$REPO_ROOT:/src" \
"${git_mounts[@]}" \
-w /src \
"$DOCKER_IMAGE" \
/bin/bash -c "$cmd"
echo ""
echo "[test] Done! Check git status for changes:"
echo " git status -s docs/"
echo ""
echo "[test] To see what changed:"
echo " git diff docs/"
fi
}
main "$@"
+24 -4
View File
@@ -1,27 +1,47 @@
#! /bin/bash
if [ -z ${PX4_DOCKER_REPO+x} ]; then
PX4_DOCKER_REPO="px4io/px4-dev:v1.17.0-beta1"
echo "guessing PX4_DOCKER_REPO based on input";
if [[ $@ =~ .*clang.* ]] || [[ $@ =~ .*scan-build.* ]]; then
# clang tools
PX4_DOCKER_REPO="px4io/px4-dev-clang:2021-02-04"
elif [[ $@ =~ .*tests* ]]; then
# run all tests with simulation
PX4_DOCKER_REPO="px4io/px4-dev-simulation-bionic:2021-12-11"
fi
else
echo "PX4_DOCKER_REPO is set to '$PX4_DOCKER_REPO'";
fi
# otherwise default to nuttx
if [ -z ${PX4_DOCKER_REPO+x} ]; then
PX4_DOCKER_REPO="px4io/px4-dev:v1.16.0-rc1-258-g0369abd556"
fi
echo "PX4_DOCKER_REPO: $PX4_DOCKER_REPO";
SCRIPT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
SRC_DIR=${SCRIPT_DIR}/../
PWD=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
SRC_DIR=$PWD/../
CCACHE_DIR=${HOME}/.ccache
mkdir -p "${CCACHE_DIR}"
docker run -it --rm -w "${SRC_DIR}" \
--user="$(id -u):$(id -g)" \
--env=AWS_ACCESS_KEY_ID \
--env=AWS_SECRET_ACCESS_KEY \
--env=BRANCH_NAME \
--env=CCACHE_DIR="${CCACHE_DIR}" \
--env=CI \
--env=CODECOV_TOKEN \
--env=COVERALLS_REPO_TOKEN \
--env=PX4_ASAN \
--env=PX4_MSAN \
--env=PX4_TSAN \
--env=PX4_UBSAN \
--env=TRAVIS_BRANCH \
--env=TRAVIS_BUILD_ID \
--publish 14556:14556/udp \
--volume=${CCACHE_DIR}:${CCACHE_DIR}:rw \
--volume=${SRC_DIR}:${SRC_DIR}:rw \
${PX4_DOCKER_REPO} /bin/bash -c "$@"
${PX4_DOCKER_REPO} /bin/bash -c "$1 $2 $3"
+3 -13
View File
@@ -128,9 +128,6 @@ class SourceParser:
# start waiting for the next part - long comment.
if state == "wait-short-end":
state = "wait-long"
elif state == "wait-long-end":
# Preserve paragraph breaks in long description
long_desc += "\n"
else:
m = self.re_comment_tag.match(comment_content)
if m:
@@ -211,7 +208,8 @@ class SourceParser:
raise Exception('short description too long (150 max, is {:}, parameter: {:})'.format(len(short_desc), name))
param.fields["short_desc"] = self.re_remove_dots.sub('', short_desc)
if long_desc is not None:
param.fields["long_desc"] = long_desc.rstrip('\n')
long_desc = self.re_remove_carriage_return.sub(' ', long_desc)
param.fields["long_desc"] = long_desc
for tag in tags:
if tag == "group":
group = tags[tag]
@@ -409,15 +407,7 @@ def generate_yaml(filename: str, groups: list[ParameterGroup]) -> str:
g["definitions"][parameter.name] = p
data["parameters"].append(g)
# Use block scalar style for multi-line strings
class BlockStyleDumper(yaml.SafeDumper):
pass
def str_representer(dumper, data):
if '\n' in data:
return dumper.represent_scalar('tag:yaml.org,2002:str', data, style='|')
return dumper.represent_scalar('tag:yaml.org,2002:str', data)
BlockStyleDumper.add_representer(str, str_representer)
return yaml.dump(data, Dumper=BlockStyleDumper, sort_keys=False)
return yaml.dump(data, sort_keys=False)
def main():
@@ -234,14 +234,14 @@ def get_actuator_output(yaml_config, output_functions, timer_config_file, verbos
param_prefix = process_param_prefix(group['param_prefix'])
standard_params = group.get('standard_params', {})
standard_params_array = [
( 'function', 'Function', 'FUNC', False, True ),
( 'disarmed', 'Disarmed', 'DIS', False, True ),
( 'min', 'Minimum', 'MIN', False, True ),
( 'center', 'Center\n(for Servos)', 'CENT', False, False ),
( 'max', 'Maximum', 'MAX', False, True ),
( 'failsafe', 'Failsafe', 'FAIL', True, True ),
( 'function', 'Function', 'FUNC', False ),
( 'disarmed', 'Disarmed', 'DIS', False ),
( 'min', 'Minimum', 'MIN', False ),
( 'max', 'Maximum', 'MAX', False ),
( 'center', 'Center\n(for Servos)', 'CENT', False ),
( 'failsafe', 'Failsafe', 'FAIL', True ),
]
for key, label, param_suffix, advanced, has_function in standard_params_array:
for key, label, param_suffix, advanced in standard_params_array:
show_if = None
if key in standard_params and 'show_if' in standard_params[key]:
show_if = standard_params[key]['show_if']
@@ -250,12 +250,13 @@ def get_actuator_output(yaml_config, output_functions, timer_config_file, verbos
param = {
'label': label,
'name': param_prefix+'_'+param_suffix+'${i}',
'function': key,
}
if has_function: param['function'] = key
if advanced: param['advanced'] = True
if show_if: param['show-if'] = show_if
per_channel_params.append(param)
param = {
'label': 'Rev Range\n(for Servos)',
'name': param_prefix+'_REV',
+6 -5
View File
@@ -108,7 +108,7 @@ def parse_yaml_parameters_config(yaml_config, ethernet_supported):
tags = '@group {:}'.format(param_group)
if param['type'] == 'enum':
param_type = 'INT32'
for key in sorted(param['values'], key=float):
for key in param['values']:
tags += '\n * @value {:} {:}'.format(key, param['values'][key])
elif param['type'] == 'bitmask':
param_type = 'INT32'
@@ -124,9 +124,6 @@ def parse_yaml_parameters_config(yaml_config, ethernet_supported):
param_type = 'INT32'
elif param['type'] == 'float':
param_type = 'FLOAT'
if 'values' in param:
for key in sorted(param['values'], key=float):
tags += '\n * @value {:} {:}'.format(key, param['values'][key])
else:
raise Exception("unknown param type {:}".format(param['type']))
@@ -315,7 +312,11 @@ When set to -1 (default), the value depends on the function (see {:}).
if standard_params[key]['max'] >= 1<<16:
raise Exception('maximum value for {:} expected <= {:} (got {:})'.format(key, 1<<16, standard_params[key]['max']))
if key == 'failsafe' or key == 'center':
if key == 'failsafe':
standard_params[key]['default'] = -1
standard_params[key]['min'] = -1
if key == 'center':
standard_params[key]['default'] = -1
standard_params[key]['min'] = -1
+68 -833
View File
@@ -8,805 +8,6 @@ Also generates docs/en/middleware/dds_topics.md from dds_topics.yaml
import os
import argparse
import sys
import re
VALID_FIELDS = { #Note, also have to add the message types as those can be fields
'uint64',
'uint16',
'uint8',
'uint32'
}
ALLOWED_UNITS = set(["m", "m/s", "m/s^2", "(m/s)^2", "deg", "deg/s", "rad", "rad/s", "rad^2", "rpm" ,"V", "A", "mA", "mAh", "W", "Wh", "dBm", "h", "minutes", "s", "ms", "us", "Ohm", "MB", "Kb/s", "degC","Pa", "%", "norm", "-"])
invalid_units = set()
ALLOWED_FRAMES = set(["NED", "Body", "FRD", "ENU"])
ALLOWED_INVALID_VALUES = set(["NaN", "0", "-1"])
ALLOWED_CONSTANTS_NOT_IN_ENUM = set(["ORB_QUEUE_LENGTH","MESSAGE_VERSION"])
class Error:
def __init__(self, type, message, linenumber=None, issueString = None, field = None):
self.type = type
self.message = message
self.linenumber = linenumber
self.issueString = issueString
self.field = field
def display_error(self):
#print(f"Debug: Error: display_error")
if 'trailing_whitespace' == self.type:
if self.issueString.strip():
print(f"NOTE: Line has trailing whitespace ({self.message}: {self.linenumber}): {self.issueString}")
else:
print(f"NOTE: Line has trailing whitespace ({self.message}: {self.linenumber})")
elif 'leading_whitespace_field_or_constant' == self.type:
print(f"NOTE: Whitespace before field or constant ({self.message}: {self.linenumber}): {self.issueString}")
elif 'field_or_constant_has_multiple_whitepsace' == self.type:
print(f"NOTE: Field/constant has more than one sequential whitespace character ({self.message}: {self.linenumber}): {self.issueString}")
elif 'empty_start_line' == self.type:
print(f"NOTE: Empty line at start of file ({self.message}: {self.linenumber})")
elif 'internal_comment' == self.type:
print(f"NOTE: Internal Comment ({self.message}: {self.linenumber})\n {self.issueString}")
elif 'internal_comment_empty' == self.type:
print(f"NOTE: Empty Internal Comment ({self.message}: {self.linenumber})")
elif 'summary_missing' == self.type:
print(f"WARNING: No message description ({self.message})")
elif 'topic_error' == self.type:
print(f"NOTE: TOPIC ISSUE: {self.issueString}")
elif 'unknown_unit' == self.type:
print(f"WARNING: Unknown Unit: [{self.issueString}] on `{self.field}` ({self.message}: {self.linenumber})")
elif 'constant_not_in_assigned_enum' == self.type:
print(f"WARNING: `{self.issueString}` constant: Prefix not in `@enum` field metadata ({self.message}: {self.linenumber})")
elif 'unknown_invalid_value' == self.type:
print(f"WARNING: Unknown @invalid value: [{self.issueString}] on `{self.field}` ({self.message}: {self.linenumber})")
elif 'unknown_frame' == self.type:
print(f"WARNING: Unknown @frame: [{self.issueString}] on `{self.field}` ({self.message}: {self.linenumber})")
elif 'command_no_params_pipes' == self.type:
print(f"WARNING: `{self.field}` command has no parameters (pipes): [{self.issueString}] ({self.message}: {self.linenumber})")
elif 'command_missing_params' == self.type:
print(f"WARNING: `{self.field}` command missing params - should be 7 params surrounded by 8 pipes: [{self.issueString}] ({self.message}: {self.linenumber})")
elif 'command_too_many_params' == self.type:
print(f"WARNING: `{self.field}` command too many params (should be 7). Extras: [{self.issueString}] ({self.message}: {self.linenumber})")
else:
self.display_info()
def display_info(self):
"""
Display info about an error.
Used as a fallback if error does not have specific printout in display_error()
"""
#print(f"Debug: Error: display_info")
print(f" type: {self.type}, message: {self.message}, linenumber: {self.linenumber}, issueString: {self.issueString}, field: {self.field}")
class Enum:
def __init__(self, name, parentMessage):
self.name = name
self.parent = parentMessage
self.enumValues = dict()
def display_info(self):
"""
Display info about an enum
"""
print(f"Debug: Enum: display_info")
print(f" name: {self.name}")
for key, value in self.enumValues.items():
value.display_info()
class ConstantValue:
def __init__(self, name, type, value, comment, line_number):
self.name = name.strip()
self.type = type.strip()
self.value = value.strip()
self.comment = comment
self.line_number = line_number
if not self.value:
print(f"Debug WARNING: NO VALUE in ConstantValue: {self.name}") ## TODO make into ERROR
exit()
# TODO if value or name are empty, error
def display_info(self):
print(f"Debug: ConstantValue: display_info")
print(f" name: {self.name}, type: {self.type}, value: {self.value}, comment: {self.comment}, line: {self.line_number}")
class CommandParam:
"""
Represents an individual param in a command constant
Encapsulates parsing of the param to extract units etc.
"""
def __init__(self, num, paramText, line_number, parentCommand):
self.paramNum = num
self.paramText = paramText.strip()
self.enum = None
self.range = None
#self.type = type
self.units = []
self.enums = []
self.minValue = None
self.maxValue = None
self.invalidValue = None
self.frameValue = None
self.lineNumber = line_number
self.parent = parentCommand
self.parentMessage = self.parent.parent
match = None
if self.paramText:
match = re.match(r'^((?:\[[^\]]*\]\s*)+)(.*)$', paramText)
self.description = paramText
bracketed_part = None
if match:
bracketed_part = match.group(1).strip() # .strip() removes trailing whitespace from the bracketed part
self.description = match.group(2).strip()
if bracketed_part:
# get units
bracket_content_matches = re.findall(r'\[(.*?)\]', bracketed_part)
#print(f"DEBUG: bracket_content_matches: {bracket_content_matches}")
for item in bracket_content_matches:
item = item.strip()
if item.startswith('@'): # Not a unit:
if item.startswith('@enum'):
item = item.split(" ")
enum = item[1].strip()
if enum and enum not in self.enums:
self.enums.append(enum)
# Create parent enum objects for any enums created in this step
for enumName in self.enums:
if not enumName in self.parentMessage.enums:
self.parentMessage.enums[enumName]=Enum(enumName,self.parentMessage)
elif item.startswith('@range'):
item = item[6:].strip().split(",")
self.range = item
self.minValue = item[0].strip()
self.maxValue = item[1].strip()
elif item.startswith('@invalid'):
self.invalidValue = item[8:].strip()
#TODO: Do we require a description? (not currently)
if self.invalidValue.split(" ")[0] not in ALLOWED_INVALID_VALUES:
print(f"TODO: Command param do not support @invalid: {self.invalidValue}")
"""
error = Error("unknown_invalid_value", self.parent.filename, self.lineNumber, self.invalidValue, self.name)
#error.display_error()
if not "unknown_invalid_value" in self.parent.errors:
self.parent.errors["unknown_invalid_value"] = []
self.parent.errors["unknown_invalid_value"].append(error)
"""
elif item.startswith('@frame'):
self.frameValue = item[6:].strip()
print(f"TODO: Command param do not support @frame: {self.frameValue}")
"""
if self.frameValue not in ALLOWED_FRAMES:
error = Error("unknown_frame", self.parent.filename, self.lineNumber, self.frameValue, self.name)
#error.display_error()
if not "unknown_frame" in self.parent.errors:
self.parent.errors["unknown_frame"] = []
self.parent.errors["unknown_frame"].append(error)
"""
else:
print(f"WARNING: Unhandled metadata in message comment: {item}")
# TODO - report errors for different kinds of metadata
exit()
else: # bracket is a unit
unit = item.strip()
if item == "-":
unit = ""
if unit and unit not in self.units:
self.units.append(unit)
if unit not in ALLOWED_UNITS:
invalid_units.add(unit)
error = Error("unknown_unit", self.parentMessage.filename, self.lineNumber, unit, self.parent.name)
#error.display_error()
if not "unknown_unit" in self.parentMessage.errors:
self.parentMessage.errors["unknown_unit"] = []
self.parentMessage.errors["unknown_unit"].append(error)
def display_info(self):
print(f"Debug: CommandParam: display_info")
print(f" id: {self.paramNum}")
print(f" paramText: {self.paramText}\n unit: {self.units}\n enums: {self.enums}\n lineNumber: {self.lineNumber}\n range: {self.range}\n minValue: {self.minValue}\n maxValue: {self.maxValue}\n invalidValue: {self.invalidValue}\n frameValue: {self.frameValue}\n parent: {self.parent}\n ")
class CommandConstant:
"""
Represents a constant that is a command definition.
Encapsulates parsing of the command format.
The individual params are further parsed in CommandParam
"""
def __init__(self, name, type, value, comment, line_number, parentMessage):
self.name = name.strip()
self.type = type.strip()
self.value = value.strip()
self.comment = comment
self.line_number = line_number
self.parent = parentMessage
self.description = self.comment
self.param1 = None
self.param2 = None
self.param3 = None
self.param4 = None
self.param5 = None
self.param6 = None
self.param7 = None
if not self.value:
print(f"Debug WARNING: NO VALUE in CommandConstant: {self.name}") ## TODO make into ERROR
exit()
if not self.comment: # This is an bug for a command
#print(f"Debug WARNING: NO COMMENT in CommandConstant: {self.name}") ## TODO make into ERROR
return
# Parse command comment to get the description and parameters.
# print(f"Debug CommandConstant: {self.comment}")
if not "|" in self.comment:
# This is an error for a command constant
error = Error("command_no_params_pipes", self.parent.filename, self.line_number, self.comment, self.name)
#error.display_error()
if not "command_no_params_pipes" in self.parent.errors:
self.parent.errors["command_no_params_pipes"] = []
self.parent.errors["command_no_params_pipes"].append(error)
return
# Split on pipes
commandSplit = self.comment.split("|")
if len(commandSplit) < 9:
# Should 7 pipes, so each command is fully surrounded
error = Error("command_missing_params", self.parent.filename, self.line_number, self.comment, self.name)
#error.display_error()
if not "command_missing_params" in self.parent.errors:
self.parent.errors["command_missing_params"] = []
self.parent.errors["command_missing_params"].append(error)
self.description = commandSplit[0].strip()
self.description = self.description if self.description else None
params_to_update = commandSplit[1:8]
for i, value in enumerate(params_to_update, start=1):
if value.strip():
# parse the param
param = CommandParam(i, value, self.line_number, self)
#param.display_info() # DEBUG CODE XXX
setattr(self, f"param{i}", param)
# parse the param
if len(commandSplit) > 8:
extras = commandSplit[8:]
error = Error("command_too_many_params", self.parent.filename, self.line_number, extras, self.name)
if not "command_too_many_params" in self.parent.errors:
self.parent.errors["command_too_many_params"] = []
self.parent.errors["command_too_many_params"].append(error)
# TODO if value or name are empty, error
def markdown_out(self):
#print("DEBUG: CommandConstant.markdown_out")
output = f"""### {self.name} ({self.value})
{self.description}
Param | Units | Range/Enum | Description
--- | --- | --- | ---
"""
for i in range(1, 8):
attr_name = f"param{i}"
# getattr returns None if the attribute doesn't exist
val = getattr(self, attr_name, None)
if val is not None:
rangeVal = ""
if val.minValue or val.maxValue:
rangeVal = f"[{val.minValue if val.minValue else '-'} : {val.maxValue if val.maxValue else '-' }]"
units_str = ", ".join(val.units)
enums_str = ', '.join("[{}](#{})".format(e, e) for e in val.enums)
output+=f"{i} | {units_str}|{enums_str}{rangeVal} | {val.description}\n"
else:
output+=f"{i} | | | ?\n"
output+=f"\n"
return output
def display_info(self):
print(f"Debug: CommandConstant: display_info")
print(f" name: {self.name}, type: {self.type}, value: {self.value}, comment: {self.comment}, line: {self.line_number}")
print(f" description: {self.description}\n param1: {self.param1}\n param2: {self.param2}\n param3: {self.param3}\n param4: {self.param4}\n param5: {self.param5}\n param6: {self.param6}\n param7: {self.param7}")
class MessageField:
"""
Represents a field.
Encapsulates parsing of the field information.
"""
def __init__(self, name, type, comment, line_number, parentMessage):
self.name = name
self.type = type
self.comment = comment
self.unit = None
self.enums = None
self.minValue = None
self.maxValue = None
self.invalidValue = None
self.frameValue = None
self.lineNumber = line_number
self.parent = parentMessage
#print(f"MessageComment: {comment}")
match = None
if self.comment:
match = re.match(r'^((?:\[[^\]]*\]\s*)+)(.*)$', comment)
self.description = comment
bracketed_part = None
if match:
bracketed_part = match.group(1).strip() # .strip() removes trailing whitespace from the bracketed part
self.description = match.group(2).strip()
if bracketed_part:
# get units
bracket_content_matches = re.findall(r'\[(.*?)\]', bracketed_part)
#print(f"bracket_content_matches: {bracket_content_matches}")
for item in bracket_content_matches:
item = item.strip()
if item.startswith('@'): # Not a unit:
if item.startswith('@enum'):
item = item.split(" ")
self.enums = item[1:]
# Create parent enum objects
for enumName in self.enums:
if not enumName in parentMessage.enums:
parentMessage.enums[enumName]=Enum(enumName,parentMessage)
elif item.startswith('@range'):
item = item[6:].strip().split(",")
self.minValue = item[0].strip()
self.maxValue = item[1].strip()
elif item.startswith('@invalid'):
self.invalidValue = item[8:].strip()
#TODO: Do we require a description? (not currently)
if self.invalidValue.split(" ")[0] not in ALLOWED_INVALID_VALUES:
error = Error("unknown_invalid_value", self.parent.filename, self.lineNumber, self.invalidValue, self.name)
#error.display_error()
if not "unknown_invalid_value" in self.parent.errors:
self.parent.errors["unknown_invalid_value"] = []
self.parent.errors["unknown_invalid_value"].append(error)
elif item.startswith('@frame'):
self.frameValue = item[6:].strip()
if self.frameValue not in ALLOWED_FRAMES:
error = Error("unknown_frame", self.parent.filename, self.lineNumber, self.frameValue, self.name)
#error.display_error()
if not "unknown_frame" in self.parent.errors:
self.parent.errors["unknown_frame"] = []
self.parent.errors["unknown_frame"].append(error)
else:
print(f"WARNING: Unhandled metadata in message comment: {item}")
# TODO - report errors for different kinds of metadata
exit()
else: # bracket is a unit
self.unit = item
if self.unit not in ALLOWED_UNITS:
invalid_units.add(self.unit)
error = Error("unknown_unit", self.parent.filename, self.lineNumber, self.unit, self.name)
#error.display_error()
if not "unknown_unit" in self.parent.errors:
self.parent.errors["unknown_unit"] = []
self.parent.errors["unknown_unit"].append(error)
if item == "-":
self.unit = ""
def display_info(self):
print(f"Debug: MessageField: display_info")
print(f" name: {self.name}, type: {self.type}, description: {self.description}, enums: {self.enums}, minValue: {self.minValue}, maxValue: {self.maxValue}, invalidValue: {self.invalidValue}, frameValue: {self.frameValue}")
class UORBMessage:
"""
Represents a whole message, including fields, enums, commands, constants.
The parser function delegates the parsing of each part of the message to
more appropriate classes, once the specific type of line has been identified.
"""
def __init__(self, filename):
self.filename = filename
msg_path = os.path.join(os.path.dirname(os.path.realpath(__file__)),"../../msg")
self.msg_filename = os.path.join(msg_path, self.filename)
self.name = os.path.splitext(os.path.basename(msg_file))[0]
self.shortDescription = ""
self.longDescription = ""
self.fields = []
self.constantFields = dict()
self.commandConstants = dict()
self.enums = dict()
self.output_file = os.path.join(output_dir, f"{self.name}.md")
self.topics = []
self.errors = dict()
self.parseFile()
if args.errors:
#print(f"DEBUG: args.errors: {args.errors}")
if args.error_messages:
messages = args.error_messages.split(" ")
#print(f"DEBUG: args.errors: {messages},self.name: {self.name}")
if self.name in messages:
self.reportErrors()
#print(f"Debug: {self.name} in {messages}")
else:
self.reportErrors()
def reportErrors(self):
#print(f"Debug: UORBMessage: reportErrors()")
for errorType, errors in self.errors.items():
for error in errors:
error.display_error()
def markdown_out(self):
#print(f"Debug: UORBMessage: markdown_out()")
# Add page header (forces wide pages)
markdown = f"""---
pageClass: is-wide-page
---
# {self.name} (UORB message)
"""
## Append description info if present
markdown += f"{self.shortDescription}\n\n" if self.shortDescription else ""
markdown += f"{self.longDescription}\n\n" if self.longDescription else ""
topicList = " ".join(self.topics)
markdown += f"**TOPICS:** {topicList}\n\n"
# Generate field docs
markdown += f"## Fields\n\n"
markdown += "Name | Type | Unit [Frame] | Range/Enum | Description\n"
markdown += "--- | --- | --- | --- | ---\n"
for field in self.fields:
unit = f"{field.unit}" if field.unit else ""
frame = f"[{field.frameValue}]" if field.frameValue else ""
unit = f"{unit} {frame}"
unit.strip()
unit = f" {unit}"
value = " "
if field.enums:
value = ""
for enum in field.enums:
value += f"[{enum}](#{enum})"
value = value.strip()
value = f"{value}"
elif field.minValue or field.maxValue:
value = f"[{field.minValue if field.minValue else '-'} : {field.maxValue if field.maxValue else '-' }]"
description = f" {field.description}" if field.description else ""
invalid = f" (Invalid: {field.invalidValue}) " if field.invalidValue else ""
markdown += f"{field.name} | `{field.type}` |{unit}|{value}|{description}{invalid}\n"
# Generate table for command docs
if len(self.commandConstants) > 0:
#print("DEBUGCOMMAND")
markdown += f"\n## Commands\n\n"
"""
markdown += "Name | Type | Value | Description\n"
markdown += "--- | --- | --- |---\n"
for name, command in self.commandConstants.items():
description = f" {command.comment} " if enum.comment else " "
markdown += f'<a id="#{name}"></a> {name} | `{command.type}` | {command.value} |{description}\n'
"""
for commandConstant in self.commandConstants.values():
#print(commandConstant)
markdown += commandConstant.markdown_out()
# Generate enum docs
if len(self.enums) > 0:
markdown += f"\n## Enums\n"
for name, enum in self.enums.items():
markdown += f"\n### {name} {{#{name}}}\n\n"
markdown += "Name | Type | Value | Description\n"
markdown += "--- | --- | --- | ---\n"
for enumValueName, enumValue in enum.enumValues.items():
description = f" {enumValue.comment} " if enumValue.comment else " "
markdown += f'<a id="#{enumValueName}"></a> {enumValueName} | `{enumValue.type}` | {enumValue.value} |{description}\n'
# Generate table for constants docs
if len(self.constantFields) > 0:
markdown += f"\n## Constants\n\n"
markdown += "Name | Type | Value | Description\n"
markdown += "--- | --- | --- |---\n"
for name, enum in self.constantFields.items():
description = f" {enum.comment} " if enum.comment else " "
markdown += f'<a id="#{name}"></a> {name} | `{enum.type}` | {enum.value} |{description}\n'
# Append msg contents to the end
with open(self.msg_filename, 'r') as source_file:
msg_contents = source_file.read()
msg_contents = msg_contents.strip()
#Format markdown using msg name, comment, url, contents.
markdown += f"""
## Source Message
[Source file (GitHub)](https://github.com/PX4/PX4-Autopilot/blob/main/msg/{self.filename})
::: details Click here to see original file
```c
{msg_contents}
```
:::
"""
with open(self.output_file, 'w') as content_file:
content_file.write(markdown)
#exit()
def display_info(self):
print(f"UORBMessage: display_info")
print(f" name: {self.name}")
print(f" filename: {self.filename}, ")
print(f" msg_filename: {self.msg_filename}, ")
print(f"self.shortDescription: {self.shortDescription}")
print(f"self.longDescription: {self.longDescription}")
print(f"self.enums: {self.enums}")
for enum, enumObject in self.enums.items():
enumObject.display_info()
# Output our data so far
for field in self.fields:
field.display_info()
for enumvalue in self.constantFields:
print(enumvalue)
self.constantFields[enumvalue].display_info()
def handleField(self, line, line_number, parentMessage):
#print(f"debug: handleField: (line): \n {line}")
# Note, here we know we don't have a comment or a topic.
# We expect it to be a field.
# Check field doesn't have leading whitespace (trailing spaces already checked)
if line[:1].isspace(): # Returns True for ' ', '\t', '\n', '\r', etc.
#print("First character is whitespace")
error = Error("leading_whitespace_field_or_constant", self.filename, line_number, line)
if not "leading_whitespace_field_or_constant" in self.errors:
self.errors["leading_whitespace_field_or_constant"] = []
self.errors["leading_whitespace_field_or_constant"].append(error)
# Now we can parse the stripped line
fieldOrConstant = line.strip()
# Check that the field or constant has only single whitespace separators
stripped_fieldOrConstant = re.sub(r'\s+', ' ', fieldOrConstant) # Collapse all spaces to a single space (LHS already stripped).
if stripped_fieldOrConstant != fieldOrConstant:
#print("Field/Constant has multiple whitespace characters") # Since the collapsed version shows them.
error = Error("field_or_constant_has_multiple_whitepsace", self.filename, line_number, line)
if not "field_or_constant_has_multiple_whitepsace" in self.errors:
self.errors["field_or_constant_has_multiple_whitepsace"] = []
self.errors["field_or_constant_has_multiple_whitepsace"].append(error)
fieldOrConstant = stripped_fieldOrConstant
comment = None
if "#" in line:
commentExtract = line.split("#", 1) # Split once on left-most '#'
fieldOrConstant = commentExtract[0].strip()
comment = commentExtract[-1].strip()
if "=" not in fieldOrConstant:
# Is a field:
field = fieldOrConstant.split(" ")
type = field[0].strip()
name = field[1].strip()
field = MessageField(name, type, comment, line_number, parentMessage)
self.fields.append(field)
else:
temp = fieldOrConstant.split("=")
value = temp[-1]
typeAndName = temp[0].split(" ")
type = typeAndName[0].strip()
name = typeAndName[1].strip()
if name.startswith("VEHICLE_CMD_") and parentMessage.name == 'VehicleCommand': #it's a command.
#print(f"DEBUG: startswith VEHICLE_CMD_ {name}")
commandConstant = CommandConstant(name, type, value, comment, line_number, parentMessage)
#commandConstant.display_info()
self.commandConstants[name]=commandConstant
else: #it's a constant (or part of an enum)
constantField = ConstantValue(name, type, value, comment, line_number)
self.constantFields[name]=constantField
def parseFile(self):
initial_block_lines = []
#stopping_token = None
found_first_relevant_content = False
gettingInitialComments = False
gettingFields = False
with open(self.msg_filename, 'r', encoding='utf-8') as uorbfile:
lines = uorbfile.read().splitlines()
for line_number, line in enumerate(lines, 1):
if line != line.rstrip():
#print(f"[{self.filename}] Trailing whitespace on line {line_number}: XX{line}YY")
error = Error("trailing_whitespace", self.filename, line_number, line)
if not "trailing_whitespace" in self.errors:
self.errors["trailing_whitespace"] = []
self.errors["trailing_whitespace"].append(error)
#print(f"line: {line}")
stripped_line = re.sub(r'\s+', ' ', line).strip() # Collapse all spaces to a single space and strip stuff off end.
#print(f"stripped_line: {stripped_line}")
# TODO? Perhaps report whitespace if the size of those two is different and it is empty
# Or perhaps we just fix it on request
isEmptyLine = False if line.strip() else True
if not found_first_relevant_content and isEmptyLine: #Empty line
#print(f"{self.filename}: Empty line at start of file: [{line_number}]\n {line}")
error = Error("empty_start_line", self.filename, line_number, line)
if not "empty_start_line" in self.errors:
self.errors["empty_start_line"] = []
self.errors["empty_start_line"].append(error)
#error.display_error()
continue
if not found_first_relevant_content and not isEmptyLine:
found_first_relevant_content = True
if stripped_line.startswith("#"):
gettingInitialComments = True
else:
gettingInitialComments = False
gettingFields = True
if gettingInitialComments and stripped_line.startswith("#"):
stripped_line=stripped_line[1:].strip()
#print(f"DEBUG: gettingInitialComments: comment line: {stripped_line}")
initial_block_lines.append(stripped_line)
else:
gettingInitialComments = False
gettingFields = True #Getting fields and constants
if gettingFields:
if isEmptyLine:
continue # empty line
if stripped_line.startswith("# TOPICS "):
stripped_line = stripped_line[9:]
stripped_line = stripped_line.split(" ")
self.topics+= stripped_line
# Note, default topic and topic errors handled after all lines parsed
continue
if stripped_line.startswith("#"):
# Its an internal comment
stripped_line=stripped_line[1:].strip()
if stripped_line:
#print(f"{self.filename}: Internal comment: [{line_number}]\n {line}")
error = Error("internal_comment", self.filename, line_number, line)
if not "internal_comment" in self.errors:
self.errors["internal_comment"] = []
self.errors["internal_comment"].append(error)
else:
#print(f"{self.filename}: Empty internal comment: [{line_number}]\n {line}")
error = Error("internal_comment_empty", self.filename, line_number, line)
if not "internal_comment_empty" in self.errors:
self.errors["internal_comment_empty"] = []
self.errors["internal_comment_empty"].append(error)
#pass # Empty comment
continue
# Must be a field or a comment.
self.handleField(line, line_number, parentMessage=self)
# Fix up topics if the topic is empty
def camel_to_snake(name):
# Insert underscore between lowercase/digit and uppercase letter
s1 = re.sub('([a-z0-9])([A-Z])', r'\1_\2', name)
# Insert underscore between consecutive uppercase and uppercase+lowercase
return re.sub('([A-Z]+)([A-Z][a-z])', r'\1_\2', s1).lower()
defaultTopic = camel_to_snake(self.name)
if len(self.topics) == 0:
# We have no topic declared, so set the default topic
self.topics.append(defaultTopic)
elif len(self.topics) == 1:
# We have 1 topic declared - either it is default or there is some issue.
if defaultTopic in self.topics:
# Declared topic is default topic
error = Error("topic_error", self.filename, "", f"WARNING: TOPIC {defaultTopic} unnecessarily declared for {self.name}")
else:
# Declared topic is not default topic
error = Error("topic_error", self.filename, "", f"NOTE: TOPIC {self.topics[0]}: Only Declared topic is not default topic {defaultTopic} for {self.name}")
if not "topic_error" in self.errors:
self.errors["topic_error"] = []
self.errors["topic_error"].append(error)
elif len(self.topics) > 1:
if defaultTopic not in self.topics:
error = Error("topic_error", self.filename, "", f"NOTE: TOPIC - Default topic {defaultTopic} for {self.name} not in {self.topics}")
# Parse our short and long description
#print(f"DEBUG: initial_block_lines: {initial_block_lines}")
doingLongDescription = False
for summaryline in initial_block_lines:
if not self.shortDescription and summaryline.strip() == '':
continue
if not doingLongDescription and not summaryline.strip() == '':
self.shortDescription += f" {summaryline}"
self.shortDescription = self.shortDescription.strip()
if not self.shortDescription[-1:] == ".": # Add terminating fullstop if not present.
self.shortDescription += "."
if not doingLongDescription and summaryline.strip() == '':
doingLongDescription = True
continue
if doingLongDescription:
self.longDescription += f"{summaryline}\n"
if self.longDescription:
self.longDescription.strip()
if not self.shortDescription:
# Summary has not been defined
error = Error("summary_missing", self.filename)
if not "summary_missing" in self.errors:
self.errors["summary_missing"] = []
self.errors["summary_missing"].append(error)
# TODO Parse our constantValues into enums, leaving only constants
constantValuesToRemove = []
#print(f"DEBUG: Self.enums: {self.enums}")
for enumName, enumObject in self.enums.items():
for enumValueName, enumValueObject in self.constantFields.items():
if enumValueName.startswith(enumName):
# Copy this value into the object (cant be duplicate because parent is dict)
enumObject.enumValues[enumValueName]=enumValueObject
constantValuesToRemove.append(enumValueName)
# Now delete the original enumvalues
for enumValName in constantValuesToRemove:
del self.constantFields[enumValName]
constantsNotAssignedToEnums = len(self.constantFields)
if constantsNotAssignedToEnums > 0:
#print(f"Debug: WARNING constantsNotAssignedToEnums: {constantsNotAssignedToEnums}")
for enumValueName, enumValue in self.constantFields.items():
if enumValueName in ALLOWED_CONSTANTS_NOT_IN_ENUM: # Ignore constants
pass
else:
error = Error("constant_not_in_assigned_enum", self.filename, enumValue.line_number, enumValueName)
if not "constant_not_in_assigned_enum" in self.errors:
self.errors["constant_not_in_assigned_enum"] = []
self.errors["constant_not_in_assigned_enum"].append(error)
# TODO Maybe present as list of possible enums.
import yaml
@@ -835,9 +36,11 @@ def generate_dds_yaml_doc(allMessageFiles, output_file = 'dds_topics.md'):
for message in data["subscriptions"]:
all_message_types.add(message['type'].split("::")[-1])
all_topics.add(message['topic'].split('/')[-1])
for message in (data.get("subscriptions_multi") or []):
all_message_types.add(message['type'].split("::")[-1])
all_topics.add(message['topic'].split('/')[-1])
if data["subscriptions_multi"]: # There is none now
dds_markdown += "None\n"
for message in data["subscriptions_multi"]:
all_message_types.add(message['type'].split("::")[-1])
all_topics.add(message['topic'].split('/')[-1])
for message in allMessageFiles:
all_messages_in_source.add(message.split('/')[-1].split('.')[0])
messagesNotExported = all_messages_in_source - all_message_types
@@ -874,17 +77,13 @@ Topic | Type| Rate Limit
dds_markdown += "\n## Subscriptions Multi\n\n"
subscriptions_multi = data.get("subscriptions_multi") or []
if not subscriptions_multi:
if not data["subscriptions_multi"]: # There is none now
dds_markdown += "None\n"
else:
dds_markdown += "Topic | Type | Route Field | Max Instances\n--- | --- | --- | ---\n"
for message in subscriptions_multi:
type = message['type']
px4Type = type.split("::")[-1]
route_field = f"`{message['route_field']}`" if 'route_field' in message else "-"
max_instances = message.get('max_instances', '-')
dds_markdown += f"{message['topic']} | [{type}](../msg_docs/{px4Type}.md) | {route_field} | {max_instances}\n"
print("Warning - we now have subscription_multi data - check format")
dds_markdown += "Topic | Type\n--- | ---\n"
for message in data["subscriptions_multi"]:
dds_markdown += f"{message['topic']} | {message['type']}\n"
if messagesNotExported:
# Print the topics that are not exported to DDS
@@ -928,47 +127,83 @@ if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Generate docs from .msg files')
parser.add_argument('-d', dest='dir', help='output directory', required=True)
parser.add_argument('-e', dest='errors', action='store_true', help='Report errors')
parser.add_argument('-m', dest='error_messages', help='Message to report errors against (by default all)')
args = parser.parse_args()
output_dir = args.dir
if not os.path.isdir(output_dir):
print(f"making output_dir {output_dir}")
os.mkdir(output_dir)
msg_path = os.path.join(os.path.dirname(os.path.realpath(__file__)),"../../msg")
msg_files = get_msgs_list(msg_path)
msg_files.sort()
versioned_msgs_list = ''
unversioned_msgs_list = ''
msgTypes = set()
for msg_file in msg_files:
# Add messages to set of allowed types (compound types)
msg_name = os.path.splitext(os.path.basename(msg_file))[0]
msgTypes.add(msg_name)
output_file = os.path.join(output_dir, msg_name+'.md')
msg_filename = os.path.join(msg_path, msg_file)
print("{:} -> {:}".format(msg_filename, output_file))
for msg_file in msg_files:
message = UORBMessage(msg_file)
# Any additional tests that can't be in UORBMessage parser go here.
message.markdown_out()
#Format msg url
msg_url="[source file](https://github.com/PX4/PX4-Autopilot/blob/main/msg/%s)" % msg_file
msg_description = ""
summary_description = ""
#Get msg description (first non-empty comment line from top of msg)
with open(msg_filename, 'r') as lineparser:
line = lineparser.readline()
while line.startswith('#') or (line.strip() == ''):
print('DEBUG: line: %s' % line)
line=line[1:].strip()+'\n'
stripped_line=line.strip()
if msg_description and not summary_description and stripped_line=='':
summary_description = msg_description.strip()
msg_description+=line
line = lineparser.readline()
msg_description=msg_description.strip()
if not summary_description and msg_description:
summary_description = msg_description
print('msg_description: Z%sZ' % msg_description)
print('summary_description: Z%sZ' % summary_description)
summary_description
msg_contents = ""
#Get msg contents (read the file)
with open(msg_filename, 'r') as source_file:
msg_contents = source_file.read()
#Format markdown using msg name, comment, url, contents.
markdown_output="""# %s (UORB message)
%s
%s
```c
%s
```
""" % (msg_name, msg_description, msg_url, msg_contents)
with open(output_file, 'w') as content_file:
content_file.write(markdown_output)
# Categorize as versioned or unversioned
if "versioned" in msg_file:
versioned_msgs_list += f"- [{message.name}]({message.name}.md)"
if message.shortDescription:
versioned_msgs_list += f"{message.shortDescription}"
versioned_msgs_list += '- [%s](%s.md)' % (msg_name, msg_name)
if summary_description:
versioned_msgs_list += "%s" % summary_description
versioned_msgs_list += "\n"
else:
unversioned_msgs_list += f"- [{message.name}]({message.name}.md)"
if message.shortDescription:
unversioned_msgs_list += f"{message.shortDescription}"
unversioned_msgs_list += '- [%s](%s.md)' % (msg_name, msg_name)
if summary_description:
unversioned_msgs_list += "%s" % summary_description
unversioned_msgs_list += "\n"
# Write out the index.md file
index_text=f"""# uORB Message Reference
index_text="""# uORB Message Reference
::: info
This list is [auto-generated](https://github.com/PX4/PX4-Autopilot/blob/main/Tools/msg/generate_msg_docs.py) from the source code.
@@ -983,14 +218,14 @@ Graphs showing how these are used [can be found here](../middleware/uorb_graph.m
## Versioned Messages
{versioned_msgs_list}
%s
## Unversioned Messages
{unversioned_msgs_list}
"""
%s
""" % (versioned_msgs_list, unversioned_msgs_list)
index_file = os.path.join(output_dir, 'index.md')
with open(index_file, 'w', encoding='utf-8') as content_file:
with open(index_file, 'w') as content_file:
content_file.write(index_text)
generate_dds_yaml_doc(msg_files)
-2
View File
@@ -1138,8 +1138,6 @@ if num_mags >= 1:
if not math.isnan(sensor_mag_0['temperature'][0]):
mag_0_params['TC_M0_ID'] = int(np.median(sensor_mag_0['device_id']))
# find the min, max and reference temperature
mag_0_params['TC_M0_TMIN'] = np.amin(sensor_mag_0['temperature'])
mag_0_params['TC_M0_TMAX'] = np.amax(sensor_mag_0['temperature'])
File diff suppressed because it is too large Load Diff
+8 -8
View File
@@ -42,7 +42,6 @@
import argparse
import json
import base64
import os
import zlib
import time
import subprocess
@@ -100,13 +99,14 @@ if args.summary != None:
if args.description != None:
desc['description'] = str(args.description)
if args.git_identity != None:
git_dir = os.path.join(args.git_identity, '.git')
p = subprocess.run(["git", "--git-dir", git_dir, "describe", "--exclude", "ext/*", "--always", "--tags"],
stdout=subprocess.PIPE, text=True)
desc['git_identity'] = p.stdout.strip()
p = subprocess.run(["git", "--git-dir", git_dir, "rev-parse", "--verify", "HEAD"],
stdout=subprocess.PIPE, text=True)
desc['git_hash'] = p.stdout.strip()
cmd = "git --git-dir '{:}/.git' describe --exclude ext/* --always --tags".format(args.git_identity)
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout
desc['git_identity'] = p.read().strip().decode('utf-8')
p.close()
cmd = "git --git-dir '{:}/.git' rev-parse --verify HEAD".format(args.git_identity)
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout
desc['git_hash'] = p.read().strip().decode('utf-8')
p.close()
if args.parameter_xml != None:
f = open(args.parameter_xml, "rb")
bytes = f.read()
+983
View File
@@ -0,0 +1,983 @@
#!/usr/bin/env python3
############################################################################
#
# Copyright (c) 2012-2024 PX4 Development Team. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# 3. Neither the name PX4 nor the names of its contributors may be
# used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
# OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
############################################################################
#
# Serial firmware uploader for the PX4FMU bootloader
#
# The PX4 firmware file is a JSON-encoded Python object, containing
# metadata fields and a zlib-compressed base64-encoded firmware image.
#
# The uploader uses the following fields from the firmware file:
#
# image
# The firmware that will be uploaded.
# image_size
# The size of the firmware in bytes.
# board_id
# The board for which the firmware is intended.
# board_revision
# Currently only used for informational purposes.
#
import sys
import argparse
import binascii
import socket
import struct
import json
import zlib
import base64
import time
import array
import os
from sys import platform as _platform
try:
import serial
except ImportError as e:
print(f"Failed to import serial: {e}")
print("")
print("You may need to install it using:")
print(" python -m pip install pyserial")
print("")
sys.exit(1)
# Detect python version
if sys.version_info[0] < 3:
raise RuntimeError("Python 2 is not supported. Please try again using Python 3.")
sys.exit(1)
# Use monotonic time where available
def _time():
try:
return time.monotonic()
except Exception:
return time.time()
class FirmwareNotSuitableException(Exception):
def __init__(self, message):
super(FirmwareNotSuitableException, self).__init__(message)
class firmware(object):
'''Loads a firmware file'''
desc = {}
image = bytes()
crctab = array.array('I', [
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d])
crcpad = bytearray(b'\xff\xff\xff\xff')
def __init__(self, path):
# read the file
f = open(path, "r")
self.desc = json.load(f)
f.close()
self.image = bytearray(zlib.decompress(base64.b64decode(self.desc['image'])))
# pad image to 4-byte length
while ((len(self.image) % 4) != 0):
self.image.extend(b'\xff')
def property(self, propname):
return self.desc[propname]
def __crc32(self, bytes, state):
for byte in bytes:
index = (state ^ byte) & 0xff
state = self.crctab[index] ^ (state >> 8)
return state
def crc(self, padlen):
state = self.__crc32(self.image, int(0))
for _ in range(len(self.image), (padlen - 1), 4):
state = self.__crc32(self.crcpad, state)
return state
class uploader:
'''Uploads a firmware file to the PX4 bootloader'''
# protocol bytes
INSYNC = b'\x12'
EOC = b'\x20'
# reply bytes
OK = b'\x10'
FAILED = b'\x11'
INVALID = b'\x13' # rev3+
BAD_SILICON_REV = b'\x14' # rev5+
# command bytes
NOP = b'\x00' # guaranteed to be discarded by the bootloader
GET_SYNC = b'\x21'
GET_DEVICE = b'\x22'
CHIP_ERASE = b'\x23'
CHIP_VERIFY = b'\x24' # rev2 only
PROG_MULTI = b'\x27'
READ_MULTI = b'\x28' # rev2 only
GET_CRC = b'\x29' # rev3+
GET_OTP = b'\x2a' # rev4+ , get a word from OTP area
GET_SN = b'\x2b' # rev4+ , get a word from SN area
GET_CHIP = b'\x2c' # rev5+ , get chip version
SET_BOOT_DELAY = b'\x2d' # rev5+ , set boot delay
GET_CHIP_DES = b'\x2e' # rev5+ , get chip description in ASCII
GET_VERSION = b'\x2f' # rev5+ , get bootloader version in ASCII
CHIP_FULL_ERASE = b'\x40' # full erase of flash, rev6+
MAX_DES_LENGTH = 20
REBOOT = b'\x30'
INFO_BL_REV = b'\x01' # bootloader protocol revision
BL_REV_MIN = 2 # minimum supported bootloader protocol
BL_REV_MAX = 5 # maximum supported bootloader protocol
INFO_BOARD_ID = b'\x02' # board type
INFO_BOARD_REV = b'\x03' # board revision
INFO_FLASH_SIZE = b'\x04' # max firmware size in bytes
PROG_MULTI_MAX = 252 # protocol max is 255, must be multiple of 4
READ_MULTI_MAX = 252 # protocol max is 255
NSH_INIT = bytearray(b'\x0d\x0d\x0d')
NSH_REBOOT_BL = b"reboot -b\n"
NSH_REBOOT = b"reboot\n"
MAVLINK_REBOOT_ID1 = bytearray(b'\xfe\x21\x72\xff\x00\x4c\x00\x00\x40\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf6\x00\x01\x00\x00\x53\x6b')
MAVLINK_REBOOT_ID0 = bytearray(b'\xfe\x21\x45\xff\x00\x4c\x00\x00\x40\x40\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf6\x00\x00\x00\x00\xcc\x37')
MAX_FLASH_PRGRAM_TIME = 0.001 # Time on an F7 to send SYNC, RESULT from last data in multi RXed
def __init__(self, portname, baudrate_bootloader, baudrate_flightstack):
# Open the port, keep the default timeout short so we can poll quickly.
# On some systems writes can suddenly get stuck without having a
# write_timeout > 0 set.
# chartime 8n1 * bit rate is us
self.chartime = 10 * (1.0 / baudrate_bootloader)
# we use a window approche to SYNC,<result> gathring
self.window = 0
self.window_max = 256
self.window_per = 2 # Sync,<result>
self.ackWindowedMode = False # Assume Non Widowed mode for all USB CDC
self.port = serial.Serial(portname, baudrate_bootloader, timeout=0.5, write_timeout=0)
self.otp = b''
self.sn = b''
self.baudrate_bootloader = baudrate_bootloader
self.baudrate_flightstack = baudrate_flightstack
self.baudrate_flightstack_idx = -1
self.force_erase = False
def close(self):
if self.port is not None:
self.port.close()
def open(self):
# upload timeout
timeout = _time() + 0.2
# attempt to open the port while it exists and until timeout occurs
while self.port is not None:
portopen = True
try:
portopen = self.port.is_open
except AttributeError:
portopen = self.port.isOpen()
if not portopen and _time() < timeout:
try:
self.port.open()
except OSError:
# wait for the port to be ready
time.sleep(0.04)
except serial.SerialException:
# if open fails, try again later
time.sleep(0.04)
else:
break
# debugging code
def __probe(self, state):
# self.port.setRTS(state)
return
def __send(self, c):
# print("send " + binascii.hexlify(c))
self.port.write(c)
def __recv(self, count=1):
c = self.port.read(count)
if len(c) < 1:
raise RuntimeError("timeout waiting for data (%u bytes)" % count)
# print("recv " + binascii.hexlify(c))
return c
def __recv_int(self):
raw = self.__recv(4)
val = struct.unpack("<I", raw)
return val[0]
def __getSync(self, doFlush=True):
if (doFlush):
self.port.flush()
c = bytes(self.__recv())
if c != self.INSYNC:
raise RuntimeError("unexpected %s instead of INSYNC" % c)
c = self.__recv()
if c == self.INVALID:
raise RuntimeError("bootloader reports INVALID OPERATION")
if c == self.FAILED:
raise RuntimeError("bootloader reports OPERATION FAILED")
if c != self.OK:
raise RuntimeError("unexpected response 0x%x instead of OK" % ord(c))
# The control flow for receiving Sync is on the order of 16 Ms per Sync
# This will validate all the SYNC,<results> for a window of programing
# in about 13.81 Ms for 256 blocks written
def __ackSyncWindow(self, count):
if (count > 0):
data = bytearray(bytes(self.__recv(count)))
if (len(data) != count):
raise RuntimeError("Ack Window %i not %i " % (len(data), count))
for i in range(0, len(data), 2):
if bytes([data[i]]) != self.INSYNC:
raise RuntimeError("unexpected %s instead of INSYNC" % data[i])
if bytes([data[i+1]]) == self.INVALID:
raise RuntimeError("bootloader reports INVALID OPERATION")
if bytes([data[i+1]]) == self.FAILED:
raise RuntimeError("bootloader reports OPERATION FAILED")
if bytes([data[i+1]]) != self.OK:
raise RuntimeError("unexpected response 0x%x instead of OK" % ord(data[i+1]))
# attempt to get back into sync with the bootloader
def __sync(self):
# send a stream of ignored bytes longer than the longest possible conversation
# that we might still have in progress
# self.__send(uploader.NOP * (uploader.PROG_MULTI_MAX + 2))
self.port.flushInput()
self.__send(uploader.GET_SYNC +
uploader.EOC)
self.__getSync()
def __trySync(self):
try:
self.port.flush()
if (self.__recv() != self.INSYNC):
# print("unexpected 0x%x instead of INSYNC" % ord(c))
return False
c = self.__recv()
if (c == self.BAD_SILICON_REV):
raise NotImplementedError()
if (c != self.OK):
# print("unexpected 0x%x instead of OK" % ord(c))
return False
return True
except NotImplementedError:
raise RuntimeError("Programing not supported for this version of silicon!\n"
"See https://docs.px4.io/main/en/flight_controller/silicon_errata.html")
except RuntimeError:
# timeout, no response yet
return False
# attempt to determins if the device is CDCACM or A FTDI
def __determineInterface(self):
self.port.flushInput()
# Set a baudrate that can not work on a real serial port
# in that it is 233% off.
try:
self.port.baudrate = self.baudrate_bootloader * 2.33
except NotImplementedError as e:
# This error can occur because pySerial on Windows does not support odd baudrates
print(f"{e} -> could not check for FTDI device, assuming USB connection")
return
self.__send(uploader.GET_SYNC +
uploader.EOC)
try:
self.__getSync(False)
except RuntimeError:
# if it fails we are on a real serial port - only leave this enabled on Windows
if sys.platform.startswith('win'):
self.ackWindowedMode = True
finally:
try:
self.port.baudrate = self.baudrate_bootloader
except Exception:
pass
# send the GET_DEVICE command and wait for an info parameter
def __getInfo(self, param):
self.__send(uploader.GET_DEVICE + param + uploader.EOC)
value = self.__recv_int()
self.__getSync()
return value
# send the GET_OTP command and wait for an info parameter
def __getOTP(self, param):
t = struct.pack("I", param) # int param as 32bit ( 4 byte ) char array.
self.__send(uploader.GET_OTP + t + uploader.EOC)
value = self.__recv(4)
self.__getSync()
return value
# send the GET_SN command and wait for an info parameter
def __getSN(self, param):
t = struct.pack("I", param) # int param as 32bit ( 4 byte ) char array.
self.__send(uploader.GET_SN + t + uploader.EOC)
value = self.__recv(4)
self.__getSync()
return value
# send the GET_CHIP command
def __getCHIP(self):
self.__send(uploader.GET_CHIP + uploader.EOC)
value = self.__recv_int()
self.__getSync()
return value
# send the GET_CHIP command
def __getCHIPDes(self):
self.__send(uploader.GET_CHIP_DES + uploader.EOC)
length = self.__recv_int()
value = self.__recv(length)
self.__getSync()
pieces = value.split(b",")
return pieces
def __getVersion(self):
self.__send(uploader.GET_VERSION + uploader.EOC)
try:
length = self.__recv_int()
value = self.__recv(length)
self.__getSync()
except RuntimeError:
# Bootloader doesn't support version call
return "unknown"
return value.decode()
def __drawProgressBar(self, label, progress, maxVal):
if maxVal < progress:
progress = maxVal
percent = (float(progress) / float(maxVal)) * 100.0
redraw = "\r" if sys.stdout.isatty() else "\n"
sys.stdout.write("%s%s: [%-20s] %.1f%%" % (redraw, label, '='*int(percent/5.0), percent))
sys.stdout.flush()
# send the CHIP_ERASE command and wait for the bootloader to become ready
def __erase(self, label):
print(f"Windowed mode: {self.ackWindowedMode}")
print("\n", end='')
if self.force_erase:
print("Trying force erase of full chip...\n")
self.__send(uploader.CHIP_FULL_ERASE +
uploader.EOC)
else:
self.__send(uploader.CHIP_ERASE +
uploader.EOC)
# erase is very slow, give it 30s
deadline = _time() + 30.0
while _time() < deadline:
usualEraseDuration = 15.0
estimatedTimeRemaining = deadline-_time()
if estimatedTimeRemaining >= usualEraseDuration:
self.__drawProgressBar(label, 30.0-estimatedTimeRemaining, usualEraseDuration)
else:
self.__drawProgressBar(label, 10.0, 10.0)
sys.stdout.write(" (timeout: %d seconds) " % int(deadline-_time()))
sys.stdout.flush()
if self.__trySync():
self.__drawProgressBar(label, 10.0, 10.0)
if self.force_erase:
print("\nForce erase done.\n")
return
if self.force_erase:
raise RuntimeError("timed out waiting for erase, force erase is likely not supported by bootloader!")
else:
raise RuntimeError("timed out waiting for erase")
# send a PROG_MULTI command to write a collection of bytes
def __program_multi(self, data, windowMode):
length = len(data).to_bytes(1, byteorder='big')
self.__send(uploader.PROG_MULTI)
self.__send(length)
self.__send(data)
self.__send(uploader.EOC)
if (not windowMode):
self.__getSync(False)
else:
# The following is done to have minimum delay on the transmission
# of the ne fw. The per block cost of __getSync was about 16 mS per.
# Passively wait on Sync and Result using board rates and
# N.B. attempts to activly wait on InWating still carried 8 mS of overhead
self.__probe(False)
self.__probe(True)
time.sleep((ord(length) * self.chartime) + uploader.MAX_FLASH_PRGRAM_TIME)
self.__probe(False)
# verify multiple bytes in flash
def __verify_multi(self, data):
length = len(data).to_bytes(1, byteorder='big')
self.__send(uploader.READ_MULTI)
self.__send(length)
self.__send(uploader.EOC)
self.port.flush()
programmed = self.__recv(len(data))
if programmed != data:
print("got " + binascii.hexlify(programmed))
print("expect " + binascii.hexlify(data))
return False
self.__getSync()
return True
# send the reboot command
def __reboot(self):
self.__send(uploader.REBOOT +
uploader.EOC)
self.port.flush()
# v3+ can report failure if the first word flash fails
if self.bl_rev >= 3:
self.__getSync()
# split a sequence into a list of size-constrained pieces
def __split_len(self, seq, length):
return [seq[i:i+length] for i in range(0, len(seq), length)]
# upload code
def __program(self, label, fw):
self.__probe(False)
print("\n", end='')
code = fw.image
groups = self.__split_len(code, uploader.PROG_MULTI_MAX)
# Give imedate feedback
self.__drawProgressBar(label, 0, len(groups))
uploadProgress = 0
for bytes in groups:
self.__program_multi(bytes, self.ackWindowedMode)
# If in Window mode, extend the window size for the __ackSyncWindow
if self.ackWindowedMode:
self.window += self.window_per
# Print upload progress (throttled, so it does not delay upload progress)
uploadProgress += 1
if uploadProgress % 256 == 0:
self.__probe(True)
self.__probe(False)
self.__probe(True)
self.__ackSyncWindow(self.window)
self.__probe(False)
self.window = 0
self.__drawProgressBar(label, uploadProgress, len(groups))
# Do any remaining fragment
self.__ackSyncWindow(self.window)
self.window = 0
self.__drawProgressBar(label, 100, 100)
# verify code
def __verify_v2(self, label, fw):
print("\n", end='')
self.__send(uploader.CHIP_VERIFY +
uploader.EOC)
self.__getSync()
code = fw.image
groups = self.__split_len(code, uploader.READ_MULTI_MAX)
verifyProgress = 0
for bytes in groups:
verifyProgress += 1
if verifyProgress % 256 == 0:
self.__drawProgressBar(label, verifyProgress, len(groups))
if (not self.__verify_multi(bytes)):
raise RuntimeError("Verification failed")
self.__drawProgressBar(label, 100, 100)
def __verify_v3(self, label, fw):
print("\n", end='')
self.__drawProgressBar(label, 1, 100)
expect_crc = fw.crc(self.fw_maxsize)
self.__send(uploader.GET_CRC + uploader.EOC)
time.sleep(0.5)
report_crc = self.__recv_int()
self.__getSync()
if report_crc != expect_crc:
print("Expected 0x%x" % expect_crc)
print("Got 0x%x" % report_crc)
raise RuntimeError("Program CRC failed")
self.__drawProgressBar(label, 100, 100)
def __set_boot_delay(self, boot_delay):
self.__send(uploader.SET_BOOT_DELAY +
struct.pack("b", boot_delay) +
uploader.EOC)
self.__getSync()
# get basic data about the board
def identify(self):
self.__determineInterface()
# make sure we are in sync before starting
self.__sync()
# get the bootloader protocol ID first
self.bl_rev = self.__getInfo(uploader.INFO_BL_REV)
if (self.bl_rev < uploader.BL_REV_MIN) or (self.bl_rev > uploader.BL_REV_MAX):
print("Unsupported bootloader protocol %d" % uploader.INFO_BL_REV)
raise RuntimeError("Bootloader protocol mismatch")
self.board_type = self.__getInfo(uploader.INFO_BOARD_ID)
self.board_rev = self.__getInfo(uploader.INFO_BOARD_REV)
self.fw_maxsize = self.__getInfo(uploader.INFO_FLASH_SIZE)
self.version = self.__getVersion()
# upload the firmware
def upload(self, fw_list, force=False, boot_delay=None, boot_check=False, force_erase=False):
self.force_erase = force_erase
# select correct binary
found_suitable_firmware = False
for file in fw_list:
fw = firmware(file)
if self.board_type == fw.property('board_id'):
if len(fw_list) > 1: print("using firmware binary {}".format(file))
found_suitable_firmware = True
break
if not found_suitable_firmware:
msg = "Firmware not suitable for this board (Firmware board_type=%u board_id=%u)" % (
self.board_type, fw.property('board_id'))
print("WARNING: %s" % msg)
if force:
if len(fw_list) > 1:
raise FirmwareNotSuitableException("force flashing failed, more than one file provided, none suitable")
print("FORCED WRITE, FLASHING ANYWAY!")
else:
raise FirmwareNotSuitableException(msg)
percent = fw.property('image_size') / fw.property('image_maxsize')
binary_size = float(fw.property('image_size'))
binary_max_size = float(fw.property('image_maxsize'))
percent = (binary_size / binary_max_size) * 100
print("Loaded firmware for board id: %s,%s size: %d bytes (%.2f%%) " % (fw.property('board_id'), fw.property('board_revision'), fw.property('image_size'), percent))
print()
print(f"Bootloader version: {self.version}")
# Make sure we are doing the right thing
start = _time()
if self.board_type != fw.property('board_id'):
msg = "Firmware not suitable for this board (Firmware board_type=%u board_id=%u)" % (
self.board_type, fw.property('board_id'))
print("WARNING: %s" % msg)
if force:
print("FORCED WRITE, FLASHING ANYWAY!")
else:
raise FirmwareNotSuitableException(msg)
# Prevent uploads where the image would overflow the flash
if self.fw_maxsize < fw.property('image_size'):
raise RuntimeError("Firmware image is too large for this board")
# OTP added in v4:
if self.bl_rev >= 4:
for byte in range(0, 32*6, 4):
x = self.__getOTP(byte)
self.otp = self.otp + x
# print(binascii.hexlify(x).decode('Latin-1') + ' ', end='')
# see src/modules/systemlib/otp.h in px4 code:
self.otp_id = self.otp[0:4]
self.otp_idtype = self.otp[4:5]
self.otp_vid = self.otp[8:4:-1]
self.otp_pid = self.otp[12:8:-1]
self.otp_coa = self.otp[32:160]
# show user:
try:
print("Sn: ", end='')
for byte in range(0, 12, 4):
x = self.__getSN(byte)
x = x[::-1] # reverse the bytes
self.sn = self.sn + x
print(binascii.hexlify(x).decode('Latin-1'), end='') # show user
print('')
print("Chip: %08x" % self.__getCHIP())
otp_id = self.otp_id.decode('Latin-1')
if ("PX4" in otp_id):
print("OTP id: " + otp_id)
print("OTP idtype: " + binascii.b2a_qp(self.otp_idtype).decode('Latin-1'))
print("OTP vid: " + binascii.hexlify(self.otp_vid).decode('Latin-1'))
print("OTP pid: " + binascii.hexlify(self.otp_pid).decode('Latin-1'))
print("OTP coa: " + binascii.b2a_base64(self.otp_coa).decode('Latin-1'))
except Exception as e:
# ignore bad character encodings
print(f"Exception ignored: {e}")
pass
# Silicon errata check was added in v5
if (self.bl_rev >= 5):
des = self.__getCHIPDes()
if (len(des) == 2):
family, revision = des
print(f"Family: {family.decode()}")
print(f"Revision: {revision.decode()}")
print(f"Flash: {self.fw_maxsize} bytes")
# Prevent uploads where the maximum image size of the board config is smaller than the flash
# of the board. This is a hint the user chose the wrong config and will lack features
# for this particular board.
# This check should also check if the revision is an unaffected revision
# and thus can support the full flash, see
# https://github.com/PX4/Firmware/blob/master/src/drivers/boards/common/stm32/board_mcu_version.c#L125-L144
if self.fw_maxsize > fw.property('image_maxsize') and not force:
print(f"WARNING: Board can accept larger flash images ({self.fw_maxsize} bytes) than board config ({fw.property('image_maxsize')} bytes)")
else:
# If we're still on bootloader v4 on a Pixhawk, we don't know if we
# have the silicon errata and therefore need to flash px4_fmu-v2
# with 1MB flash or if it supports px4_fmu-v3 with 2MB flash.
if fw.property('board_id') == 9 \
and fw.property('image_size') > 1032192 \
and not force:
raise RuntimeError("\nThe Board uses bootloader revision 4 and can therefore not determine\n"
"if flashing more than 1 MB (px4_fmu-v3_default) is safe, chances are\n"
"high that it is not safe! If unsure, use px4_fmu-v2_default.\n"
"\n"
"If you know you that the board does not have the silicon errata, use\n"
"this script with --force, or update the bootloader. If you are invoking\n"
"upload using make, you can use force-upload target to force the upload.\n")
self.__erase("Erase ")
self.__program("Program", fw)
if self.bl_rev == 2:
self.__verify_v2("Verify ", fw)
else:
self.__verify_v3("Verify ", fw)
if boot_delay is not None:
self.__set_boot_delay(boot_delay)
print("\nRebooting.", end='')
self.__reboot()
self.port.close()
print(" Elapsed Time %3.3f\n" % (_time() - start))
def __next_baud_flightstack(self):
if self.baudrate_flightstack_idx + 1 >= len(self.baudrate_flightstack):
return False
try:
self.port.baudrate = self.baudrate_flightstack[self.baudrate_flightstack_idx + 1]
self.baudrate_flightstack_idx = self.baudrate_flightstack_idx + 1
except serial.SerialException:
# Sometimes _configure_port fails
time.sleep(0.04)
return True
def send_protocol_splitter_format(self, data):
# Header Structure:
# bits: 1 2 3 4 5 6 7 8
# header[0] - | Magic | (='S')
# header[1] - |T| LenH | (T - 0: mavlink; 1: rtps)
# header[2] - | LenL |
# header[3] - | Checksum |
MAGIC = 83
len_bytes = len(data).to_bytes(2, "big")
LEN_H = len_bytes[0] & 127
LEN_L = len_bytes[1] & 255
CHECKSUM = MAGIC ^ LEN_H ^ LEN_L
header_ints = [MAGIC, LEN_H, LEN_L, CHECKSUM]
header_bytes = struct.pack("{}B".format(len(header_ints)), *header_ints)
self.__send(header_bytes)
self.__send(data)
def send_reboot(self, use_protocol_splitter_format=False):
if (not self.__next_baud_flightstack()):
return False
print("Attempting reboot on %s with baudrate=%d..." % (self.port.port, self.port.baudrate), file=sys.stderr)
if "ttyS" in self.port.port:
print("If the board does not respond, check the connection to the Flight Controller")
else:
print("If the board does not respond, unplug and re-plug the USB connector.", file=sys.stderr)
try:
send_fct = self.__send
if use_protocol_splitter_format:
send_fct = self.send_protocol_splitter_format
# try MAVLINK command first
self.port.flush()
send_fct(uploader.MAVLINK_REBOOT_ID1)
send_fct(uploader.MAVLINK_REBOOT_ID0)
# then try reboot via NSH
send_fct(uploader.NSH_INIT)
send_fct(uploader.NSH_REBOOT_BL)
send_fct(uploader.NSH_INIT)
send_fct(uploader.NSH_REBOOT)
self.port.flush()
self.port.baudrate = self.baudrate_bootloader
except Exception:
try:
self.port.flush()
self.port.baudrate = self.baudrate_bootloader
except Exception:
pass
return True
def main():
# Parse commandline arguments
parser = argparse.ArgumentParser(description="Firmware uploader for the PX autopilot system.")
parser.add_argument('--port', action="store", required=True, help="Comma-separated list of serial port(s) to which the FMU may be attached")
parser.add_argument('--baud-bootloader', action="store", type=int, default=115200, help="Baud rate of the serial port (default is 115200) when communicating with bootloader, only required for true serial ports.")
parser.add_argument('--baud-flightstack', action="store", default="57600", help="Comma-separated list of baud rate of the serial port (default is 57600) when communicating with flight stack (Mavlink or NSH), only required for true serial ports.")
parser.add_argument('--force', action='store_true', default=False, help='Override board type check, or silicon errata checks and continue loading')
parser.add_argument('--force-erase', action="store_true", help="Do not perform the blank check, always erase every sector of the application space")
parser.add_argument('--boot-delay', type=int, default=None, help='minimum boot delay to store in flash')
parser.add_argument('--use-protocol-splitter-format', action='store_true', help='use protocol splitter format for reboot')
parser.add_argument('firmware', action="store", nargs='+', help="Firmware file(s)")
args = parser.parse_args()
if args.use_protocol_splitter_format:
print("Using protocol splitter format to reboot pixhawk!")
# warn people about ModemManager which interferes badly with Pixhawk
if os.path.exists("/usr/sbin/ModemManager"):
print("==========================================================================================================")
print("WARNING: You should uninstall ModemManager as it conflicts with any non-modem serial device (like Pixhawk)")
print("==========================================================================================================")
print("Waiting for bootloader...")
# tell any GCS that might be connected to the autopilot to give up
# control of the serial port
# send to localhost and default GCS port
ipaddr = '127.0.0.1'
portnum = 14550
# COMMAND_LONG in MAVLink 1
heartbeatpacket = bytearray.fromhex('fe097001010000000100020c5103033c8a')
commandpacket = bytearray.fromhex('fe210101014c00000000000000000000000000000000000000000000803f00000000f6000000008459')
# initialize an UDP socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# send heartbeat to initialize connection and command to free the link
s.sendto(heartbeatpacket, (ipaddr, portnum))
s.sendto(commandpacket, (ipaddr, portnum))
# close the socket
s.close()
# Spin waiting for a device to show up
try:
while True:
portlist = []
patterns = args.port.split(",")
# on unix-like platforms use glob to support wildcard ports. This allows
# the use of /dev/serial/by-id/usb-3D_Robotics on Linux, which prevents the upload from
# causing modem hangups etc
if "linux" in _platform or "darwin" in _platform or "cygwin" in _platform:
import glob
for pattern in patterns:
portlist += glob.glob(pattern)
else:
portlist = patterns
baud_flightstack = [int(x) for x in args.baud_flightstack.split(',')]
successful = False
unsuitable_board = False
for port in portlist:
# print("Trying %s" % port)
# create an uploader attached to the port
try:
if "linux" in _platform:
# Linux, don't open Mac OS and Win ports
if "COM" not in port and "tty.usb" not in port:
up = uploader(port, args.baud_bootloader, baud_flightstack)
elif "darwin" in _platform:
# OS X, don't open Windows and Linux ports
if "COM" not in port and "ACM" not in port:
up = uploader(port, args.baud_bootloader, baud_flightstack)
elif "cygwin" in _platform:
# Cygwin, don't open native Windows COM and Linux ports
if "COM" not in port and "ACM" not in port:
up = uploader(port, args.baud_bootloader, baud_flightstack)
elif "win" in _platform:
# Windows, don't open POSIX ports
if "/" not in port:
up = uploader(port, args.baud_bootloader, baud_flightstack)
except Exception as e:
# open failed, rate-limit our attempts
time.sleep(0.05)
print(f"Exception ignored: {e}")
# and loop to the next port
continue
found_bootloader = False
while True:
up.open()
# port is open, try talking to it
try:
# identify the bootloader
up.identify()
found_bootloader = True
print()
print(f"Found board id: {up.board_type},{up.board_rev} bootloader protocol revision {up.bl_rev} on {port}")
break
except (RuntimeError, serial.SerialException):
if not up.send_reboot(args.use_protocol_splitter_format):
break
# wait for the reboot, without we might run into Serial I/O Error 5
time.sleep(0.25)
# always close the port
up.close()
# wait for the close, without we might run into Serial I/O Error 6
time.sleep(0.3)
if not found_bootloader:
# Go to the next port
continue
try:
# ok, we have a bootloader, try flashing it
up.upload(args.firmware, force=args.force, boot_delay=args.boot_delay, force_erase=args.force_erase)
# if we made this far without raising exceptions, the upload was successful
successful = True
except RuntimeError as e:
# print the error
print(f"\n\nError: {e}")
except FirmwareNotSuitableException:
unsuitable_board = True
up.close()
continue
except IOError:
up.close()
continue
finally:
# always close the port
up.close()
# we could loop here if we wanted to wait for more boards...
if successful:
sys.exit(0)
else:
sys.exit(1)
if unsuitable_board:
# If we land here, we went through all ports, did not flash any
# board and found at least one unsuitable board.
# Exit with 2, so a caller can distinguish from other errors
sys.exit(2)
# Delay retries to < 20 Hz to prevent spin-lock from hogging the CPU
time.sleep(0.05)
# CTRL+C aborts the upload/spin-lock by interrupt mechanics
except KeyboardInterrupt:
print("\n Upload aborted by user.")
sys.exit(0)
if __name__ == '__main__':
main()
# vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4
-5
View File
@@ -144,8 +144,6 @@ def main():
help='number of tidy instances to be run in parallel.')
parser.add_argument('files', nargs='*', default=['.*'],
help='files to be processed (regex on path)')
parser.add_argument('-exclude', dest='exclude', default=None,
help='regular expression matching files to exclude')
parser.add_argument('-fix', action='store_true', help='apply fix-its')
parser.add_argument('-format', action='store_true', help='Reformat code '
'after applying fixes')
@@ -194,7 +192,6 @@ def main():
# Build up a big regexy filter from all command line arguments.
file_name_re = re.compile('(' + ')|('.join(args.files) + ')')
exclude_re = re.compile(args.exclude) if args.exclude else None
try:
# Spin up a bunch of tidy-launching threads.
@@ -208,8 +205,6 @@ def main():
# Fill the queue with files.
for name in files:
if file_name_re.search(name):
if exclude_re and exclude_re.search(name):
continue
queue.put(name)
# Wait for all threads to be done.
+3 -1
View File
@@ -4,7 +4,7 @@ GREEN='\033[0;32m'
NO_COLOR='\033[0m' # No Color
SCRIPTID="${GREEN}[docker-entrypoint.sh]${NO_COLOR}"
echo -e "$SCRIPTID $( uname -m ) | $(date -u +%FT%TZ)"
echo -e "$SCRIPTID Starting"
# Start virtual X server in the background
# - DISPLAY default is :99, set in dockerfile
@@ -22,4 +22,6 @@ if [ -n "${ROS_DISTRO}" ]; then
source "/opt/ros/$ROS_DISTRO/setup.bash"
fi
echo -e "$SCRIPTID ($( uname -m ))"
exec "$@"
+1 -1
View File
@@ -74,7 +74,7 @@ python3 -m pip install --user -r ${DIR}/requirements.txt
# Optional, but recommended additional simulation tools:
if [[ $INSTALL_SIM == "--sim-tools" ]]; then
if ! brew ls --versions px4-sim > /dev/null; then
if brew ls --versions px4-sim > /dev/null; then
brew install px4-sim
elif [[ $REINSTALL_FORMULAS == "--reinstall" ]]; then
brew reinstall px4-sim
+33 -21
View File
@@ -6,9 +6,9 @@ set -e
## Can also be used in docker.
##
## Installs:
## - Common dependencies and tools for nuttx, Gazebo
## - Common dependencies and tools for nuttx, jMAVSim, Gazebo
## - NuttX toolchain (omit with arg: --no-nuttx)
## - Gazebo Harmonic simulator (omit with arg: --no-sim-tools)
## - jMAVSim and Gazebo9 simulator (omit with arg: --no-sim-tools)
##
INSTALL_NUTTX="true"
@@ -176,10 +176,8 @@ if [[ $INSTALL_NUTTX == "true" ]]; then
echo
echo "Fetching Xtensa compilers"
XTENSA_FILE_NAME=xtensa-esp-elf-13.2.0_20240530-x86_64-linux-gnu.tar.xz
wget -q -P $DIR https://github.com/espressif/crosstool-NG/releases/download/esp-13.2.0_20240530/$XTENSA_FILE_NAME
sudo tar -xf $DIR/$XTENSA_FILE_NAME -C /opt
rm $DIR/$XTENSA_FILE_NAME
wget -q -P $DIR https://github.com/espressif/crosstool-NG/releases/download/esp-13.2.0_20240530/xtensa-esp-elf-13.2.0_20240530-x86_64-linux-gnu.tar.xz
sudo tar -xf $DIR/xtensa-esp-elf-13.2.0_20240530-x86_64-linux-gnu.tar.xz -C /opt
echo 'export PATH=$PATH:/opt/xtensa-esp-elf/bin/' >> /home/$USER/.bashrc
fi
@@ -196,11 +194,6 @@ if [[ $INSTALL_NUTTX == "true" ]]; then
fi
fi
if [[ "${UBUNTU_RELEASE}" == "25.10" ]]; then
echo "[ubuntu.sh] Gazebo binaries are not available for 25.10, skipping installation"
INSTALL_SIM="false"
fi
# Simulation tools
if [[ $INSTALL_SIM == "true" ]]; then
@@ -212,18 +205,37 @@ if [[ $INSTALL_SIM == "true" ]]; then
bc \
;
# Gazebo Harmonic installation (Ubuntu 22.04+)
echo "[ubuntu.sh] Gazebo (Harmonic) will be installed"
# Add Gazebo binary repository
sudo wget https://packages.osrfoundation.org/gazebo.gpg -O /usr/share/keyrings/pkgs-osrf-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/pkgs-osrf-archive-keyring.gpg] http://packages.osrfoundation.org/gazebo/ubuntu-stable $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/gazebo-stable.list > /dev/null
sudo apt-get update -y --quiet
# Gazebo / Gazebo classic installation
if [[ "${UBUNTU_RELEASE}" == "18.04" || "${UBUNTU_RELEASE}" == "20.04" ]]; then
sudo sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-stable.list'
wget http://packages.osrfoundation.org/gazebo.key -O - | sudo apt-key add -
# Update list, since new gazebo-stable.list has been added
sudo apt-get update -y --quiet
# Install Gazebo
gazebo_packages="gz-harmonic libunwind-dev"
# Install Gazebo classic
if [[ "${UBUNTU_RELEASE}" == "18.04" ]]; then
gazebo_classic_version=9
gazebo_packages="gazebo$gazebo_classic_version libgazebo$gazebo_classic_version-dev"
else
# default and Ubuntu 20.04
gazebo_classic_version=11
gazebo_packages="gazebo$gazebo_classic_version libgazebo$gazebo_classic_version-dev"
fi
else
# Expects Ubuntu 22.04 > by default
echo "[ubuntu.sh] Gazebo (Harmonic) will be installed"
echo "[ubuntu.sh] Earlier versions will be removed"
# Add Gazebo binary repository
sudo wget https://packages.osrfoundation.org/gazebo.gpg -O /usr/share/keyrings/pkgs-osrf-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/pkgs-osrf-archive-keyring.gpg] http://packages.osrfoundation.org/gazebo/ubuntu-stable $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/gazebo-stable.list > /dev/null
sudo apt-get update -y --quiet
if [[ "${UBUNTU_RELEASE}" == "24.04" ]]; then
gazebo_packages="$gazebo_packages cppzmq-dev"
# Install Gazebo
gazebo_packages="gz-harmonic libunwind-dev"
if [[ "${UBUNTU_RELEASE}" == "24.04" ]]; then
gazebo_packages="$gazebo_packages cppzmq-dev"
fi
fi
sudo DEBIAN_FRONTEND=noninteractive apt-get -y --quiet --no-install-recommends install \
@@ -21,6 +21,7 @@ CONFIG_DRIVERS_IMU_INVENSENSE_ICM20948=y
CONFIG_DRIVERS_IRLOCK=y
CONFIG_COMMON_LIGHT=y
CONFIG_COMMON_MAGNETOMETER=y
CONFIG_COMMON_OPTICAL_FLOW=y
CONFIG_DRIVERS_PCA9685_PWM_OUT=y
CONFIG_DRIVERS_POWER_MONITOR_INA226=y
CONFIG_DRIVERS_PWM_OUT=y
@@ -34,6 +34,7 @@ CONFIG_DRIVERS_OSD_MSP_OSD=y
CONFIG_DRIVERS_POWER_MONITOR_INA226=y
CONFIG_DRIVERS_POWER_MONITOR_INA228=y
CONFIG_DRIVERS_POWER_MONITOR_INA238=y
CONFIG_DRIVERS_POWER_MONITOR_PM_SELECTOR_AUTERION=y
CONFIG_DRIVERS_PWM_OUT=y
CONFIG_DRIVERS_PX4IO=y
CONFIG_DRIVERS_RC_INPUT=y

Some files were not shown because too many files have changed in this diff Show More