Compare commits

..

2 Commits

Author SHA1 Message Date
Matthias Grob 71b8aba3e3 refactor(lightware_laser): simplify device_information handling
- publish every 3 seconds
- adjust to name instead of vendor and model
- buffer size adjusted according to datasheet
- switch to PublicationData
2026-03-31 20:56:01 +02:00
Claudio Chies 5e570f2d5c SENS: RNG: Add Device Information publishing to Lightware Rangefinder 2026-03-31 20:55:58 +02:00
582 changed files with 12074 additions and 18327 deletions
-207
View File
@@ -1,207 +0,0 @@
---
name: review-pr
description: Review a pull request with structured, domain-aware feedback
argument-hint: "<PR number or URL>"
allowed-tools: Bash, Read, Glob, Grep, Agent
---
# PX4 Pull Request Review
Review a pull request with domain-aware checks based on which files are changed.
**No Claude attribution anywhere.**
## Steps
1. **Fetch PR context.** Run these in parallel:
- `gh pr view <PR> --json number,title,body,baseRefName,headRefName,files,commits,reviewRequests,reviews,author`
- `gh pr checks <PR>` (exit code 8 means some checks are pending, this is normal, not an error)
- `gh pr diff <PR>` -- if this fails with HTTP 406 (300+ files), do NOT retry. Instead use `gh api repos/OWNER/REPO/pulls/NUMBER/files --paginate` to get the full file list in one call, then fetch patches for key infrastructure files individually and sample representative changes from each domain touched.
- `gh api repos/OWNER/REPO/pulls/NUMBER/comments --paginate --jq '.[] | {user: .user.login, body: .body, path: .path, created_at: .created_at}'` to get inline review comments
- `gh api repos/OWNER/REPO/issues/NUMBER/comments --paginate --jq '.[] | {user: .user.login, body: .body, created_at: .created_at}'` to get PR conversation comments
From the PR metadata, note:
- **Assigned reviewers**: who has been requested to review (from `reviewRequests`)
- **Existing reviews**: who has already reviewed and their verdict (from `reviews` -- approved, changes_requested, commented, dismissed)
- **PR comments and inline comments**: read all existing feedback to avoid duplicating points already raised by other reviewers, and to build on their discussion rather than ignoring it
2. **Check CI status.** From the `gh pr checks` output in step 1, summarize pass/fail/pending. If there are failures, fetch logs with `gh run view <run-id> --log-failed`. Include CI status in the output.
3. **Recommend merge strategy.** Analyze the commit history and recommend squash or rebase merge. This decision informs all subsequent commit hygiene feedback.
**Recommend rebase merge** when:
- Commits are atomic, each builds/works independently
- Each commit has a proper `type(scope): description` message
- The PR intentionally separates logical changes (e.g., refactor + feature, or one commit per module)
- The commit history tells a useful story that would be lost by squashing
**Recommend squash merge** when:
- There are WIP, fixup, or review-response commits
- Commit messages are messy or inconsistent
- The PR is a single logical change spread across multiple commits
- There are "oops" or "make format" commits mixed in
Include the recommendation in the output. If recommending rebase, flag any commits that break atomicity or have bad messages. If recommending squash, don't bother flagging individual commit messages (they'll be discarded) but ensure the PR title is correct since it becomes the squash commit message.
4. **Check conventional commit title.** Verify the PR title follows `type(scope): description` per CONTRIBUTING.md. The PR title becomes the commit message on squash-merge, so it must be accurate and descriptive. Verify the scope matches the primary area of changed files. If the PR introduces breaking changes, the title must include `!` before the colon. If rebase merge was recommended in step 3, also scan individual commit messages for anti-patterns: vague messages ("fix", "update"), missing type prefix, review-response noise ("apply suggestions from code review", "do make format"), or WIP markers. Flag these for rewording.
5. **Identify domains touched.** Classify changed files into domains based on paths (a PR may touch multiple):
- **Estimation**: `src/modules/ekf2/`, `src/lib/wind_estimator/`, `src/lib/world_magnetic_model/`
- **Control**: `src/modules/mc_*control*/`, `src/modules/fw_*control*/`, `src/modules/flight_mode_manager/`, `src/lib/rate_control/`, `src/lib/npfg/`, `src/modules/vtol_att_control/`
- **Drivers/CAN**: `src/drivers/`, `src/modules/cyphal/`, `src/drivers/uavcan*/`
- **Simulation**: `src/modules/simulation/`, `Tools/simulation/`
- **System**: `src/modules/commander/`, `src/modules/logger/`, `src/systemcmds/`, `platforms/`, `src/modules/dataman/`
- **Board Addition**: `boards/{manufacturer}/{board}/` (new directories only, not modifications to existing boards)
- **CI/Build**: `.github/`, `CMakeLists.txt`, `Makefile`, `cmake/`, `Tools/`, `Kconfig`
- **Messages/Protocol**: `msg/`, `src/modules/mavlink/`, `src/modules/uxrce_dds_client/`
6. **Apply core checks** (always):
- **Correctness**: logic errors, off-by-ones, unhandled edge cases
- **Type safety**: int16 overflow, float/double promotion, unsigned subtraction, use `uint64_t` for absolute time
- **Initialization**: uninitialized variables, missing default construction
- **Buffer safety**: unchecked array access, stack allocation of large buffers, snprintf bounds
- **Magic numbers**: every numeric literal needs a named constant or justification
- **Framework reuse**: use PX4_ERR/WARN/INFO, existing libraries (AlphaFilter, SlewRate, RateControl), MAVLink constants from the library
- **Naming**: accurate, no unjustified abbreviations, current terminology (GPS -> GNSS for new code)
- **Unnecessary complexity**: can code be removed instead of added? Is there a simpler pattern?
- **Test coverage**: new features should include unit or integration tests; bug fixes should include regression tests where practical. When automated testing is infeasible (hardware-specific), require a flight log link from https://logs.px4.io or bench test evidence.
- **PR hygiene**: focused scope, no unrelated formatting, no stale submodule changes. Commits should be atomic and independently revertable. Multiple WIP or review-response commits should be squashed. Clean, logical commits will be preserved individually on main via rebase merge. **Do NOT assume PRs are squash-merged. Both squash and rebase merge are enabled; merge commits are disabled.** Verify the PR targets `main` unless it is a backport or release-specific fix.
- **Formatting**: `make format` / `make check_format` (astyle) for C/C++ files; `clang-tidy` clean. Python files checked with `mypy` and `flake8`. PRs failing CI format or lint checks will not be merged.
- **Coding style**: C/C++ must follow the [PX4 coding style](https://docs.px4.io/main/en/contribute/code.html)
- **Necessity**: challenge every addition with "Why?" Is this actually needed or just copied? Can we change a default instead of adding runtime detection?
- **Root cause vs symptom**: is this fixing the real problem or masking it?
- **Ecosystem impact**: what does this change mean for QGC users, log analysis tools, and third-party integrations?
- **Sustainability**: who will maintain this? Does it create long-term burden?
- **Architecture fit**: does the code live in the module that naturally owns the data? Are there unnecessary cross-module dependencies?
- **End user impact**: will parameters confuse less-technical users? Are error messages actionable in QGC?
7. **Apply domain checks** based on step 5:
**Estimation:**
- Singularities in aerospace math (euler angles near gimbal lock, sideslip at low airspeed)
- Aliasing from downsampling sensor data without filtering
- Kalman filter correctness (Joseph form, innovation variance, covariance symmetry)
- CPU cost on embedded targets (avoid unnecessary sqrt, limit fusion rate)
- Frame/coordinate system correctness (FRD vs NED, body vs earth)
**Control:**
- Phase margin: output filters consume margin for no benefit; prefer adjusting gyro/d-gyro cutoffs
- Circular dependencies: sensor data feeding back into its own control loop (e.g., throttle-based airspeed in TECS)
- NaN propagation in flight-critical math; check `PX4_ISFINITE` before magnitude checks
- Setpoint generation vs output-stage hacks: prefer proper setpoint smoothing over controller output filtering
- Yaw control edge cases: heading lock, drift, setpoint propagation
- Flight task inheritance chain: correct base class for the desired behavior
- Control allocation: actuator function ordering, motor index mapping
**Drivers/CAN:**
- CAN bus devices behave differently from serial/SPI; check driver assumptions
- ESC index mapping: telemetry index != channel when motors are disabled
- ESC hardware quirks: 4-in-1 ESCs may report current on only one channel
- device_id correctness and I2CSPIDriver patterns
- Time representation: prefer `hrt_abstime` over iteration counts
**Simulation:**
- Physics fidelity: noise models should match reality (GPS noise is not Gaussian)
- Keep gz_bridge generic; vehicle-specific logic belongs in plugins
- Prefer gz-transport over ROS2 dependencies when possible
- Wrench commands for physics correctness vs kinematic constraints
- Library generic/specific boundary: only base classes in common libs
**System:**
- Race conditions and concurrency: no partial fixes, demand complete solutions
- Semaphore/scheduling edge cases; understand RTOS guarantees
- State machine sequential-logic bugs (consecutive RTL, armed/disarmed alternation)
- uORB-driven scheduling (`SubscriptionCallback`), not extra threads
- param_set triggers auto-save; no redundant param_save_default
- Flash/memory efficiency: avoid `std::string` on embedded, minimize SubscriptionData usage
- Constructor initialization order matters
**CI/Build:**
- Pipeline race conditions (tag + branch push double-trigger, git describe correctness)
- Container image size (check layer bloat)
- Ubuntu LTS support policy (latest + one prior only)
- Build time impact
- CMake preferred over Makefiles
**Messages/Protocol:**
- Backwards compatibility: will this break QGC, post-flight tools, or uLog parsers?
- uORB: `timestamp` for publication metadata, `timestamp_sample` close to physical sample, include `device_id`
- Don't version messages unless strictly needed
- Parameter UX: will this confuse users in a GCS? Every new param is a configuration burden
- MAVLink: use library constants, don't implement custom stream rates
**Board Addition:**
- **Flight logs**: require a link to https://logs.px4.io demonstrating basic operation for the vehicle type (hover for multicopters, stable flight for fixed-wing, driving for rovers, etc.); short bench-only logs are insufficient
- **Documentation**: require a docs page in `docs/en/flight_controller/` with pinout, where-to-buy, connector types, version badge, and manufacturer-supported notice block
- **USB VID/PID**: must not reuse another manufacturer's Vendor ID; manufacturer must use their own
- **Board naming**: directory is `boards/{manufacturer}/{board}/`, both lowercase, hyphens for board name
- **Unique board_id**: registered in `boards/boards.json`, no collisions
- **Copied code cleanup**: check for leftover files, configs, or comments from the template board; "Is this real or leftover?"
- **RC configuration**: prefer `CONFIG_DRIVERS_COMMON_RC` over legacy `CONFIG_DRIVERS_RC_INPUT`
- **No board-specific custom modules**: reject copy-pasted drivers (e.g., custom heater) when existing infrastructure works
- **Bootloader**: expect a bootloader defconfig (`nuttx-config/bootloader/defconfig`) or explanation of shared bootloader
- **CI integration**: board must be added to CI compile workflows so it builds on every PR
- **Flash constraints**: verify enabled modules fit in flash; we are running low across all board targets
- **Port labels**: serial port labels must match what is physically printed on the board
- **Hardware availability**: for unknown manufacturers, verify the product exists and is purchasable (no vaporware)
8. **Format output** as:
- **CI status**: pass/fail summary, link to failed runs if any
- **Merge strategy**: recommend squash or rebase merge with reasoning
- **Title check**: pass/fail with suggestion
- **Review status**: list assigned reviewers and any existing reviews (who approved, who requested changes, key points already raised). Note if your review would duplicate feedback already given.
- **Domains detected**: list which domain checks were applied
- **Summary**: one paragraph on what the PR does and whether the approach is sound
- **Issues**: numbered list, each with file:line, severity (blocker/warning/nit), and explanation. Skip issues already raised by other reviewers unless you have something to add.
- **Verdict**: approve, request changes, or needs discussion
After the structured output, also display a **draft PR comment** formatted using the PR comment formatting rules from step 9. This gives the user a preview of what would be posted.
9. **Interactive dialog.** After displaying the review, present the user with these options:
Present options based on the verdict:
If verdict is **approve**:
```
What would you like to do?
1. Chat about this PR (ask questions, explore code) [default]
2. Approve this PR and post the review comment
3. Adjust the review or draft (tell me what to change)
4. Done for now
```
If verdict is **request changes**:
```
What would you like to do?
1. Chat about this PR (ask questions, explore code) [default]
2. Request changes on this PR and post the review comment
3. Adjust the review or draft (tell me what to change)
4. Done for now
```
If verdict is **needs discussion**:
```
What would you like to do?
1. Chat about this PR (ask questions, explore code) [default]
2. Post the review as a comment (no approval or rejection)
3. Adjust the review or draft (tell me what to change)
4. Done for now
```
Wait for the user to choose before proceeding. If they pick:
- **1 (chat)**: enter a free-form conversation about the PR. The user can ask about specific files, code paths, or decisions. When done, loop back to the options. This is the default if the user just presses enter.
- **2 (submit)**: use the draft PR comment already shown. Before posting, check if you have review permissions: run `gh api repos/OWNER/REPO/collaborators/$(gh api user --jq .login)/permission --jq .permission` -- if `admin` or `write`, submit as a formal review with `gh pr review <PR> --approve --body "..."` or `gh pr review <PR> --request-changes --body "..."` based on the verdict. If no write access, fall back to `gh pr comment <PR> --body "..."`. Always confirm with the user before posting.
- **3 (adjust)**: ask what to change, update the review and draft, then loop back to the options.
- **4 (done)**: stop.
**PR comment formatting rules** (for the draft):
When writing the GitHub comment, rewrite the review to sound like a human reviewer, not a structured report. Do NOT include the full skill output. Instead:
- Drop most meta-sections (CI status, title check, domains detected, severity labels) but keep the merge strategy recommendation (e.g., "I'd suggest a rebase merge here since the commits are clean and atomic" or "This should be squash-merged, the commit history is messy")
- Write conversationally: "Nice work on this. A few things I noticed:" not "Issues: 1. file:line (warning):"
- Lead with a brief take on the overall change (1-2 sentences)
- List only actionable feedback as natural review comments, not numbered checklists
- Skip nits unless they are particularly useful
- End with a clear stance: looks good to merge, needs a few changes, or let's discuss X
- Post with `gh pr comment <PR> --body "$(cat <<'EOF' ... EOF)"`. Do not post without explicit confirmation.
If the user provided arguments, use them as context: $ARGUMENTS
-1
View File
@@ -1 +0,0 @@
build/
-115
View File
@@ -1,115 +0,0 @@
name: Build PX4 .deb Package
description: Build PX4 SITL, run cpack, validate the .deb, and upload artifact
inputs:
target:
description: 'Build target: default or sih'
required: true
artifact-name:
description: Name for the uploaded artifact
required: true
ccache-key-prefix:
description: Prefix for ccache cache keys
default: deb-ccache
ccache-max-size:
description: Maximum ccache size
default: 400M
runs:
using: composite
steps:
- name: Restore ccache
id: ccache-restore
uses: actions/cache/restore@v5
with:
path: ~/.ccache
key: ${{ inputs.ccache-key-prefix }}-${{ github.ref_name }}-${{ github.sha }}
restore-keys: |
${{ inputs.ccache-key-prefix }}-${{ github.ref_name }}-
${{ inputs.ccache-key-prefix }}-${{ github.base_ref || 'main' }}-
${{ inputs.ccache-key-prefix }}-
- name: Configure ccache
shell: bash
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = ${{ inputs.ccache-max-size }}" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Build PX4 SITL
shell: bash
run: make px4_sitl_${{ inputs.target }}
- name: ccache stats
if: always()
shell: bash
run: ccache -s
- name: Save ccache
uses: actions/cache/save@v5
if: always()
with:
path: ~/.ccache
key: ${{ inputs.ccache-key-prefix }}-${{ github.ref_name }}-${{ github.sha }}
- name: Build .deb package
shell: bash
run: |
cd build/px4_sitl_${{ inputs.target }}
cpack -G DEB
- name: Print package info and contents
shell: bash
run: |
cd build/px4_sitl_${{ inputs.target }}
echo "--- Package info ---"
dpkg-deb -I *.deb
echo "--- Package contents ---"
dpkg-deb -c *.deb
- name: Validate sih package
if: inputs.target == 'sih'
shell: bash
run: |
cd build/px4_sitl_sih
echo "--- Verify NO Gazebo resources ---"
! dpkg-deb -c px4_*.deb | grep share/gz > /dev/null && echo "PASS: no Gazebo" || { echo "FAIL: Gazebo found"; exit 1; }
echo "--- Install test ---"
dpkg -i px4_*.deb
test -x /opt/px4/bin/px4 || { echo "FAIL: px4 binary not found"; exit 1; }
test -L /usr/bin/px4 || { echo "FAIL: symlink not created"; exit 1; }
test ! -d /opt/px4/share/gz || { echo "FAIL: Gazebo dir should not exist"; exit 1; }
echo "--- Smoke test ---"
/opt/px4/bin/px4 -h
echo "PASS: sih package validation successful"
- name: Validate gazebo package
if: inputs.target == 'default'
shell: bash
run: |
cd build/px4_sitl_default
echo "--- Verify Gazebo resources in package ---"
dpkg-deb -c px4-gazebo_*.deb | grep share/gz/models > /dev/null || { echo "FAIL: models missing"; exit 1; }
dpkg-deb -c px4-gazebo_*.deb | grep share/gz/worlds > /dev/null || { echo "FAIL: worlds missing"; exit 1; }
echo "--- Install test ---"
dpkg -i px4-gazebo_*.deb
test -x /opt/px4-gazebo/bin/px4 || { echo "FAIL: px4 binary not found"; exit 1; }
test -x /opt/px4-gazebo/bin/px4-gazebo || { echo "FAIL: wrapper not found"; exit 1; }
test -L /usr/bin/px4-gazebo || { echo "FAIL: symlink not created"; exit 1; }
test -d /opt/px4-gazebo/share/gz/models || { echo "FAIL: Gazebo models not installed"; exit 1; }
echo "--- Smoke test ---"
/opt/px4-gazebo/bin/px4 -h
echo "PASS: gazebo package validation successful"
- name: Upload .deb artifacts
uses: actions/upload-artifact@v7
with:
name: ${{ inputs.artifact-name }}
path: build/px4_sitl_${{ inputs.target }}/*.deb
if-no-files-found: error
@@ -1,21 +0,0 @@
name: Build Gazebo Classic SITL
description: Build PX4 firmware and Gazebo Classic plugins with ccache stats
runs:
using: composite
steps:
- name: Build - PX4 Firmware (SITL)
shell: bash
run: make px4_sitl_default
- name: Cache - Stats after PX4 Firmware
shell: bash
run: ccache -s
- name: Build - Gazebo Classic Plugins
shell: bash
run: make px4_sitl_default sitl_gazebo-classic
- name: Cache - Stats after Gazebo Plugins
shell: bash
run: ccache -s
-22
View File
@@ -1,22 +0,0 @@
name: Save ccache
description: Print ccache stats and save to cache
inputs:
cache-primary-key:
description: Primary cache key from setup-ccache output
required: true
runs:
using: composite
steps:
- name: Cache - Stats
if: always()
shell: bash
run: ccache -s
- name: Cache - Save ccache
if: always()
uses: actions/cache/save@v5
with:
path: ~/.ccache
key: ${{ inputs.cache-primary-key }}
-56
View File
@@ -1,56 +0,0 @@
name: Setup ccache
description: Restore ccache from cache and configure ccache.conf
inputs:
cache-key-prefix:
description: Cache key prefix (e.g. ccache-sitl)
required: true
max-size:
description: Max ccache size (e.g. 300M)
required: false
default: '300M'
base-dir:
description: ccache base_dir value
required: false
default: '${GITHUB_WORKSPACE}'
install-ccache:
description: Install ccache via apt before configuring
required: false
default: 'false'
outputs:
cache-primary-key:
description: Primary cache key (pass to save-ccache)
value: ${{ steps.restore.outputs.cache-primary-key }}
runs:
using: composite
steps:
- name: Cache - Install ccache
if: inputs.install-ccache == 'true'
shell: bash
run: apt-get update && apt-get install -y ccache
- name: Cache - Restore ccache
id: restore
uses: actions/cache/restore@v5
with:
path: ~/.ccache
key: ${{ inputs.cache-key-prefix }}-${{ github.ref_name }}-${{ github.sha }}
restore-keys: |
${{ inputs.cache-key-prefix }}-${{ github.ref_name }}-
${{ inputs.cache-key-prefix }}-${{ github.base_ref || 'main' }}-
${{ inputs.cache-key-prefix }}-
- name: Cache - Configure ccache
shell: bash
run: |
mkdir -p ~/.ccache
echo "base_dir = ${{ inputs.base-dir }}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = ${{ inputs.max-size }}" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
@@ -1,21 +0,0 @@
---
applyTo: "boards/**"
---
# Board Addition Review Guidelines
In addition to the core code review guidelines, when reviewing new board additions:
- **Flight logs**: require a link to https://logs.px4.io demonstrating basic operation for the vehicle type (hover for multicopters, stable flight for fixed-wing, driving for rovers, etc.); short bench-only logs are insufficient
- **Documentation**: require a docs page in `docs/en/flight_controller/` with pinout, where-to-buy, connector types, version badge, and manufacturer-supported notice block
- **USB VID/PID**: must not reuse another manufacturer's Vendor ID; manufacturer must use their own
- **Board naming**: directory is `boards/{manufacturer}/{board}/`, both lowercase, hyphens for board name
- **Unique board_id**: registered in `boards/boards.json`, no collisions
- **Copied code cleanup**: check for leftover files, configs, or comments from the template board. Ask "Is this real or leftover?"
- **RC configuration**: prefer `CONFIG_DRIVERS_COMMON_RC` over legacy `CONFIG_DRIVERS_RC_INPUT`
- **No board-specific custom modules**: reject copy-pasted drivers (e.g., custom heater) when existing infrastructure works
- **Bootloader**: expect a bootloader defconfig (`nuttx-config/bootloader/defconfig`) or explanation of shared bootloader
- **CI integration**: board must be added to CI compile workflows so it builds on every PR
- **Flash constraints**: verify enabled modules fit in flash; we are running low across all board targets
- **Port labels**: serial port labels must match what is physically printed on the board
- **Hardware availability**: for unknown manufacturers, verify the product exists and is purchasable (no vaporware)
@@ -1,13 +0,0 @@
---
applyTo: ".github/**,cmake/**,Makefile,CMakeLists.txt,Tools/**,**/Kconfig"
---
# CI/Build Review Guidelines
In addition to the core code review guidelines:
- Check for pipeline race conditions (tag + branch push double-trigger, git describe correctness)
- Container image size: check for layer bloat
- Ubuntu LTS support policy: only latest + one prior LTS version
- Consider build time impact of changes
- Prefer CMake over Makefiles
@@ -1,32 +0,0 @@
---
applyTo: "src/**,boards/**,platforms/**,msg/**,cmake/**,Makefile,CMakeLists.txt,Tools/**,.github/**"
---
# PX4 Code Review Guidelines
## Conventions
- PR titles must follow conventional commits: `type(scope): description` (see CONTRIBUTING.md)
- Types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `build`, `ci`, `chore`, `revert`
- Scope should match the primary area of changed files
- Append `!` before the colon for breaking changes
- Both squash merge and rebase merge are enabled; merge commits are disabled
- Commits should be atomic and independently revertable
- WIP or review-response commits should be squashed before merge
## Core Checks (always apply)
- **Correctness**: logic errors, off-by-ones, unhandled edge cases
- **Type safety**: int16 overflow, float/double promotion, unsigned subtraction, use `uint64_t` for absolute time
- **Initialization**: uninitialized variables, missing default construction
- **Buffer safety**: unchecked array access, stack allocation of large buffers, snprintf bounds
- **Magic numbers**: every numeric literal needs a named constant or justification
- **Framework reuse**: use PX4_ERR/WARN/INFO, existing libraries (AlphaFilter, SlewRate, RateControl), MAVLink constants from the library
- **Naming**: accurate, no unjustified abbreviations, current terminology (GPS -> GNSS for new code)
- **Unnecessary complexity**: can code be removed instead of added? Is there a simpler pattern?
- **Test coverage**: new features should include unit or integration tests; bug fixes should include regression tests where practical
- **Formatting**: `make format` / `make check_format` (astyle) for C/C++ files; `clang-tidy` clean
- **Coding style**: C/C++ must follow the PX4 coding style (https://docs.px4.io/main/en/contribute/code.html)
- **Necessity**: challenge every addition. Is this actually needed or just copied?
- **Architecture fit**: does the code live in the module that naturally owns the data? No unnecessary cross-module dependencies
- **Ecosystem impact**: consider QGC users, log analysis tools, and third-party integrations
@@ -1,15 +0,0 @@
---
applyTo: "src/modules/mc_*control*/**,src/modules/fw_*control*/**,src/modules/flight_mode_manager/**,src/lib/rate_control/**,src/lib/npfg/**,src/modules/vtol_att_control/**"
---
# Control Review Guidelines
In addition to the core code review guidelines:
- Phase margin: output filters consume margin for no benefit; prefer adjusting gyro/d-gyro cutoffs
- Check for circular dependencies: sensor data feeding back into its own control loop (e.g., throttle-based airspeed in TECS)
- NaN propagation in flight-critical math; check `PX4_ISFINITE` before magnitude checks
- Prefer proper setpoint smoothing over controller output filtering (setpoint generation vs output-stage hacks)
- Check yaw control edge cases: heading lock, drift, setpoint propagation
- Verify flight task inheritance chain uses the correct base class for desired behavior
- Control allocation: verify actuator function ordering and motor index mapping
@@ -1,13 +0,0 @@
---
applyTo: "src/drivers/**,src/modules/cyphal/**"
---
# Drivers/CAN Review Guidelines
In addition to the core code review guidelines:
- CAN bus devices behave differently from serial/SPI; check driver assumptions
- ESC index mapping: telemetry index != channel when motors are disabled
- ESC hardware quirks: 4-in-1 ESCs may report current on only one channel
- Verify device_id correctness and I2CSPIDriver patterns
- Time representation: prefer `hrt_abstime` over iteration counts
@@ -1,13 +0,0 @@
---
applyTo: "src/modules/ekf2/**,src/lib/wind_estimator/**,src/lib/world_magnetic_model/**"
---
# Estimation Review Guidelines
In addition to the core code review guidelines:
- Check for singularities in aerospace math (euler angles near gimbal lock, sideslip at low airspeed)
- Flag aliasing from downsampling sensor data without proper filtering
- Verify Kalman filter correctness (Joseph form, innovation variance, covariance symmetry)
- Consider CPU cost on embedded targets (avoid unnecessary sqrt, limit fusion rate)
- Verify frame/coordinate system correctness (FRD vs NED, body vs earth frame)
@@ -1,13 +0,0 @@
---
applyTo: "msg/**,src/modules/mavlink/**,src/modules/uxrce_dds_client/**"
---
# Messages/Protocol Review Guidelines
In addition to the core code review guidelines:
- Backwards compatibility: will this break QGC, post-flight tools, or uLog parsers?
- uORB: `timestamp` for publication metadata, `timestamp_sample` close to physical sample, include `device_id`
- Don't version messages unless strictly needed
- Parameter UX: will this confuse users in a GCS? Every new param is a configuration burden
- MAVLink: use library constants, don't implement custom stream rates
@@ -1,13 +0,0 @@
---
applyTo: "src/modules/simulation/**,Tools/simulation/**"
---
# Simulation Review Guidelines
In addition to the core code review guidelines:
- Physics fidelity: noise models should match reality (GPS noise is not Gaussian)
- Keep gz_bridge generic; vehicle-specific logic belongs in plugins
- Prefer gz-transport over ROS2 dependencies when possible
- Use wrench commands for physics correctness vs kinematic constraints
- Library generic/specific boundary: only base classes in common libs
@@ -1,15 +0,0 @@
---
applyTo: "src/modules/commander/**,src/modules/logger/**,src/systemcmds/**,platforms/**,src/modules/dataman/**"
---
# System Review Guidelines
In addition to the core code review guidelines:
- Race conditions and concurrency: no partial fixes, demand complete solutions
- Semaphore/scheduling edge cases; understand RTOS guarantees
- State machine sequential-logic bugs (consecutive RTL, armed/disarmed alternation)
- Use uORB-driven scheduling (`SubscriptionCallback`), not extra threads
- `param_set` triggers auto-save; no redundant `param_save_default`
- Flash/memory efficiency: avoid `std::string` on embedded, minimize SubscriptionData usage
- Constructor initialization order matters
+50 -77
View File
@@ -69,31 +69,32 @@ jobs:
runs-on: [runs-on,runner=1cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false]
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
seeders: ${{ steps.set-seeders.outputs.seeders }}
timestamp: ${{ steps.set-timestamp.outputs.timestamp }}
branchname: ${{ steps.set-branch.outputs.branchname }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Cache Python pip
uses: actions/cache@v5
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**./Tools/setup/requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Update python packaging to avoid canonicalize_version() error
run: |
pip3 install -U packaging
- name: Install Python Dependencies
run: pip3 install -U packaging -r ./Tools/setup/requirements.txt
uses: py-actions/py-dependency-install@v4
with:
path: "./Tools/setup/requirements.txt"
- id: set-matrix
name: Generate Build Matrix
run: echo "matrix=$(./Tools/ci/generate_board_targets_json.py --group)" >> $GITHUB_OUTPUT
- id: set-seeders
name: Generate Seeder Matrix
run: echo "seeders=$(./Tools/ci/generate_board_targets_json.py --group --seeders)" >> $GITHUB_OUTPUT
- id: set-timestamp
name: Save Current Timestamp
run: echo "timestamp=$(date +"%Y%m%d%H%M%S")" >> $GITHUB_OUTPUT
@@ -115,52 +116,11 @@ jobs:
echo "${{ steps.set-branch.outputs.branchname }}"
echo "$(./Tools/ci/generate_board_targets_json.py --group --verbose)"
# ===========================================================================
# CACHE SEEDER JOBS
# ===========================================================================
# Build one representative target per chip family to warm the ccache.
# Matrix jobs fall back to these caches via restore-keys when no
# group-specific cache exists yet. If any seeder fails, the build matrix
# does not start, catching common build errors early.
# ===========================================================================
seed:
name: Seed [${{ matrix.chip_family }}]
needs: group_targets
runs-on: [runs-on,"runner=8cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
strategy:
matrix: ${{ fromJson(needs.group_targets.outputs.seeders) }}
fail-fast: false
container:
image: ${{ matrix.container }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 1
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-seeder
max-size: 400M
- name: Build seed target
run: make ${{ matrix.target }}
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
setup:
name: Build [${{ matrix.runner }}][${{ matrix.group }}]
# runs-on: ubuntu-latest
runs-on: [runs-on,"runner=4cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
needs: [group_targets, seed]
if: "!failure() && !cancelled()"
runs-on: [runs-on,"runner=8cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",spot=false]
needs: group_targets
strategy:
matrix: ${{ fromJson(needs.group_targets.outputs.matrix) }}
fail-fast: false
@@ -171,35 +131,41 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git Safe Directory
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Cache - Restore ccache
id: ccache-restore
uses: actions/cache/restore@v5
# ccache key breakdown:
# ccache-<system os>-<system arch>-<builder group>-
# ccache-<linux>-<arm64>-<aarch64-0>-
# ccache-<linux>-<x64>-<nuttx-0>-
- name: Cache Restore from Key
id: cc_restore
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-${{ matrix.group }}-${{ github.ref_name }}-${{ github.sha }}
key: ${{ format('ccache-{0}-{1}-{2}', runner.os, matrix.runner, matrix.group) }}
restore-keys: |
ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-${{ matrix.group }}-${{ github.ref_name }}-
ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-${{ matrix.group }}-${{ github.base_ref || 'main' }}-
ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-${{ matrix.group }}-
ccache-${{ matrix.chip_family }}-${{ matrix.runner }}-
ccache-${{ runner.os }}-${{ matrix.runner }}-${{ matrix.group }}-
ccache-${{ runner.os }}-${{ matrix.runner }}-
ccache-${{ runner.os }}-${{ matrix.runner }}-
ccache-${{ runner.os }}-
ccache-
- name: Cache - Configure ccache
- name: Cache Config and Stats
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = ${{ matrix.cache_size }}" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 120M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Building Artifacts for [${{ matrix.targets }}]
run: |
@@ -210,15 +176,23 @@ jobs:
./Tools/ci/package_build_artifacts.sh
- name: Upload Build Artifacts
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: px4_${{matrix.group}}_build_artifacts
path: artifacts/
- uses: ./.github/actions/save-ccache
if: success()
- name: Cache Post Build Stats
if: always()
run: |
ccache -s
ccache -z
- name: Cache Save
if: always()
uses: actions/cache/save@v4
with:
cache-primary-key: ${{ steps.ccache-restore.outputs.cache-primary-key }}
path: ~/.ccache
key: ${{ steps.cc_restore.outputs.cache-primary-key }}
# ===========================================================================
# ARTIFACT UPLOAD JOB
@@ -237,7 +211,7 @@ jobs:
uploadlocation: ${{ steps.upload-location.outputs.uploadlocation }}
steps:
- name: Download Artifacts
uses: actions/download-artifact@v8
uses: actions/download-artifact@v4
with:
path: artifacts/
merge-multiple: true
@@ -294,5 +268,4 @@ jobs:
files: |
artifacts/*.px4
artifacts/*.deb
artifacts/**/*.sbom.spdx.json
name: ${{ steps.upload-location.outputs.uploadlocation }}
-218
View File
@@ -1,218 +0,0 @@
name: SITL Packages and Containers
on:
push:
tags: ['v*']
pull_request:
paths:
- 'cmake/package.cmake'
- 'platforms/posix/CMakeLists.txt'
- 'Tools/packaging/**'
- 'boards/px4/sitl/sih.px4board'
- '.github/workflows/build_deb_package.yml'
- '.github/actions/build-deb/**'
workflow_dispatch:
inputs:
deploy_containers:
description: 'Push container images to registry'
required: false
type: boolean
default: false
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
packages: write
jobs:
# ---------------------------------------------------------------------------
# Setup: extract version and determine whether to push containers
# ---------------------------------------------------------------------------
setup:
name: Setup
runs-on: [runs-on,"runner=1cpu-linux-x64","image=ubuntu24-full-x64","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
outputs:
px4_version: ${{ steps.version.outputs.px4_version }}
should_push: ${{ steps.push.outputs.should_push }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 0
fetch-tags: true
- name: Set PX4 version
id: version
run: echo "px4_version=$(git describe --tags --match 'v[0-9]*')" >> $GITHUB_OUTPUT
- name: Check if we should push containers
id: push
run: |
if [[ "${{ startsWith(github.ref, 'refs/tags/') }}" == "true" ]] || \
[[ "${{ github.event_name }}" == "workflow_dispatch" && "${{ github.event.inputs.deploy_containers }}" == "true" ]]; then
echo "should_push=true" >> $GITHUB_OUTPUT
else
echo "should_push=false" >> $GITHUB_OUTPUT
fi
# ---------------------------------------------------------------------------
# Build .deb packages (all distros, arches, targets)
# ---------------------------------------------------------------------------
build-deb:
name: "Build .deb (${{ matrix.target }}/${{ matrix.codename }}/${{ matrix.arch }})"
needs: setup
runs-on: [runs-on,"runner=4cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
container:
image: ${{ matrix.container }}
volumes:
- /github/workspace:/github/workspace
strategy:
fail-fast: false
matrix:
include:
- { codename: noble, arch: amd64, runner: x64, container: "ubuntu:24.04", target: default, setup_flags: "" }
- { codename: noble, arch: arm64, runner: arm64, container: "ubuntu:24.04", target: default, setup_flags: "" }
- { codename: jammy, arch: amd64, runner: x64, container: "ubuntu:22.04", target: default, setup_flags: "" }
- { codename: jammy, arch: arm64, runner: arm64, container: "ubuntu:22.04", target: default, setup_flags: "" }
- { codename: noble, arch: amd64, runner: x64, container: "ubuntu:24.04", target: sih, setup_flags: "--no-sim-tools" }
- { codename: noble, arch: arm64, runner: arm64, container: "ubuntu:24.04", target: sih, setup_flags: "--no-sim-tools" }
- { codename: jammy, arch: amd64, runner: x64, container: "ubuntu:22.04", target: sih, setup_flags: "--no-sim-tools" }
- { codename: jammy, arch: arm64, runner: arm64, container: "ubuntu:22.04", target: sih, setup_flags: "--no-sim-tools" }
env:
RUNS_IN_DOCKER: true
steps:
- uses: runs-on/action@v2
- name: Fix git in container
run: |
apt-get update && apt-get install -y git
git config --global --add safe.directory $(realpath .)
- uses: actions/checkout@v6
with:
fetch-depth: 0
fetch-tags: true
- name: Use AWS regional apt mirror
if: startsWith(runner.name, 'runs-on--')
run: ./Tools/ci/use_aws_apt_mirror.sh
- name: Cache apt packages
uses: actions/cache@v5
with:
path: /var/cache/apt/archives
key: apt-${{ matrix.target }}-${{ matrix.codename }}-${{ matrix.arch }}-${{ hashFiles('Tools/setup/ubuntu.sh') }}
restore-keys: apt-${{ matrix.target }}-${{ matrix.codename }}-${{ matrix.arch }}-
- name: Install dependencies
run: ./Tools/setup/ubuntu.sh --no-nuttx ${{ matrix.setup_flags }}
- name: Build and package .deb
uses: ./.github/actions/build-deb
with:
target: ${{ matrix.target }}
artifact-name: px4-sitl-debs-${{ matrix.target }}-${{ matrix.codename }}-${{ matrix.arch }}
ccache-key-prefix: deb-ccache-${{ matrix.target }}-${{ matrix.codename }}-${{ matrix.arch }}
# ---------------------------------------------------------------------------
# Build Docker images from Noble .debs
# ---------------------------------------------------------------------------
build-docker:
name: "Build Image (${{ matrix.image }}/${{ matrix.arch }})"
needs: [setup, build-deb]
runs-on: [runs-on,"runner=4cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
strategy:
fail-fast: false
matrix:
include:
- { image: sih, repo: px4-sitl, target: sih, arch: amd64, runner: x64, platform: "linux/amd64", dockerfile: Dockerfile.sih }
- { image: sih, repo: px4-sitl, target: sih, arch: arm64, runner: arm64, platform: "linux/arm64", dockerfile: Dockerfile.sih }
- { image: gazebo, repo: px4-sitl-gazebo, target: default, arch: amd64, runner: x64, platform: "linux/amd64", dockerfile: Dockerfile.gazebo }
- { image: gazebo, repo: px4-sitl-gazebo, target: default, arch: arm64, runner: arm64, platform: "linux/arm64", dockerfile: Dockerfile.gazebo }
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 1
- name: Download Noble .deb artifact
uses: actions/download-artifact@v8
with:
name: px4-sitl-debs-${{ matrix.target }}-noble-${{ matrix.arch }}
path: docker-context
- name: Prepare build context
run: cp Tools/packaging/px4-entrypoint.sh docker-context/
- name: Login to registries
if: needs.setup.outputs.should_push == 'true'
run: |
echo "${{ secrets.DOCKERHUB_TOKEN }}" | docker login -u "${{ secrets.DOCKERHUB_USERNAME }}" --password-stdin
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
driver: docker-container
platforms: ${{ matrix.platform }}
- name: Build and push container image
uses: docker/build-push-action@v7
with:
context: docker-context
file: Tools/packaging/${{ matrix.dockerfile }}
tags: |
px4io/${{ matrix.repo }}:${{ needs.setup.outputs.px4_version }}-${{ matrix.arch }}
px4io/${{ matrix.repo }}:latest-${{ matrix.arch }}
ghcr.io/px4/${{ matrix.repo }}:${{ needs.setup.outputs.px4_version }}-${{ matrix.arch }}
ghcr.io/px4/${{ matrix.repo }}:latest-${{ matrix.arch }}
platforms: ${{ matrix.platform }}
load: false
push: ${{ needs.setup.outputs.should_push == 'true' }}
provenance: false
cache-from: type=gha,scope=sitl-${{ matrix.image }}-${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=sitl-${{ matrix.image }}-${{ matrix.arch }}
# ---------------------------------------------------------------------------
# Deploy: create multi-arch manifests and push to registries
# ---------------------------------------------------------------------------
deploy:
name: "Deploy (${{ matrix.image }})"
needs: [setup, build-docker]
if: needs.setup.outputs.should_push == 'true'
runs-on: [runs-on,"runner=1cpu-linux-x64","image=ubuntu24-full-x64","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
strategy:
matrix:
include:
- { image: sih, repo: px4-sitl }
- { image: gazebo, repo: px4-sitl-gazebo }
steps:
- uses: runs-on/action@v2
- name: Login to registries
run: |
echo "${{ secrets.DOCKERHUB_TOKEN }}" | docker login -u "${{ secrets.DOCKERHUB_USERNAME }}" --password-stdin
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
- name: Create and push multi-arch manifests
run: |
VERSION="${{ needs.setup.outputs.px4_version }}"
for REGISTRY in px4io ghcr.io/px4; do
IMAGE="${REGISTRY}/${{ matrix.repo }}"
for TAG in ${VERSION} latest; do
docker manifest create ${IMAGE}:${TAG} \
--amend ${IMAGE}:${TAG}-arm64 \
--amend ${IMAGE}:${TAG}-amd64
docker manifest annotate ${IMAGE}:${TAG} ${IMAGE}:${TAG}-arm64 --arch arm64
docker manifest annotate ${IMAGE}:${TAG} ${IMAGE}:${TAG}-amd64 --arch amd64
docker manifest push ${IMAGE}:${TAG}
done
done
+25 -64
View File
@@ -12,88 +12,49 @@ on:
paths-ignore:
- 'docs/**'
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
gate_checks:
name: Gate Checks [${{ matrix.check }}]
runs-on: [runs-on,runner=2cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
build:
runs-on: ubuntu-latest
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
fail-fast: true
fail-fast: false
matrix:
check: [
"check_format",
"check_newlines",
"tests",
"tests_coverage",
"px4_fmu-v2_default stack_check",
"validate_module_configs",
"shellcheck_all",
"NO_NINJA_BUILD=1 px4_fmu-v5_default",
"NO_NINJA_BUILD=1 px4_sitl_default",
"px4_sitl_allyes",
"module_documentation",
]
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
fetch-depth: 0
- name: Building [${{ matrix.check }}]
env:
PX4_SBOM_DISABLE: 1
run: make ${{ matrix.check }}
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
make ${{ matrix.check }}
tests:
name: Unit Tests
runs-on: [runs-on,runner=8cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
permissions:
contents: write
env:
GIT_COMMITTER_EMAIL: bot@px4.io
GIT_COMMITTER_NAME: PX4BuildBot
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- name: Uploading Coverage to Codecov.io
if: contains(matrix.check, 'coverage')
uses: codecov/codecov-action@v1
with:
fetch-depth: 1
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-sitl
max-size: 300M
- name: Build and run unit tests
env:
PX4_SBOM_DISABLE: 1
run: make tests
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: Auto-update EKF change indication baselines
if: github.event_name == 'push'
uses: stefanzweifel/git-auto-commit-action@v7
with:
file_pattern: 'src/modules/ekf2/test/change_indication/*.csv'
commit_user_name: ${{ env.GIT_COMMITTER_NAME }}
commit_user_email: ${{ env.GIT_COMMITTER_EMAIL }}
commit_message: |
[AUTO COMMIT] update EKF change indication
See .github/workflows/checks.yml for more details
- name: Check for EKF functional changes
run: git diff --exit-code
working-directory: src/modules/ekf2/test/change_indication
token: ${{ secrets.CODECOV_TOKEN }}
flags: unittests
file: coverage/lcov.info
+29 -121
View File
@@ -16,21 +16,14 @@ permissions:
contents: read
jobs:
# Push-to-main: unchanged historical behavior. Single clang build dir
# with BUILD_TESTING=OFF. `make clang-tidy` builds and analyzes every
# TU in compile_commands.json. Test files are not in the DB and
# therefore never analyzed.
clang_tidy_push:
clang_tidy:
name: Clang-Tidy
if: github.event_name != 'pull_request'
runs-on: [runs-on, runner=16cpu-linux-x64, "run-id=${{ github.run_id }}", "extras=s3-cache"]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
permissions:
contents: read
image: px4io/px4-dev:v1.17.0-beta1
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
@@ -38,124 +31,39 @@ jobs:
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
- name: Restore Compiler Cache
id: cc_restore
uses: actions/cache/restore@v4
with:
cache-key-prefix: ccache-clang-tidy
max-size: 150M
path: ~/.ccache
key: ccache-clang-tidy-${{ github.head_ref || github.ref_name }}
restore-keys: |
ccache-clang-tidy-${{ github.head_ref || github.ref_name }}-
ccache-clang-tidy-main-
ccache-clang-tidy-
- name: Build and Analyze - Clang-Tidy
run: make -j$(nproc) clang-tidy
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
# Pull request: diff-based analysis with a second BUILD_TESTING=ON
# build dir so test files in the PR diff can be linted by
# clang-tidy-diff-18.py with resolved gtest/fuzztest includes.
# Results are uploaded as a `pr-review` artifact for the PR Review
# Poster workflow to post as inline comments.
clang_tidy_pr:
name: Clang-Tidy
if: github.event_name == 'pull_request'
runs-on: [runs-on, runner=8cpu-linux-x64, "run-id=${{ github.run_id }}", "extras=s3-cache"]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
permissions:
contents: read
pull-requests: read
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 0
fetch-tags: true
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-clang-tidy
max-size: 150M
# fuzztest (enabled via BUILD_TESTING in the -test build dir) pulls
# in abseil via FetchContent, and abseil runs a try_compile with
# fuzztest's -fsanitize=address flags. The px4-dev container ships
# clang but not the clang compiler-rt runtime, so that link fails
# and the configure reports a misleading "pthreads not found".
# libclang-rt-18-dev provides libclang_rt.asan and friends.
- name: Install clang compiler-rt
- name: Configure Compiler Cache
run: |
apt-get update
apt-get install -y --no-install-recommends libclang-rt-18-dev
# `make clang-ci` prepares both clang build directories:
# - build/px4_sitl_default-clang: full build, BUILD_TESTING=OFF
# (used by run-clang-tidy-pr.py for whole-file analysis of
# changed production code)
# - build/px4_sitl_default-clang-test: configure-only, BUILD_TESTING=ON
# (used by clang-tidy-diff-18.py so test files are in the
# compilation database with resolved gtest/fuzztest includes)
- name: Build clang SITL
run: make -j$(nproc) clang-ci
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 120M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Run Clang-Tidy Analysis
run: python3 Tools/ci/run-clang-tidy-pr.py origin/${{ github.base_ref }}
run: make -j16 clang-tidy
# Produce a `pr-review` artifact for the PR Review Poster workflow
# to consume. clang-tidy-diff-18 emits a unified fixes.yml that
# the producer script translates into line-anchored review comments.
- name: Export clang-tidy fixes for PR review
- name: Compiler Cache Stats
if: always()
run: |
mkdir -p pr-review
# Drop changed C/C++ source files that are not in
# compile_commands.json for the test-enabled build. Files not
# in the DB are platform-specific sources, vendored code, or
# submodule code we don't own. Feeding them to clang-tidy-diff
# produces false positives from unresolved headers.
python3 Tools/ci/clang-tidy-diff-filter.py \
--build-dir build/px4_sitl_default-clang-test \
--base-ref origin/${{ github.base_ref }} \
--out pr-review/diff.patch
if [ -s pr-review/diff.patch ]; then
clang-tidy-diff-18.py -p1 \
-path build/px4_sitl_default-clang-test \
-export-fixes pr-review/fixes.yml \
-j0 < pr-review/diff.patch || true
else
echo "No analyzable files in diff; skipping clang-tidy-diff"
fi
run: ccache -s
- name: Build pr-review artifact
- name: Save Compiler Cache
if: always()
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python3 Tools/ci/clang-tidy-fixes-to-review.py \
--fixes pr-review/fixes.yml \
--repo-root "$GITHUB_WORKSPACE" \
--repo "$GITHUB_REPOSITORY" \
--pr-number "${{ github.event.pull_request.number }}" \
--commit-sha "${{ github.event.pull_request.head.sha }}" \
--out-dir pr-review \
--event COMMENT
- name: Upload pr-review artifact
if: always()
uses: actions/upload-artifact@v7
uses: actions/cache/save@v4
with:
name: pr-review
path: |
pr-review/manifest.json
pr-review/comments.json
retention-days: 1
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
path: ~/.ccache
key: ${{ steps.cc_restore.outputs.cache-primary-key }}
+3 -3
View File
@@ -23,7 +23,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
@@ -56,7 +56,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
@@ -102,7 +102,7 @@ jobs:
steps:
- name: Checkout
if: env.IS_FORK == 'false'
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
sparse-checkout: Tools/ci
fetch-depth: 1
+36 -36
View File
@@ -19,49 +19,49 @@ concurrency:
jobs:
build:
runs-on: macos-latest
strategy:
matrix:
config: [
px4_fmu-v5_default,
px4_sitl
]
steps:
- name: install Python 3.10
uses: actions/setup-python@v6
uses: actions/setup-python@v5
with:
python-version: "3.10"
- uses: actions/checkout@v6
- name: Cache - Restore Homebrew Packages
uses: actions/cache@v5
with:
path: ~/Library/Caches/Homebrew/downloads
key: macos-homebrew-${{ runner.arch }}-${{ hashFiles('Tools/setup/macos.sh') }}
restore-keys: |
macos-homebrew-${{ runner.arch }}-
- name: Cache - Restore pip Packages
uses: actions/cache@v5
with:
path: ~/Library/Caches/pip
key: macos-pip-${{ runner.arch }}-${{ hashFiles('Tools/setup/requirements.txt') }}
restore-keys: |
macos-pip-${{ runner.arch }}-
- uses: actions/checkout@v4
- name: setup
run: ./Tools/setup/macos.sh
run: |
./Tools/setup/macos.sh
- uses: ./.github/actions/setup-ccache
id: ccache
- name: Prepare ccache timestamp
id: ccache_cache_timestamp
shell: cmake -P {0}
run: |
string(TIMESTAMP current_date "%Y-%m-%d-%H;%M;%S" UTC)
file(APPEND "$ENV{GITHUB_OUTPUT}" "timestamp=${current_date}\n")
- name: ccache cache files
uses: actions/cache@v4
with:
cache-key-prefix: ccache-macos
max-size: 200M
path: ~/.ccache
key: macos_${{matrix.config}}-ccache-${{steps.ccache_cache_timestamp.outputs.timestamp}}
restore-keys: macos_${{matrix.config}}-ccache-
- name: setup ccache
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 40M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Build px4_sitl
run: make px4_sitl
- name: Cache - Stats after px4_sitl
run: ccache -s
- name: Build px4_fmu-v5_default
run: make px4_fmu-v5_default
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: make ${{matrix.config}}
run: |
ccache -z
make ${{matrix.config}}
ccache -s
+8 -26
View File
@@ -29,13 +29,12 @@ jobs:
fail-fast: false
matrix:
version: ['ubuntu:22.04', 'ubuntu:24.04']
runs-on: [runs-on,runner=4cpu-linux-x64,"image=ubuntu24-full-x64","run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: [runs-on,runner=4cpu-linux-x64,"image=ubuntu24-full-x64","run-id=${{ github.run_id }}",spot=false]
container:
image: ${{ matrix.version }}
volumes:
- /github/workspace:/github/workspace
steps:
- uses: runs-on/action@v2
- name: Fix git in container
run: |
@@ -48,28 +47,11 @@ jobs:
apt update && apt install git -y
git config --global --add safe.directory $(realpath .)
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Use AWS regional apt mirror
if: startsWith(runner.name, 'runs-on--')
run: ./Tools/ci/use_aws_apt_mirror.sh
- name: Install Deps
run: ./Tools/setup/ubuntu.sh
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-ubuntu-${{ matrix.version }}
max-size: 200M
- name: Build px4_sitl_default
run: make px4_sitl_default
- name: Build px4_fmu-v5_default
run: make px4_fmu-v5_default
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: Install Deps, Build, and Make Quick Check
run: |
# we need to install dependencies and build on the same step
# given the stateless nature of docker images
./Tools/setup/ubuntu.sh
make quick_check
+25 -33
View File
@@ -24,11 +24,6 @@ on:
description: 'Container tag (e.g. v1.16.0)'
required: true
type: string
build_ref:
description: 'Git ref to build from (branch, tag, or SHA). Leave empty to build from the dispatch ref.'
required: false
type: string
default: ''
deploy_to_registry:
description: 'Whether to push built images to the registry'
required: false
@@ -50,12 +45,12 @@ jobs:
meta_tags: ${{ steps.meta.outputs.tags }}
meta_labels: ${{ steps.meta.outputs.labels }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.build_ref || github.ref }}
fetch-depth: 0
fetch-tags: true
submodules: false
fetch-depth: 0
# If manual dispatch, take the userprovided input
- name: Set PX4 Tag Version
@@ -69,7 +64,7 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
uses: docker/metadata-action@v5
with:
images: |
ghcr.io/PX4/px4-dev
@@ -94,22 +89,22 @@ jobs:
runner: x64
runs-on: [runs-on,"runner=4cpu-linux-${{ matrix.runner }}","image=ubuntu24-full-${{ matrix.runner }}","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.build_ref || github.ref }}
fetch-depth: 0
fetch-tags: true
submodules: false
fetch-depth: 0
- name: Login to Docker Hub
uses: docker/login-action@v4
uses: docker/login-action@v3
if: ${{ startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry) }}
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
uses: docker/login-action@v3
if: ${{ startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry) }}
with:
registry: ghcr.io
@@ -117,13 +112,13 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
uses: docker/setup-buildx-action@v3
with:
driver: docker-container
platforms: ${{ matrix.platform }}
- name: Build and Load Container Image
uses: docker/build-push-action@v7
uses: docker/build-push-action@v6
id: docker
with:
context: Tools/setup
@@ -136,7 +131,7 @@ jobs:
push: ${{ startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry) }}
provenance: false
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }},ignore-error=true
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
deploy:
name: Deploy To Registry
@@ -145,26 +140,23 @@ jobs:
packages: write
runs-on: [runs-on,"runner=4cpu-linux-x64","image=ubuntu24-full-x64","run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
needs: [build, setup]
if: |
!cancelled() &&
needs.setup.result == 'success' &&
(startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry == 'true'))
if: ${{ startsWith(github.ref, 'refs/tags/') || (github.event_name == 'workflow_dispatch' && github.event.inputs.deploy_to_registry) }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.build_ref || github.ref }}
fetch-depth: 0
fetch-tags: true
submodules: false
fetch-depth: 0
- name: Login to Docker Hub
uses: docker/login-action@v4
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v4
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
@@ -172,10 +164,10 @@ jobs:
- name: Verify Images Exist Before Creating Manifest
run: |
docker manifest inspect px4io/px4-dev:${{ needs.setup.outputs.px4_version }}-arm64
docker manifest inspect px4io/px4-dev:${{ needs.setup.outputs.px4_version }}-amd64
docker manifest inspect ghcr.io/px4/px4-dev:${{ needs.setup.outputs.px4_version }}-arm64
docker manifest inspect ghcr.io/px4/px4-dev:${{ needs.setup.outputs.px4_version }}-amd64
docker manifest inspect px4io/px4-dev:${{ needs.setup.outputs.px4_version }}-arm64 || echo "⚠️ Warning: No ARM64 image found!"
docker manifest inspect px4io/px4-dev:${{ needs.setup.outputs.px4_version }}-amd64 || echo "⚠️ Warning: No AMD64 image found!"
docker manifest inspect ghcr.io/px4/px4-dev:${{ needs.setup.outputs.px4_version }}-arm64 || echo "⚠️ Warning: No ARM64 image found!"
docker manifest inspect ghcr.io/px4/px4-dev:${{ needs.setup.outputs.px4_version }}-amd64 || echo "⚠️ Warning: No AMD64 image found!"
- name: Create and Push Multi-Arch Manifest for Docker Hub
run: |
+36 -50
View File
@@ -46,8 +46,8 @@ jobs:
source_changed: ${{ steps.changes.outputs.source }}
docs_changed: ${{ steps.changes.outputs.docs }}
steps:
- uses: actions/checkout@v6
- uses: dorny/paths-filter@v4
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: changes
with:
filters: |
@@ -70,19 +70,22 @@ jobs:
contents: read
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.17.0-beta1
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Configure Git Safe Directory
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Cache Restore - ccache
id: cache-ccache
uses: actions/cache/restore@v5
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
@@ -101,7 +104,7 @@ jobs:
CCACHE_DIR: ~/.ccache
- name: Cache Save - ccache
uses: actions/cache/save@v5
uses: actions/cache/save@v4
if: always()
with:
path: ~/.ccache
@@ -113,7 +116,7 @@ jobs:
CCACHE_DIR: ~/.ccache
- name: Upload metadata artifact
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: pr-metadata
path: docs/
@@ -129,12 +132,12 @@ jobs:
contents: write
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.17.0-beta1
steps:
- uses: runs-on/action@v2
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
@@ -145,7 +148,7 @@ jobs:
- name: Cache Restore - ccache
id: cache-ccache
uses: actions/cache/restore@v5
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-docs-metadata-${{ github.sha }}
@@ -164,7 +167,7 @@ jobs:
CCACHE_DIR: ~/.ccache
- name: Cache Save - ccache
uses: actions/cache/save@v5
uses: actions/cache/save@v4
if: always()
with:
path: ~/.ccache
@@ -210,24 +213,25 @@ jobs:
if: always() && (github.event_name == 'pull_request')
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Download metadata artifact
if: needs.pr-metadata-regen.result == 'success'
uses: actions/download-artifact@v8
uses: actions/download-artifact@v4
with:
name: pr-metadata
path: docs/
- name: Get changed doc files
id: changed-files
uses: tj-actions/changed-files@v47
uses: tj-actions/changed-files@v46.0.5
with:
json: true
write_output_files: true
@@ -244,7 +248,7 @@ jobs:
cat ./logs/prFiles.json
- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@v4
with:
node-version: 20
@@ -277,35 +281,15 @@ jobs:
> ./logs/link-check-results.md || true
cat ./logs/link-check-results.md
- name: Prepare pr-comment artifact
id: prepare-pr-comment
run: |
if [ ! -s ./logs/filtered-link-check-results.md ]; then
echo "No link-check results file; skipping pr-comment artifact."
echo "prepared=false" >> "$GITHUB_OUTPUT"
exit 0
fi
mkdir -p pr-comment
cp ./logs/filtered-link-check-results.md pr-comment/body.md
cat > pr-comment/manifest.json <<EOF
{
"pr_number": ${{ github.event.pull_request.number }},
"marker": "<!-- pr-comment-poster:docs-link-check -->",
"mode": "upsert"
}
EOF
echo "prepared=true" >> "$GITHUB_OUTPUT"
- name: Upload pr-comment artifact
if: steps.prepare-pr-comment.outputs.prepared == 'true'
uses: actions/upload-artifact@v7
- name: Post PR comment with link check results
if: github.event.pull_request.head.repo.full_name == github.repository
uses: marocchino/sticky-pull-request-comment@v2
with:
name: pr-comment
path: pr-comment/
retention-days: 1
header: flaws
path: ./logs/filtered-link-check-results.md
- name: Upload link check results
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: link-check-results
path: logs/
@@ -329,14 +313,16 @@ jobs:
branchname: ${{ steps.set-branch.outputs.branchname }}
releaseversion: ${{ steps.set-version.outputs.releaseversion }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
- name: Download metadata artifact (PR)
if: github.event_name == 'pull_request' && needs.pr-metadata-regen.result == 'success'
uses: actions/download-artifact@v8
uses: actions/download-artifact@v4
with:
name: pr-metadata
path: docs/
@@ -360,7 +346,7 @@ jobs:
echo "releaseversion=$version" >> $GITHUB_OUTPUT
- name: Setup Node
uses: actions/setup-node@v6
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
@@ -379,7 +365,7 @@ jobs:
npm run docs:sitemap
- name: Upload artifact
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: px4_docs_build
path: docs/.vitepress/dist/
@@ -401,7 +387,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Download Artifact
uses: actions/download-artifact@v8
uses: actions/download-artifact@v4
with:
name: px4_docs_build
path: ~/_book
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
lc: [ko, uk, zh-CN] # Target languages https://developer.crowdin.com/language-codes/
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Debug Environment Variables
run: |
echo "CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_DOCS_PROJECT_ID }}"
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: crowdin push
uses: crowdin/github-action@v2
with:
+6 -5
View File
@@ -22,11 +22,12 @@ jobs:
build:
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",extras=s3-cache,spot=false]
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: runs-on/action@v1
- name: Checkout
uses: actions/checkout@v4
- name: Setup Node
uses: actions/setup-node@v6
uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
@@ -45,7 +46,7 @@ jobs:
- name: Upload artifact
if: ${{ github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.merged) || github.event_name == 'workflow_dispatch' }}
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: px4_docs_build
path: docs/.vitepress/dist/
@@ -58,7 +59,7 @@ jobs:
steps:
- name: Download Artifact
uses: actions/download-artifact@v8
uses: actions/download-artifact@v4
with:
name: px4_docs_build
path: ~/_book
@@ -0,0 +1,35 @@
name: EKF Change Indicator
on:
pull_request:
branches:
- '**'
paths-ignore:
- 'docs/**'
# If two events are triggered within a short time in the same PR, cancel the run of the oldest event
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
unit_tests:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: main test
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
make tests TESTFILTER=EKF
- name: Check if there is a functional change
run: git diff --exit-code
working-directory: src/modules/ekf2/test/change_indication
@@ -0,0 +1,54 @@
name: EKF Update Change Indicator
on:
push:
paths-ignore:
- 'docs/**'
jobs:
unit_tests:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
env:
GIT_COMMITTER_EMAIL: bot@px4.io
GIT_COMMITTER_NAME: PX4BuildBot
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: main test
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
make tests TESTFILTER=EKF
- name: Check if there exists diff and save result in variable
id: diff-check
working-directory: src/modules/ekf2/test/change_indication
run: |
if git diff --quiet; then
echo "CHANGE_INDICATED=false" >> $GITHUB_OUTPUT
else
echo "CHANGE_INDICATED=true" >> $GITHUB_OUTPUT
fi
- name: auto-commit any changes to change indication
if: steps.diff-check.outputs.CHANGE_INDICATED == 'true'
uses: stefanzweifel/git-auto-commit-action@v4
with:
file_pattern: 'src/modules/ekf2/test/change_indication/*.csv'
commit_user_name: ${{ env.GIT_COMMITTER_NAME }}
commit_user_email: ${{ env.GIT_COMMITTER_EMAIL }}
commit_message: |
[AUTO COMMIT] update change indication
See .github/workflows/ekf_update_change_indicator.yml for more details
- name: if there is a functional change, fail check
if: steps.diff-check.outputs.CHANGE_INDICATED == 'true'
run: exit 1
+8 -16
View File
@@ -18,7 +18,7 @@ concurrency:
jobs:
build:
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: ubuntu-latest
defaults:
run:
shell: bash
@@ -29,30 +29,22 @@ jobs:
"failsafe_web",
]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
options: --privileged --ulimit core=-1 --security-opt seccomp=unconfined
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- name: Install Node v20.18.0
uses: actions/setup-node@v6
uses: actions/setup-node@v4
with:
node-version: 20.18.0
- name: Cache - Restore Emscripten SDK
id: cache-emsdk
uses: actions/cache@v5
- uses: actions/checkout@v4
with:
path: _emscripten_sdk
key: emsdk-4.0.15
fetch-depth: 0
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Install empscripten
if: steps.cache-emsdk.outputs.cache-hit != 'true'
run: |
git clone https://github.com/emscripten-core/emsdk.git _emscripten_sdk
cd _emscripten_sdk
+35 -106
View File
@@ -24,9 +24,9 @@ env:
jobs:
analyze_flash:
name: Analyzing ${{ matrix.target }}
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
matrix:
target: [px4_fmu-v5x, px4_fmu-v6x]
@@ -36,58 +36,25 @@ jobs:
px4_fmu-v6x-bloaty-output: ${{ steps.gen-output.outputs.px4_fmu-v6x-bloaty-output }}
px4_fmu-v6x-bloaty-summary-map: ${{ steps.gen-output.outputs.px4_fmu-v6x-bloaty-summary-map }}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Configure Git Safe Directory
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Cache - Restore ccache (current)
id: cache_current
uses: actions/cache/restore@v5
with:
path: ~/.ccache
key: ccache-flash-${{ matrix.target }}-current-${{ github.ref_name }}-${{ github.sha }}
restore-keys: |
ccache-flash-${{ matrix.target }}-current-${{ github.ref_name }}-
ccache-flash-${{ matrix.target }}-current-
- name: Cache - Configure ccache
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 200M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
echo "compiler_check = content" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Build Target
run: make ${{ matrix.target }}_flash-analysis
- name: Store the ELF with the change
run: cp ./build/**/*.elf ./with-change.elf
- name: Cache - Stats after Current Build
run: ccache -s
- name: Cache - Save ccache (current)
if: always()
uses: actions/cache/save@v5
with:
path: ~/.ccache
key: ${{ steps.cache_current.outputs.cache-primary-key }}
- name: Clean previous build
run: |
make clean
make distclean
make submodulesclean
ccache -C
- name: If it's a PR checkout the base branch
if: ${{ github.event.pull_request }}
@@ -101,34 +68,12 @@ jobs:
- name: Update submodules
run: make submodulesupdate
- name: Cache - Restore ccache (baseline)
id: cache_baseline
uses: actions/cache/restore@v5
with:
path: ~/.ccache
key: ccache-flash-${{ matrix.target }}-baseline-${{ github.sha }}
restore-keys: |
ccache-flash-${{ matrix.target }}-baseline-
- name: Cache - Reset ccache stats
run: ccache -z
- name: Build
run: make ${{ matrix.target }}_flash-analysis
- name: Store the ELF before the change
run: cp ./build/**/*.elf ./before-change.elf
- name: Cache - Stats after Baseline Build
run: ccache -s
- name: Cache - Save ccache (baseline)
if: always()
uses: actions/cache/save@v5
with:
path: ~/.ccache
key: ${{ steps.cache_baseline.outputs.cache-primary-key }}
- name: bloaty-action
uses: PX4/bloaty-action@v1.0.0
id: bloaty-step
@@ -148,6 +93,9 @@ jobs:
echo '${{ steps.bloaty-step.outputs.bloaty-summary-map }}' >> $GITHUB_OUTPUT
echo "$EOF" >> $GITHUB_OUTPUT
# TODO:
# This part of the workflow is causing errors for forks. We should find a way to fix this and enable it again for forks.
# Track this issue https://github.com/PX4/PX4-Autopilot/issues/24408
post_pr_comment:
name: Publish Results
runs-on: [runs-on,runner=1cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}"]
@@ -157,22 +105,22 @@ jobs:
V5X-SUMMARY-MAP-PERC: ${{ fromJSON(fromJSON(needs.analyze_flash.outputs.px4_fmu-v5x-bloaty-summary-map).vm-percentage) }}
V6X-SUMMARY-MAP-ABS: ${{ fromJSON(fromJSON(needs.analyze_flash.outputs.px4_fmu-v6x-bloaty-summary-map).vm-absolute) }}
V6X-SUMMARY-MAP-PERC: ${{ fromJSON(fromJSON(needs.analyze_flash.outputs.px4_fmu-v6x-bloaty-summary-map).vm-percentage) }}
if: github.event.pull_request
if: github.event.pull_request && github.event.pull_request.head.repo.full_name == github.repository
steps:
- name: Find Comment
uses: peter-evans/find-comment@v4
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- pr-comment-poster:flash-analysis -->'
body-includes: FLASH Analysis
- name: Set Build Time
id: bt
run: |
echo "timestamp=$(date +'%Y-%m-%dT%H:%M:%S')" >> $GITHUB_OUTPUT
- name: Write pr-comment artifact
- name: Create or update comment
# This can't be moved to the job-level conditions, as GH actions don't allow a job-level if condition to access the env.
if: |
steps.fc.outputs.comment-id != '' ||
@@ -180,46 +128,27 @@ jobs:
env.V5X-SUMMARY-MAP-ABS <= fromJSON(env.MIN_FLASH_NEG_DIFF_FOR_COMMENT) ||
env.V6X-SUMMARY-MAP-ABS >= fromJSON(env.MIN_FLASH_POS_DIFF_FOR_COMMENT) ||
env.V6X-SUMMARY-MAP-ABS <= fromJSON(env.MIN_FLASH_NEG_DIFF_FOR_COMMENT)
run: |
mkdir -p pr-comment
cat > pr-comment/manifest.json <<EOF
{
"pr_number": ${{ github.event.pull_request.number }},
"marker": "<!-- pr-comment-poster:flash-analysis -->",
"mode": "upsert"
}
EOF
cat > pr-comment/body.md <<'PR_COMMENT_BODY_EOF'
<!-- pr-comment-poster:flash-analysis -->
## 🔎 FLASH Analysis
<details>
<summary>px4_fmu-v5x [Total VM Diff: ${{ env.V5X-SUMMARY-MAP-ABS }} byte (${{ env.V5X-SUMMARY-MAP-PERC}} %)]</summary>
```
${{ needs.analyze_flash.outputs.px4_fmu-v5x-bloaty-output }}
```
</details>
<details>
<summary>px4_fmu-v6x [Total VM Diff: ${{ env.V6X-SUMMARY-MAP-ABS }} byte (${{ env.V6X-SUMMARY-MAP-PERC }} %)]</summary>
```
${{ needs.analyze_flash.outputs.px4_fmu-v6x-bloaty-output }}
```
</details>
**Updated: _${{ steps.bt.outputs.timestamp }}_**
PR_COMMENT_BODY_EOF
- name: Upload pr-comment artifact
if: |
steps.fc.outputs.comment-id != '' ||
env.V5X-SUMMARY-MAP-ABS >= fromJSON(env.MIN_FLASH_POS_DIFF_FOR_COMMENT) ||
env.V5X-SUMMARY-MAP-ABS <= fromJSON(env.MIN_FLASH_NEG_DIFF_FOR_COMMENT) ||
env.V6X-SUMMARY-MAP-ABS >= fromJSON(env.MIN_FLASH_POS_DIFF_FOR_COMMENT) ||
env.V6X-SUMMARY-MAP-ABS <= fromJSON(env.MIN_FLASH_NEG_DIFF_FOR_COMMENT)
uses: actions/upload-artifact@v7
uses: peter-evans/create-or-update-comment@v4
with:
name: pr-comment
path: pr-comment/
retention-days: 1
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: |
## 🔎 FLASH Analysis
<details>
<summary>px4_fmu-v5x [Total VM Diff: ${{ env.V5X-SUMMARY-MAP-ABS }} byte (${{ env.V5X-SUMMARY-MAP-PERC}} %)]</summary>
```
${{ needs.analyze_flash.outputs.px4_fmu-v5x-bloaty-output }}
```
</details>
<details>
<summary>px4_fmu-v6x [Total VM Diff: ${{ env.V6X-SUMMARY-MAP-ABS }} byte (${{ env.V6X-SUMMARY-MAP-PERC }} %)]</summary>
```
${{ needs.analyze_flash.outputs.px4_fmu-v6x-bloaty-output }}
```
</details>
**Updated: _${{ steps.bt.outputs.timestamp }}_**
edit-mode: replace
+9 -19
View File
@@ -12,25 +12,19 @@ env:
jobs:
Fuzzing:
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc2-4-gb67c65bfe6
steps:
- uses: runs-on/action@v2
- name: Install Dependencies
run: |
apt update && apt install -y clang
- uses: actions/checkout@v6
- name: Fix git in Container
run: |
git config --global --add safe.directory $(realpath .)
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- name: Install clang
run: apt-get update && apt-get install -y clang
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-sitl
max-size: 300M
- uses: actions/checkout@v4
- name: Build and Run Fuzz Tests
run: |
@@ -44,11 +38,7 @@ jobs:
./Tools/ci/run_fuzz_tests.sh $fuzz_binary 15m
done
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
# Create a github issue in case of a failure
- name: Create Issue
if: ${{ failure() }}
uses: JasonEtco/create-an-issue@v2
+7 -17
View File
@@ -22,9 +22,9 @@ concurrency:
jobs:
check_itcm:
name: Checking ${{ matrix.target }}
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false]
container:
image: ghcr.io/px4/px4-dev:v1.17.0-rc2
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
fail-fast: false
matrix:
@@ -46,18 +46,13 @@ jobs:
boards/nxp/mr-tropic/nuttx-config/scripts/itcm_functions_includes.ld
boards/nxp/mr-tropic/nuttx-config/scripts/itcm_static_functions.ld
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
fetch-depth: 0
submodules: recursive
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-itcm-${{ matrix.target }}
max-size: 200M
- name: Git ownership workaround
run: git config --system --add safe.directory '*'
- name: Build Target
run: make ${{ matrix.target }}
@@ -70,8 +65,3 @@ jobs:
- name: Execute the itcm-check
run: python3 Tools/itcm_check.py --elf-file built.elf --script-files ${{ matrix.scripts }}
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
+1 -1
View File
@@ -16,6 +16,6 @@ jobs:
pull-requests: write
steps:
- uses: actions/labeler@v6
- uses: actions/labeler@v5
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
@@ -0,0 +1,45 @@
name: MAVROS Mission Tests
on:
push:
branches:
- 'main'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- '**'
paths-ignore:
- 'docs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build SITL and Run Tests (inside old ROS container)
run: |
docker run --rm \
-v "${GITHUB_WORKSPACE}:/workspace" \
-w /workspace \
px4io/px4-dev-ros-melodic:2021-09-08 \
bash -c '
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh \
mavros_posix_test_mission.test \
mission:=MC_mission_box \
vehicle:=iris
'
@@ -0,0 +1,44 @@
name: MAVROS Offboard Tests
on:
push:
branches:
- 'main'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- '**'
paths-ignore:
- 'docs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build SITL and Run Tests (inside old ROS container)
run: |
docker run --rm \
-v "${GITHUB_WORKSPACE}:/workspace" \
-w /workspace \
px4io/px4-dev-ros-melodic:2021-09-08 \
bash -c '
git config --global --add safe.directory /workspace
make px4_sitl_default
make px4_sitl_default sitl_gazebo-classic
./test/rostest_px4_run.sh \
mavros_posix_tests_offboard_posctl.test \
vehicle:=iris
'
-73
View File
@@ -1,73 +0,0 @@
name: MAVROS Tests
on:
push:
branches:
- 'main'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- '**'
paths-ignore:
- 'docs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
name: "MAVROS ${{ matrix.config.name }}"
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
permissions:
contents: read
container:
image: px4io/px4-dev-ros-noetic:2021-09-08
env:
PX4_SBOM_DISABLE: 1
strategy:
fail-fast: false
matrix:
config:
- {name: "Mission", test_file: "mavros_posix_test_mission.test", params: "mission:=MC_mission_box vehicle:=iris"}
- {name: "Offboard", test_file: "mavros_posix_tests_offboard_posctl.test", params: "vehicle:=iris"}
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
with:
fetch-depth: 1
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- name: Setup - Install Python Test Dependencies
run: pip3 install -r Tools/setup/requirements.txt
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-sitl-gazebo-classic
max-size: 350M
- uses: ./.github/actions/build-gazebo-sitl
- name: Test - MAVROS ${{ matrix.config.name }}
run: |
./test/rostest_px4_run.sh \
${{ matrix.config.test_file }} \
${{ matrix.config.params }}
timeout-minutes: 10
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: Upload - Failed Test Logs
if: failure()
uses: actions/upload-artifact@v7
with:
name: failed-mavros-${{ matrix.config.name }}-logs.zip
path: |
logs/**/**/**/*.log
logs/**/**/**/*.ulg
+46
View File
@@ -0,0 +1,46 @@
name: Nuttx Target with extra env config
on:
push:
branches:
- 'main'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- '**'
paths-ignore:
- 'docs/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: ubuntu-latest
container:
image: px4io/px4-dev:v1.16.0-rc1-258-g0369abd556
strategy:
matrix:
config:
- px4_fmu-v5_default
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Build PX4 and Run Test [${{ matrix.config }}]
run: |
cd "$GITHUB_WORKSPACE"
git config --global --add safe.directory "$GITHUB_WORKSPACE"
export PX4_EXTRA_NUTTX_CONFIG='CONFIG_NSH_LOGIN_PASSWORD="test";CONFIG_NSH_CONSOLE_LOGIN=y'
echo "PX4_EXTRA_NUTTX_CONFIG: $PX4_EXTRA_NUTTX_CONFIG"
make ${{ matrix.config }} nuttx_context
echo "Check that the config option is set"
grep CONFIG_NSH_LOGIN_PASSWORD build/${{ matrix.config }}/NuttX/nuttx/.config
-155
View File
@@ -1,155 +0,0 @@
name: PR Comment Poster
# Generic PR comment poster. Any analysis workflow (clang-tidy, flash_analysis,
# fuzz coverage, SITL perf, etc.) can produce a `pr-comment` artifact and this
# workflow will post or update a sticky PR comment with its contents. Designed
# so that analysis jobs running on untrusted fork PRs can still get their
# results posted back to the PR.
#
# ==============================================================================
# SECURITY INVARIANTS
# ==============================================================================
# This workflow runs on `workflow_run` which means it runs in the BASE REPO
# context with a WRITE token, even when the triggering PR comes from a fork.
# That is the entire reason it exists, and also the reason it is a loaded
# footgun. Anyone modifying this file MUST preserve the following invariants:
#
# 1. NEVER check out PR code. No `actions/checkout` with a ref. No git clone
# of a fork branch. No execution of scripts from the downloaded artifact.
# The ONLY things read from the artifact are `manifest.json` and `body.md`,
# and both are treated as opaque data (JSON parsed by the poster script
# and markdown posted verbatim via the GitHub API).
#
# 2. `pr_number` is validated to be a positive integer before use.
# `marker` is validated to be printable ASCII only before use. Validation
# happens inside Tools/ci/pr-comment-poster.py which is checked out from
# the base branch, not from the artifact.
#
# 3. The comment body is passed to the GitHub API as a JSON field, never
# interpolated into a shell command string.
#
# 4. This workflow file lives on the default branch. `workflow_run` only
# loads workflow files from the default branch, so a fork cannot modify
# THIS file as part of a PR. The fork CAN cause this workflow to fire
# by triggering a producer workflow that uploads a `pr-comment` artifact.
# That is intended.
#
# 5. The artifact-name filter (`pr-comment`) is the only gate on which
# workflow runs get processed. Any workflow in this repo that uploads
# an artifact named `pr-comment` is trusted to have written the
# manifest and body itself, NOT copied fork-controlled content into
# them. Producer workflows are responsible for that.
#
# 6. `actions/checkout@v6` below uses NO ref (so it pulls the base branch,
# the default-branch commit this workflow file was loaded from) AND uses
# sparse-checkout to materialize ONLY Tools/ci/pr-comment-poster.py and
# its stdlib-only helper module Tools/ci/_github_helpers.py. The rest of
# the repo never touches the workspace. This is safe: the only files the
# job executes are base-repo Python scripts that were reviewed through
# normal code review, never anything from the PR.
#
# ==============================================================================
# ARTIFACT CONTRACT
# ==============================================================================
# Producers upload an artifact named exactly `pr-comment` containing:
#
# manifest.json:
# {
# "pr_number": 12345, // required, int > 0
# "marker": "<!-- pr-comment-poster:flash-analysis -->", // required, printable ASCII
# "mode": "upsert" // optional, default "upsert"
# }
#
# body.md: the markdown content of the comment. Posted verbatim.
#
# The `marker` string is used to find an existing comment to update. It MUST
# be unique per producer (e.g. include the producer name). If no existing
# comment contains the marker, a new one is created. If the marker is found
# in an existing comment, that comment is edited in place.
#
# Producers MUST write `pr_number` from their own workflow context
# (`github.event.pull_request.number`) and MUST NOT read it from any
# fork-controlled source.
on:
workflow_run:
# Producers that may upload a `pr-comment` artifact. When a new producer
# is wired up, add its workflow name here. Runs of workflows not in this
# list will never trigger the poster. Every run of a listed workflow will
# trigger the poster, which will no-op if no `pr-comment` artifact exists.
workflows:
- "FLASH usage analysis"
- "Docs - Orchestrator"
types:
- completed
permissions:
pull-requests: write
actions: read
contents: read
jobs:
post:
name: Post PR Comment
runs-on: ubuntu-latest
# Only run for pull_request producer runs. Push-to-main and other
# non-PR triggers would have no comment to post, and silently no-oping
# inside the script made it look like the poster was broken. Gating at
# the job level surfaces those as a clean "Skipped" in the UI instead.
if: >-
github.event.workflow_run.conclusion != 'cancelled'
&& github.event.workflow_run.event == 'pull_request'
steps:
# Checkout runs first so the poster script is available AND so that
# actions/checkout@v6's default clean step does not delete the artifact
# zip that the next step writes into the workspace. Sparse-checkout
# restricts the materialized tree to just the poster script.
- name: Checkout poster script only
uses: actions/checkout@v6
with:
sparse-checkout: |
Tools/ci/pr-comment-poster.py
Tools/ci/_github_helpers.py
sparse-checkout-cone-mode: false
- name: Download pr-comment artifact
id: download
uses: actions/github-script@v9
with:
script: |
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id,
});
const match = artifacts.data.artifacts.find(a => a.name === 'pr-comment');
if (!match) {
core.info('No pr-comment artifact on this run; nothing to post.');
core.setOutput('found', 'false');
return;
}
const download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: match.id,
archive_format: 'zip',
});
const fs = require('fs');
fs.writeFileSync('pr-comment.zip', Buffer.from(download.data));
core.setOutput('found', 'true');
- name: Unpack artifact
if: steps.download.outputs.found == 'true'
run: |
mkdir -p pr-comment
unzip -q pr-comment.zip -d pr-comment
- name: Validate artifact
if: steps.download.outputs.found == 'true'
run: python3 Tools/ci/pr-comment-poster.py validate pr-comment
- name: Upsert sticky comment
if: steps.download.outputs.found == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 Tools/ci/pr-comment-poster.py post pr-comment
-179
View File
@@ -1,179 +0,0 @@
name: PR Review Poster
# Generic PR review-comment poster. Sibling of "PR Comment Poster": that
# workflow posts sticky issue-style comments, this one posts line-anchored
# review comments on the "Files changed" tab. Any analysis workflow that
# wants to flag specific lines can produce a `pr-review` artifact and this
# workflow will dismiss any stale matching review and post a fresh one.
# Designed so analysis jobs running on untrusted fork PRs can still get
# their inline annotations posted back to the PR.
#
# ==============================================================================
# SECURITY INVARIANTS
# ==============================================================================
# This workflow runs on `workflow_run` which means it runs in the BASE REPO
# context with a WRITE token, even when the triggering PR comes from a fork.
# That is the entire reason it exists, and also the reason it is a loaded
# footgun. Anyone modifying this file MUST preserve the following invariants:
#
# 1. NEVER check out PR code. No `actions/checkout` with a ref. No git clone
# of a fork branch. No execution of scripts from the downloaded artifact.
# The ONLY things read from the artifact are `manifest.json` and
# `comments.json`, and both are treated as opaque data (JSON parsed by
# the poster script and the comment fields posted via the GitHub API).
#
# 2. `pr_number` is validated to be a positive integer before use.
# `marker` is validated to be printable ASCII only before use.
# `commit_sha` is validated to be 40 lowercase hex characters.
# `event` is validated against an allowlist of `COMMENT` only.
# `APPROVE` and `REQUEST_CHANGES` are intentionally forbidden:
# bots should not approve PRs, and REQUEST_CHANGES reviews cannot
# be dismissed by the GITHUB_TOKEN under branch protection rules.
# Validation happens inside
# Tools/ci/pr-review-poster.py which is checked out from the base
# branch, not from the artifact.
#
# 3. Comment bodies and the optional summary are passed to the GitHub API
# as JSON fields, never interpolated into a shell command string.
#
# 4. This workflow file lives on the default branch. `workflow_run` only
# loads workflow files from the default branch, so a fork cannot modify
# THIS file as part of a PR. The fork CAN cause this workflow to fire
# by triggering a producer workflow that uploads a `pr-review`
# artifact. That is intended.
#
# 5. The artifact-name filter (`pr-review`) is the only gate on which
# workflow runs get processed. Any workflow in this repo that uploads
# an artifact named `pr-review` is trusted to have written the
# manifest and comments itself, NOT copied fork-controlled content
# into them. Producer workflows are responsible for that.
#
# 6. `actions/checkout@v6` below uses NO ref (so it pulls the base branch,
# the default-branch commit this workflow file was loaded from) AND
# uses sparse-checkout to materialize ONLY
# Tools/ci/pr-review-poster.py and its stdlib-only helper module
# Tools/ci/_github_helpers.py. The rest of the repo never touches the
# workspace. This is safe: the only files the job executes are
# base-repo Python scripts that were reviewed through normal code
# review, never anything from the PR.
#
# 7. Stale-review dismissal is restricted to reviews whose AUTHOR is
# `github-actions[bot]` AND whose body contains the producer's
# marker. A fork PR cannot impersonate the bot login, and cannot
# inject the marker into a human reviewer's body without API
# access. Both filters together prevent the poster from ever
# dismissing a human review.
#
# ==============================================================================
# ARTIFACT CONTRACT
# ==============================================================================
# Producers upload an artifact named exactly `pr-review` containing:
#
# manifest.json:
# {
# "pr_number": 12345, // required, int > 0
# "marker": "<!-- pr-review-poster:clang-tidy -->", // required, printable ASCII
# "event": "COMMENT", // required, "COMMENT" only
# "commit_sha": "0123456789abcdef0123456789abcdef01234567", // required, 40 hex chars
# "summary": "Optional review summary text" // optional
# }
#
# comments.json: JSON array of line-anchored review comment objects:
# [
# {"path": "src/foo.cpp", "line": 42, "side": "RIGHT", "body": "..."},
# {"path": "src/bar.hpp", "start_line": 10, "line": 15,
# "side": "RIGHT", "start_side": "RIGHT", "body": "..."}
# ]
#
# The `marker` string is used to find an existing matching review to
# dismiss before posting a new one. It MUST be unique per producer (e.g.
# include the producer name).
#
# Producers MUST write `pr_number` and `commit_sha` from their own
# workflow context (`github.event.pull_request.number` and
# `github.event.pull_request.head.sha`) and MUST NOT read either from any
# fork-controlled source.
on:
workflow_run:
# Producers that may upload a `pr-review` artifact. When a new
# producer is wired up, add its workflow name here. Runs of workflows
# not in this list will never trigger the poster. Every run of a
# listed workflow will trigger the poster, which will no-op if no
# `pr-review` artifact exists.
workflows:
- "Static Analysis"
types:
- completed
permissions:
pull-requests: write
actions: read
contents: read
jobs:
post:
name: Post PR Review
runs-on: ubuntu-latest
# Only run for pull_request producer runs. Push-to-main and other
# non-PR triggers have no review to post, so gating at the job level
# surfaces those as a clean "Skipped" in the UI instead of a
# silent no-op buried inside the script.
if: >-
github.event.workflow_run.conclusion != 'cancelled'
&& github.event.workflow_run.event == 'pull_request'
steps:
# Checkout runs first so the poster scripts are available AND so
# that actions/checkout@v6's default clean step does not delete the
# artifact zip that the next step writes into the workspace.
# Sparse-checkout restricts the materialized tree to just the
# poster script and its stdlib helper module.
- name: Checkout poster script only
uses: actions/checkout@v6
with:
sparse-checkout: |
Tools/ci/pr-review-poster.py
Tools/ci/_github_helpers.py
sparse-checkout-cone-mode: false
- name: Download pr-review artifact
id: download
uses: actions/github-script@v9
with:
script: |
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.payload.workflow_run.id,
});
const match = artifacts.data.artifacts.find(a => a.name === 'pr-review');
if (!match) {
core.info('No pr-review artifact on this run; nothing to post.');
core.setOutput('found', 'false');
return;
}
const download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: match.id,
archive_format: 'zip',
});
const fs = require('fs');
fs.writeFileSync('pr-review.zip', Buffer.from(download.data));
core.setOutput('found', 'true');
- name: Unpack artifact
if: steps.download.outputs.found == 'true'
run: |
mkdir -p pr-review
unzip -q pr-review.zip -d pr-review
- name: Validate artifact
if: steps.download.outputs.found == 'true'
run: python3 Tools/ci/pr-review-poster.py validate pr-review
- name: Post PR review
if: steps.download.outputs.found == 'true'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: python3 Tools/ci/pr-review-poster.py post pr-review
+8 -11
View File
@@ -14,23 +14,20 @@ on:
jobs:
build:
runs-on: [runs-on,runner=1cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}"]
runs-on: ubuntu-24.04
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 1
fetch-depth: 0
- name: Setup Python
uses: actions/setup-python@v6
with:
python-version: "3.10"
- name: Install Python3
run: sudo apt-get install python3 python3-setuptools python3-pip -y
- name: Install tools
run: pip install mypy types-requests flake8
run: python3 -m pip install mypy types-requests flake8 --break-system-packages
- name: Check MAVSDK test scripts with mypy
run: mypy --strict test/mavsdk_tests/*.py
run: $HOME/.local/bin/mypy --strict test/mavsdk_tests/*.py
- name: Check MAVSDK test scripts with flake8
run: flake8 test/mavsdk_tests/*.py
run: $HOME/.local/bin/flake8 test/mavsdk_tests/*.py
+43 -42
View File
@@ -23,18 +23,16 @@ concurrency:
jobs:
build:
runs-on: [runs-on,runner=8cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu22-full-x64,"run-id=${{ github.run_id }}",spot=false]
container:
image: px4io/px4-dev-ros2-galactic:2021-09-08
options: --privileged --ulimit core=-1 --security-opt seccomp=unconfined
env:
PX4_SBOM_DISABLE: 1
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Configure Git Safe Directory
fetch-depth: 0
- name: Git Ownership Workaround
run: git config --system --add safe.directory '*'
- name: Update ROS Keys
@@ -47,21 +45,30 @@ jobs:
run: |
apt update && apt install -y gazebo11 libgazebo11-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly libgstreamer-plugins-base1.0-dev
- uses: ./.github/actions/setup-ccache
id: ccache
- name: Prepare ccache timestamp
id: ccache_cache_timestamp
shell: cmake -P {0}
run: |
string(TIMESTAMP current_date "%Y-%m-%d-%H;%M;%S" UTC)
message("::set-output name=timestamp::${current_date}")
- name: ccache cache files
uses: actions/cache@v4
with:
cache-key-prefix: ccache-ros-integration
max-size: 400M
path: ~/.ccache
key: ros_integration_tests-${{matrix.config.build_type}}-ccache-${{steps.ccache_cache_timestamp.outputs.timestamp}}
restore-keys: ros_integration_tests-${{matrix.config.build_type}}-ccache-
- name: setup ccache
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 300M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Cache - Restore Micro-XRCE-DDS Agent
id: cache-xrce-agent
uses: actions/cache@v5
with:
path: /opt/Micro-XRCE-DDS-Agent
key: xrce-agent-v2.2.1-fastdds-2.8.2-galactic-2021-09-08
- name: Build - Micro-XRCE-DDS Agent (v2.2.1)
if: steps.cache-xrce-agent.outputs.cache-hit != 'true'
- name: Get and build micro-xrce-dds-agent
run: |
cd /opt
git clone --recursive https://github.com/eProsima/Micro-XRCE-DDS-Agent.git
@@ -72,27 +79,17 @@ jobs:
cd build
cmake ..
make -j2
- name: ccache post-run micro-xrce-dds-agent
run: ccache -s
- name: Cache - Restore PX4 ROS 2 Interface Library Workspace
id: cache-px4-ros2-ws
uses: actions/cache@v5
with:
path: /opt/px4_ws
# Bump 'v1' when the cached workspace layout changes in a way
# that is not captured by the message/service hash below.
key: px4-ros2-ws-v1-galactic-2021-09-08-${{ hashFiles('msg/*.msg', 'msg/versioned/*.msg', 'srv/*.srv') }}
- name: Build - PX4 ROS 2 Interface Library
if: steps.cache-px4-ros2-ws.outputs.cache-hit != 'true'
- name: Get and build the ros2 interface library
shell: bash
run: |
PX4_DIR="$(pwd)"
. /opt/ros/galactic/setup.bash
mkdir -p /opt/px4_ws/src
cd /opt/px4_ws/src
# On a PR, target the branch we're merging into (main or release/X.Y).
# On a direct push, fall back to the branch we're running on.
BRANCH="${GITHUB_BASE_REF:-$GITHUB_REF_NAME}"
BRANCH="${GITHUB_HEAD_REF:-$GITHUB_REF_NAME}"
REPO_URL="https://github.com/Auterion/px4-ros2-interface-lib.git"
if git ls-remote --heads "$REPO_URL" "$BRANCH" | grep -q "$BRANCH"; then
echo "Cloning px4-ros2-interface-lib with matching branch: $BRANCH"
@@ -109,8 +106,17 @@ jobs:
"${PX4_DIR}/Tools/copy_to_ros_ws.sh" "$(pwd)"
rm -rf src/translation_node src/px4_msgs_old
colcon build --symlink-install
- name: ccache post-run ros workspace
run: ccache -s
- uses: ./.github/actions/build-gazebo-sitl
- name: Build PX4
run: make px4_sitl_default
- name: ccache post-run px4/firmware
run: ccache -s
- name: Build SITL Gazebo
run: make px4_sitl_default sitl_gazebo-classic
- name: ccache post-run sitl_gazebo-classic
run: ccache -s
- name: Core dump settings
run: |
@@ -122,17 +128,12 @@ jobs:
run: |
. /opt/px4_ws/install/setup.bash
/opt/Micro-XRCE-DDS-Agent/build/MicroXRCEAgent udp4 localhost -p 8888 -v 0 &
test/ros_test_runner.py --verbose --model iris --force-color
test/ros_test_runner.py --verbose --model iris --upload --force-color
timeout-minutes: 45
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: Upload failed logs
if: failure()
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: failed-logs.zip
path: |
+16 -29
View File
@@ -10,9 +10,6 @@ on:
- '**'
paths-ignore:
- 'docs/**'
permissions:
contents: read
defaults:
run:
shell: bash
@@ -23,8 +20,8 @@ concurrency:
jobs:
build_and_test:
name: Build and test [${{ matrix.config.ros_version }}]
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
name: Build and test
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false]
strategy:
fail-fast: false
matrix:
@@ -32,43 +29,33 @@ jobs:
- {ros_version: "humble", ubuntu: "jammy"}
- {ros_version: "jazzy", ubuntu: "noble"}
container:
image: ros:${{ matrix.config.ros_version }}-ros-base-${{ matrix.config.ubuntu }}
image: rostooling/setup-ros-docker:ubuntu-${{ matrix.config.ubuntu }}-latest
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- name: Setup ROS 2 (${{ matrix.config.ros_version }})
uses: ros-tooling/setup-ros@v0.7
with:
required-ros-distributions: ${{ matrix.config.ros_version }}
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git Safe Directory
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-ros-translation-${{ matrix.config.ros_version }}
max-size: 150M
base-dir: /ros_ws
install-ccache: 'true'
# Workaround for https://github.com/actions/runner/issues/2033
- name: ownership workaround
run: git config --system --add safe.directory '*'
- name: Check .msg file versioning
if: github.event_name == 'pull_request'
run: |
./Tools/ci/check_msg_versioning.sh ${{ github.event.pull_request.base.sha }} ${{github.event.pull_request.head.sha}}
- name: Build - Translation Node
- name: Build and test
run: |
ros_ws=/ros_ws
mkdir -p $ros_ws/src
./Tools/copy_to_ros_ws.sh $ros_ws
cd $ros_ws
source /opt/ros/${{ matrix.config.ros_version }}/setup.sh
colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DCMAKE_C_COMPILER_LAUNCHER=ccache --symlink-install --event-handlers=console_cohesion+
- name: Test - Translation Node Unit Tests
run: |
source /ros_ws/install/setup.sh
/ros_ws/build/translation_node/translation_node_unit_tests
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release --symlink-install --event-handlers=console_cohesion+
source ./install/setup.sh
./build/translation_node/translation_node_unit_tests
-42
View File
@@ -1,42 +0,0 @@
name: SBOM License Check
on:
push:
branches:
- 'main'
- 'release/**'
- 'stable'
paths:
- '.gitmodules'
- 'Tools/ci/license-overrides.yaml'
- 'Tools/ci/generate_sbom.py'
pull_request:
branches:
- '**'
paths:
- '.gitmodules'
- 'Tools/ci/license-overrides.yaml'
- 'Tools/ci/generate_sbom.py'
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
verify-licenses:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 1
submodules: false
- name: Install PyYAML
run: pip install pyyaml --break-system-packages
- name: Verify submodule licenses
run: python3 Tools/ci/generate_sbom.py --verify-licenses --source-dir .
-132
View File
@@ -1,132 +0,0 @@
name: SBOM Monthly Audit
on:
schedule:
# First Monday of each month at 09:00 UTC
- cron: '0 9 1-7 * 1'
workflow_dispatch:
inputs:
branch:
description: 'Branch to audit (leave empty for current)'
required: false
type: string
permissions:
contents: read
issues: write
jobs:
audit:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.branch || github.ref }}
fetch-depth: 1
submodules: recursive
- name: Install PyYAML
run: pip install pyyaml --break-system-packages
- name: Run license verification
id: verify
continue-on-error: true
run: |
python3 Tools/ci/generate_sbom.py --verify-licenses --source-dir . 2>&1 | tee /tmp/sbom-verify.txt
echo "exit_code=$?" >> "$GITHUB_OUTPUT"
- name: Check for issues
id: check
run: |
if grep -q "<-- UNRESOLVED" /tmp/sbom-verify.txt; then
echo "has_issues=true" >> "$GITHUB_OUTPUT"
# Extract only genuinely unresolved license lines
grep "<-- UNRESOLVED" /tmp/sbom-verify.txt > /tmp/sbom-issues.txt || true
# Extract copyleft lines
sed -n '/Copyleft licenses detected/,/^$/p' /tmp/sbom-verify.txt > /tmp/sbom-copyleft.txt || true
else
echo "has_issues=false" >> "$GITHUB_OUTPUT"
fi
- name: Create issue if problems found
if: steps.check.outputs.has_issues == 'true'
uses: actions/github-script@v9
with:
script: |
const fs = require('fs');
const fullOutput = fs.readFileSync('/tmp/sbom-verify.txt', 'utf8');
let issueLines = '';
try {
issueLines = fs.readFileSync('/tmp/sbom-issues.txt', 'utf8');
} catch (e) {
issueLines = 'No specific NOASSERTION lines captured.';
}
let copyleftLines = '';
try {
copyleftLines = fs.readFileSync('/tmp/sbom-copyleft.txt', 'utf8');
} catch (e) {
copyleftLines = '';
}
const date = new Date().toISOString().split('T')[0];
const branch = '${{ inputs.branch || github.ref_name }}';
// Check for existing open issue to avoid duplicates
const existing = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
labels: 'sbom-audit',
state: 'open',
});
if (existing.data.length > 0) {
// Update existing issue with new findings
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.data[0].number,
body: `## Monthly audit update (${date})\n\nIssues still present:\n\n\`\`\`\n${issueLines}\n\`\`\`\n${copyleftLines ? `\n### Copyleft warnings\n\`\`\`\n${copyleftLines}\n\`\`\`` : ''}`,
});
return;
}
await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: `chore(sbom): license audit found NOASSERTION entries on ${branch} (${date})`,
labels: ['sbom-audit'],
assignees: ['mrpollo'],
body: [
`## SBOM Monthly Audit -- ${branch} -- ${date}`,
'',
'The automated SBOM license audit found submodules with unresolved licenses.',
'',
'### NOASSERTION entries',
'',
'```',
issueLines,
'```',
'',
copyleftLines ? `### Copyleft warnings\n\n\`\`\`\n${copyleftLines}\n\`\`\`\n` : '',
'### How to fix',
'',
'1. Check the submodule repo for a LICENSE file',
'2. Add an override to `Tools/ci/license-overrides.yaml`',
'3. Run `python3 Tools/ci/generate_sbom.py --verify-licenses --source-dir .` to confirm',
'',
'### Full output',
'',
'<details>',
'<summary>Click to expand</summary>',
'',
'```',
fullOutput,
'```',
'',
'</details>',
'',
'cc @mrpollo',
].join('\n'),
});
+49 -21
View File
@@ -24,7 +24,7 @@ concurrency:
jobs:
build:
name: Testing PX4 ${{ matrix.config.model }}
runs-on: [runs-on,runner=8cpu-linux-x64,image=ubuntu24-full-x64,"run-id=${{ github.run_id }}",spot=false,extras=s3-cache]
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu22-full-x64,"run-id=${{ github.run_id }}",spot=false]
container:
image: px4io/px4-dev-simulation-focal:2021-09-08
options: --privileged --ulimit core=-1 --security-opt seccomp=unconfined
@@ -37,25 +37,52 @@ jobs:
# transitions). Re-enable once the test infrastructure is stabilized.
# - {model: "tailsitter" , latitude: "29.660316", longitude: "-82.316658", altitude: "30", build_type: "RelWithDebInfo" } # Florida
# - {model: "standard_vtol", latitude: "47.397742", longitude: "8.545594", altitude: "488", build_type: "Coverage" } # Zurich
env:
PX4_CMAKE_BUILD_TYPE: ${{ matrix.config.build_type }}
PX4_SBOM_DISABLE: 1
steps:
- uses: runs-on/action@v2
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Configure Git Safe Directory
fetch-depth: 0
- name: Git Ownership Workaround
run: git config --system --add safe.directory '*'
- uses: ./.github/actions/setup-ccache
id: ccache
with:
cache-key-prefix: ccache-sitl-gazebo-classic
max-size: 350M
- id: set-timestamp
name: Set timestamp for cache
run: echo "::set-output name=timestamp::$(date +"%Y%m%d%H%M%S")"
- uses: ./.github/actions/build-gazebo-sitl
- name: Cache Key Config
uses: actions/cache@v4
with:
path: ~/.ccache
key: sitl-ccache-${{ steps.set-timestamp.outputs.timestamp }}
restore-keys: sitl-ccache-${{ steps.set-timestamp.outputs.timestamp }}
- name: Cache Conf Config
run: |
mkdir -p ~/.ccache
echo "base_dir = ${GITHUB_WORKSPACE}" > ~/.ccache/ccache.conf
echo "compression = true" >> ~/.ccache/ccache.conf
echo "compression_level = 6" >> ~/.ccache/ccache.conf
echo "max_size = 120M" >> ~/.ccache/ccache.conf
echo "hash_dir = false" >> ~/.ccache/ccache.conf
ccache -s
ccache -z
- name: Build PX4
env:
PX4_CMAKE_BUILD_TYPE: ${{matrix.config.build_type}}
run: make px4_sitl_default
- name: Cache Post-Run [px4_sitl_default]
run: ccache -s
- name: Build SITL Gazebo
env:
PX4_CMAKE_BUILD_TYPE: ${{matrix.config.build_type}}
run: make px4_sitl_default sitl_gazebo-classic
- name: Cache Post-Run [sitl_gazebo-classic]
run: ccache -s
- name: Download MAVSDK
run: wget "https://github.com/mavlink/MAVSDK/releases/download/v$(cat test/mavsdk_tests/MAVSDK_VERSION)/libmavsdk-dev_$(cat test/mavsdk_tests/MAVSDK_VERSION)_ubuntu20.04_amd64.deb"
@@ -68,19 +95,19 @@ jobs:
PX4_HOME_LAT: ${{matrix.config.latitude}}
PX4_HOME_LON: ${{matrix.config.longitude}}
PX4_HOME_ALT: ${{matrix.config.altitude}}
PX4_CMAKE_BUILD_TYPE: ${{matrix.config.build_type}}
run: |
export
ulimit -a
- name: Build PX4 / MAVSDK tests
env:
PX4_CMAKE_BUILD_TYPE: ${{matrix.config.build_type}}
DONT_RUN: 1
run: make px4_sitl_default sitl_gazebo-classic mavsdk_tests
- uses: ./.github/actions/save-ccache
if: always()
with:
cache-primary-key: ${{ steps.ccache.outputs.cache-primary-key }}
- name: Cache Post-Run [px4_sitl_default sitl_gazebo-classic mavsdk_tests]
run: ccache -s
- name: Core Dump Settings
run: |
@@ -92,12 +119,13 @@ jobs:
PX4_HOME_LAT: ${{matrix.config.latitude}}
PX4_HOME_LON: ${{matrix.config.longitude}}
PX4_HOME_ALT: ${{matrix.config.altitude}}
PX4_CMAKE_BUILD_TYPE: ${{matrix.config.build_type}}
run: test/mavsdk_tests/mavsdk_test_runner.py --speed-factor 10 --abort-early --model ${{matrix.config.model}} test/mavsdk_tests/configs/sitl.json --verbose --force-color
timeout-minutes: 45
- name: Upload failed logs
if: failure()
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: failed-${{matrix.config.model}}-logs.zip
path: |
@@ -111,7 +139,7 @@ jobs:
- name: Upload PX4 coredump
if: failure() && ${{ hashFiles('px4.core') != '' }}
uses: actions/upload-artifact@v7
uses: actions/upload-artifact@v4
with:
name: coredump
path: px4.core
@@ -128,7 +156,7 @@ jobs:
- name: Upload Coverage Information to Codecov
if: contains(matrix.config.build_type, 'Coverage')
uses: codecov/codecov-action@v6
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: mavsdk
+1 -2
View File
@@ -2,7 +2,6 @@ name: 'Handle stale issues and PRs'
on:
schedule:
- cron: '30 1 * * *'
workflow_dispatch:
jobs:
stale:
@@ -10,7 +9,7 @@ jobs:
steps:
- uses: actions/stale@v10
with:
operations-per-run: 1500
operations-per-run: 250
days-before-stale: 90
days-before-close: 30
stale-issue-label: 'stale'
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
runs-on: [runs-on,runner=4cpu-linux-x64,image=ubuntu22-full-x64,"run-id=${{ github.run_id }}",spot=false]
steps:
- name: Checkout PX4 repo
uses: actions/checkout@v6
uses: actions/checkout@v4
- name: Setup git credentials
run: |
@@ -22,7 +22,7 @@ jobs:
TAG_NAME: ${{ github.event_name == 'workflow_dispatch' && inputs.tag_name || github.ref_name }}
steps:
- name: Checkout PX4 repo
uses: actions/checkout@v6
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
-3
View File
@@ -115,6 +115,3 @@ _emscripten_sdk/
# virtual Python environment
.venv
# Claude Code local settings
.claude/settings.local.json
+1 -31
View File
@@ -229,28 +229,6 @@ endif()
#
project(px4 CXX C ASM)
# Silence Apple ranlib "has no symbols" warnings. Several PX4 sources are
# wrapped in #if defined(CONFIG_*) guards (e.g. platforms/common/i2c.cpp,
# spi.cpp, board_common.c, pab_manifest.c, px4_log_history.cpp) and some
# libraries carry a dummy.cpp placeholder, all of which legitimately produce
# empty object files on POSIX/SITL. GNU ranlib ignores this; Apple's warns.
#
# The warning is actually emitted by `ar qc` (which implicitly builds a symbol
# table), not by the standalone ranlib call. So we use `ar qcS` to skip the
# implicit symbol table, then let CMAKE_*_ARCHIVE_FINISH run ranlib with the
# -no_warning_for_no_symbols flag to add it quietly.
if(APPLE)
set(CMAKE_C_ARCHIVE_CREATE "<CMAKE_AR> qcS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_CXX_ARCHIVE_CREATE "<CMAKE_AR> qcS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_ASM_ARCHIVE_CREATE "<CMAKE_AR> qcS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_C_ARCHIVE_APPEND "<CMAKE_AR> qS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_CXX_ARCHIVE_APPEND "<CMAKE_AR> qS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_ASM_ARCHIVE_APPEND "<CMAKE_AR> qS <TARGET> <LINK_FLAGS> <OBJECTS>")
set(CMAKE_C_ARCHIVE_FINISH "<CMAKE_RANLIB> -no_warning_for_no_symbols -c <TARGET>")
set(CMAKE_CXX_ARCHIVE_FINISH "<CMAKE_RANLIB> -no_warning_for_no_symbols -c <TARGET>")
set(CMAKE_ASM_ARCHIVE_FINISH "<CMAKE_RANLIB> -no_warning_for_no_symbols -c <TARGET>")
endif()
# CMake build type (Debug Release RelWithDebInfo MinSizeRel Coverage)
if(NOT CMAKE_BUILD_TYPE)
if(${PX4_PLATFORM} STREQUAL "nuttx")
@@ -262,15 +240,8 @@ if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE ${PX4_BUILD_TYPE} CACHE STRING "Build type" FORCE)
endif()
if(CONFIG_BOARD_SUPPORT_FORTIFIED_TOOLCHAIN)
set(PX4_DEBUG_OPT_LEVEL -Og)
message(STATUS "fortified toolchain support enabled: PX4_DEBUG_OPT_LEVEL=${PX4_DEBUG_OPT_LEVEL}")
else()
set(PX4_DEBUG_OPT_LEVEL -O0)
endif()
if((CMAKE_BUILD_TYPE STREQUAL "Debug") OR (CMAKE_BUILD_TYPE STREQUAL "Coverage"))
set(MAX_CUSTOM_OPT_LEVEL ${PX4_DEBUG_OPT_LEVEL})
set(MAX_CUSTOM_OPT_LEVEL -O0)
elseif(CMAKE_BUILD_TYPE MATCHES "Sanitizer")
set(MAX_CUSTOM_OPT_LEVEL -O1)
elseif(CMAKE_BUILD_TYPE MATCHES "Release")
@@ -513,7 +484,6 @@ include(bloaty)
include(metadata)
include(package)
include(sbom)
# install python requirements using configured python
add_custom_target(install_python_requirements
+20 -57
View File
@@ -2,82 +2,45 @@
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
Examples of behavior that contributes to creating a positive environment include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior include:
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
## Our Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at coc@dronecode.org. All complaints will be reviewed and investigated promptly and fairly.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at lorenz@px4.io. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/
-14
View File
@@ -67,16 +67,6 @@ menu "Toolchain"
help
Enables Cmake Release for -O3 optimization
config BOARD_SUPPORT_FORTIFIED_TOOLCHAIN
bool "Fortified toolchain support"
default n
help
Enable compatibility with toolchains that define
_FORTIFY_SOURCE.
This switches PX4_DEBUG_OPT_LEVEL from -O0 to -Og. Keep this
disabled unless the fortified toolchain requires optimization.
config BOARD_ROMFSROOT
string "ROMFSROOT"
default "px4fmu_common"
@@ -220,10 +210,6 @@ menu "examples"
source "src/examples/Kconfig"
endmenu
menu "templates"
source "src/templates/Kconfig"
endmenu
menu "platforms"
depends on PLATFORM_QURT || PLATFORM_POSIX || PLATFORM_NUTTX
source "platforms/Kconfig"
+2 -13
View File
@@ -1,11 +1,9 @@
Maintainers
===========
PX4 is maintained by a group of contributors trusted to steward the project. All maintainers listed below are members of the @PX4/dev-team, have write access, and participate in maintainer decisions. We recognize two types: **Code Owners**, responsible for specific components, and **Reviewers**, who help across the project without a fixed component.
See [the documentation on Maintainers](https://docs.px4.io/main/en/contribute/maintainers.html) to learn about the role of the maintainers and the process to become one.
See [the documentation on Maintainers](https://docs.px4.io/main/en/contribute/maintainers) to learn about the role of the maintainers and the process to become one.
**Code Owners**
**Active Maintainers**
| Name | Sector | GitHub | Chat | email
|-------------------------|--------|--------|------|----------------
@@ -25,15 +23,6 @@ See [the documentation on Maintainers](https://docs.px4.io/main/en/contribute/ma
| Jacob Dahl | Simulation | [@dakejahl](https://github.com/dakejahl) | dakejahl | <dahl.jakejacob@gmail.com>
**Reviewers**
Reviewers help maintain PX4 across the project without ownership of a specific component.
| Name | GitHub | Chat | email
|------|--------|------|----------------------
| Onur Ozkan | [@onur-ozkan](https://github.com/onur-ozkan) | onur_ozkan0126 | <onur@orkavian.com>
**Documentation Maintainers**
| Name | GitHub | Chat | email
+1 -21
View File
@@ -494,7 +494,7 @@ python_coverage:
# static analyzers (scan-build, clang-tidy, cppcheck)
# --------------------------------------------------------------------
.PHONY: scan-build px4_sitl_default-clang px4_sitl_default-clang-test clang-ci clang-tidy clang-tidy-fix
.PHONY: scan-build px4_sitl_default-clang clang-tidy clang-tidy-fix
.PHONY: cppcheck shellcheck_all validate_module_configs
scan-build:
@@ -512,26 +512,6 @@ px4_sitl_default-clang:
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang && cmake "$(SRC_DIR)" $(CMAKE_ARGS) -G"$(PX4_CMAKE_GENERATOR)" -DCONFIG=px4_sitl_default -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
@$(PX4_MAKE) -C "$(SRC_DIR)"/build/px4_sitl_default-clang
# Clang SITL configure with BUILD_TESTING=ON so test files land in
# compile_commands.json with resolved gtest/fuzztest includes. Used by CI
# to produce a compilation database for diff-based clang-tidy that can
# lint test files. Configure only: we don't build the test binaries here,
# just generate the database.
px4_sitl_default-clang-test:
@mkdir -p "$(SRC_DIR)"/build/px4_sitl_default-clang-test
@cd "$(SRC_DIR)"/build/px4_sitl_default-clang-test && cmake "$(SRC_DIR)" $(CMAKE_ARGS) -G"$(PX4_CMAKE_GENERATOR)" -DCONFIG=px4_sitl_default -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_TESTING=ON
# CI-oriented target that prepares both clang build directories used by
# the Static Analysis workflow:
# - px4_sitl_default-clang: full build, BUILD_TESTING=OFF.
# Used by `make clang-tidy` (push-to-main) and run-clang-tidy-pr.py.
# - px4_sitl_default-clang-test: configure-only, BUILD_TESTING=ON.
# Used by clang-tidy-diff-18.py so test files are in the
# compilation database with resolved gtest/fuzztest includes.
# Running one target ensures both dirs exist before any clang-tidy
# variant runs, and keeps the workflow free of raw cmake invocations.
clang-ci: px4_sitl_default-clang px4_sitl_default-clang-test
# Paths to exclude from clang-tidy (auto-generated from .gitmodules + manual additions):
# - All submodules (external code we consume, not edit)
# - Test code (allowed looser style)
+5 -36
View File
@@ -9,16 +9,11 @@
</p>
<p align="center">
<a href="https://github.com/PX4/PX4-Autopilot/releases"><img src="https://img.shields.io/github/release/PX4/PX4-Autopilot.svg" alt="Release"></a>
<a href="https://zenodo.org/badge/latestdoi/22634/PX4/PX4-Autopilot"><img src="https://zenodo.org/badge/22634/PX4/PX4-Autopilot.svg" alt="DOI"></a>
<a href="https://discord.gg/dronecode"><img src="https://img.shields.io/discord/1022170275984457759?label=discord&logo=discord&logoColor=white&color=5865F2" alt="Discord"></a>
</p>
<p align="center">
<a href="https://github.com/PX4/PX4-Autopilot/releases"><img src="https://img.shields.io/github/release/PX4/PX4-Autopilot.svg" alt="Releases"></a>
<a href="https://www.bestpractices.dev/projects/6520"><img src="https://www.bestpractices.dev/projects/6520/badge" alt="OpenSSF Best Practices"></a>
<a href="https://insights.linuxfoundation.org/project/px4"><img src="https://insights.linuxfoundation.org/api/badge/health-score?project=px4" alt="LFX Health Score"></a>
<a href="https://insights.linuxfoundation.org/project/px4"><img src="https://insights.linuxfoundation.org/api/badge/contributors?project=px4" alt="LFX Contributors"></a>
<a href="https://insights.linuxfoundation.org/project/px4"><img src="https://insights.linuxfoundation.org/api/badge/active-contributors?project=px4" alt="LFX Active Contributors"></a>
<a href="https://zenodo.org/badge/latestdoi/22634/PX4/PX4-Autopilot"><img src="https://zenodo.org/badge/22634/PX4/PX4-Autopilot.svg" alt="DOI"></a>
<a href="https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml"><img src="https://github.com/PX4/PX4-Autopilot/actions/workflows/build_all_targets.yml/badge.svg?branch=main" alt="Build Targets"></a>
<a href="https://discord.gg/dronecode"><img src="https://discordapp.com/api/guilds/1022170275984457759/widget.png?style=shield" alt="Discord"></a>
</p>
---
@@ -70,17 +65,7 @@ PX4 is an open-source autopilot stack for drones and unmanned vehicles. It suppo
<sub>…and many more: helicopters, autogyros, airships, submarines, boats, and other experimental platforms. These frames have basic support but are not part of the regular flight-test program. See the <a href="https://docs.px4.io/main/en/airframes/airframe_reference.html">full airframe reference</a>.</sub>
## Try PX4
Run PX4 in simulation with a single command. No build tools, no dependencies beyond Docker:
```bash
docker run --rm -it -p 14550:14550/udp px4io/px4-sitl:latest
```
Open [QGroundControl](https://qgroundcontrol.com) and fly. See [PX4 Simulation Quickstart](../dev_setup/px4_simulation_quickstart.md) for more options.
## Build from Source
## Quick Start
```bash
git clone https://github.com/PX4/PX4-Autopilot.git --recursive
@@ -114,22 +99,6 @@ make px4_sitl
We welcome contributions of all kinds — bug reports, documentation, new features, and code reviews. Please read the [Contribution Guide](https://docs.px4.io/main/en/contribute/) to get started.
## Citation
If you use PX4 in academic work, please cite it. BibTeX:
```bibtex
@software{px4_autopilot,
author = {Meier, Lorenz and {The PX4 Contributors}},
title = {{PX4 Autopilot}},
publisher = {Zenodo},
doi = {10.5281/zenodo.595432},
url = {https://px4.io}
}
```
The DOI above is a Zenodo concept DOI that always resolves to the latest release. For a version-pinned citation, see the [Zenodo record](https://doi.org/10.5281/zenodo.595432) or our [`CITATION.cff`](CITATION.cff).
## Governance
The PX4 Autopilot project is hosted by the [Dronecode Foundation](https://www.dronecode.org/), a [Linux Foundation](https://www.linuxfoundation.org/) Collaborative Project. Dronecode holds all PX4 trademarks and serves as the project's legal guardian, ensuring vendor-neutral stewardship — no single company owns the name or controls the roadmap. The source code is licensed under the [BSD 3-Clause](LICENSE) license, so you are free to use, modify, and distribute it in your own projects.
-1
View File
@@ -193,7 +193,6 @@ endif()
# board custom init files
set(OPTIONAL_BOARD_RC)
list(APPEND OPTIONAL_BOARD_RC
rc.board_early
rc.board_defaults
rc.board_sensors
rc.board_extras
@@ -18,7 +18,6 @@ param set-default SENS_EN_BAROSIM 1
param set-default SENS_EN_MAGSIM 1
param set SIH_VEHICLE_TYPE 4
param set-default MAV_TYPE 13
# Symmetric hexacopter X clockwise motor numbering
param set-default CA_ROTOR_COUNT 6
@@ -14,7 +14,6 @@
param set UAVCAN_ENABLE 0
param set-default CA_AIRFRAME 1
param set-default CA_ROTOR_COUNT 1
param set-default CA_ROTOR0_PX 0.3
@@ -39,6 +38,7 @@ param set-default SYS_HITL 2
# - without real battery
param set-default CBRK_SUPPLY_CHK 894281
param set SIH_T_MAX 6
param set SIH_MASS 0.3
param set SIH_IXX 0.00402
param set SIH_IYY 0.0144
@@ -48,21 +48,3 @@ param set SIH_KDV 0.2
param set SIH_VEHICLE_TYPE 1 # sih as fixed wing
param set RWTO_TKOFF 1 # enable takeoff from runway (as opposed to launched)
# pusher propeller model with advance ratio, model from UIUC APC 8x6"
param set SIH_F_T_MAX 6
param set SIH_F_Q_MAX 0.03
# if SIH_F_CT0 > 0, SIH_F_T_MAX and SIH_F_Q_MAX will be overridden
param set SIH_F_CT0 0.131
param set SIH_F_CT1 0.004
param set SIH_F_CT2 -0.146
param set SIH_F_CP0 0.0777
param set SIH_F_CP1 0.0498
param set SIH_F_CP2 -0.11
param set SIH_F_DIA_INCH 8
param set SIH_F_RPM_MAX 9000
param set-default FW_AIRSPD_MIN 7
param set-default FW_AIRSPD_TRIM 10
param set-default FW_AIRSPD_MAX 12
param set-default FW_PSP_OFF 0.5
@@ -28,7 +28,6 @@ param set-default VT_FW_DIFTHR_EN 1
param set-default VT_FW_DIFTHR_S_Y 0.3
param set-default MPC_MAN_Y_MAX 60
param set-default MC_PITCH_P 5
param set-default FW_PSP_OFF 5
param set-default CA_AIRFRAME 4
param set-default CA_ROTOR_COUNT 2
@@ -57,6 +56,7 @@ param set-default HIL_ACT_REV 32
param set-default MAV_TYPE 19
# set SYS_HITL to 2 to start the SIH and avoid sensors startup
param set-default SYS_HITL 2
@@ -66,9 +66,8 @@ param set-default CBRK_SUPPLY_CHK 894281
param set-default SENS_DPRES_OFF 0.001
# tailsitter is equipped with two forward propellers
param set SIH_F_T_MAX 2
param set SIH_F_Q_MAX 0.0165
param set SIH_T_MAX 2.0
param set SIH_Q_MAX 0.0165
param set SIH_MASS 0.2
# IXX and IZZ are inverted from the thesis as the body frame is pitched by 90 deg
param set SIH_IXX 0.00354
@@ -78,19 +77,6 @@ param set SIH_IXZ 0
param set SIH_KDV 0.2
param set SIH_L_ROLL 0.145
# propeller diameter, rpm, and coeffs coming from the thesis
# Modeling and control of a flying wing tailsitter unmanned aerial vehicle."
# Chiappinelli, Romain, supervised by Nahon, Meyer, McGill University, Masters thesis, 2018.
# if SIH_F_CT0 > 0, SIH_F_T_MAX and SIH_F_Q_MAX will be overridden
param set SIH_F_CT0 0.1342
param set SIH_F_CT1 -0.1196
param set SIH_F_CT2 -0.1281
param set SIH_F_CP0 0.0522
param set SIH_F_CP1 -0.0146
param set SIH_F_CP2 -0.0602
param set SIH_F_DIA_INCH 5
param set SIH_F_RPM_MAX 14000
# sih as tailsitter
param set SIH_VEHICLE_TYPE 2
@@ -56,7 +56,6 @@ param set-default CA_SV_CS2_TYPE 4 # rudder
param set-default FW_AIRSPD_MIN 7
param set-default FW_AIRSPD_TRIM 10
param set-default FW_AIRSPD_MAX 12
param set-default VT_FWD_THRUST_EN 1
param set-default HIL_ACT_FUNC1 101
param set-default HIL_ACT_FUNC2 102
@@ -78,7 +77,6 @@ param set-default CBRK_SUPPLY_CHK 894281
param set-default SENS_DPRES_OFF 0.001
# quadrotor propellers
param set SIH_T_MAX 2.0
param set SIH_Q_MAX 0.0165
param set SIH_MASS 0.2
@@ -90,18 +88,5 @@ param set SIH_IXZ 0
param set SIH_KDV 0.2
param set SIH_L_ROLL 0.2
# pusher propeller model with advance ratio, model from UIUC APC 8x6"
param set SIH_F_T_MAX 6
param set SIH_F_Q_MAX 0.03
# if SIH_F_CT0 > 0, SIH_F_T_MAX and SIH_F_Q_MAX will be overridden
param set SIH_F_CT0 0.131
param set SIH_F_CT1 0.004
param set SIH_F_CT2 -0.146
param set SIH_F_CP0 0.0777
param set SIH_F_CP1 0.0498
param set SIH_F_CP2 -0.11
param set SIH_F_DIA_INCH 8
param set SIH_F_RPM_MAX 9000
# sih as standard vtol
param set SIH_VEHICLE_TYPE 3
@@ -1,49 +0,0 @@
#!/bin/sh
#
# @name SIH Hexacopter X
#
# @type Simulation
# @class Copter
#
# @maintainer Romain Chiappinelli <romain.chiap@gmail.com>
#
# @board px4_fmu-v2 exclude
#
. ${R}etc/init.d/rc.mc_defaults
param set UAVCAN_ENABLE 0
# set SYS_HITL to 2 to start the SIH and avoid sensors startup
param set SYS_HITL 2
# disable some checks to allow to fly:
# - without real battery
param set-default CBRK_SUPPLY_CHK 894281
param set SIH_VEHICLE_TYPE 4
# Symmetric hexacopter X clockwise motor numbering
param set-default CA_ROTOR_COUNT 6
param set-default CA_ROTOR0_PX 0.866
param set-default CA_ROTOR0_PY 0.5
param set-default CA_ROTOR1_PX 0
param set-default CA_ROTOR1_PY 1
param set-default CA_ROTOR1_KM -0.05
param set-default CA_ROTOR2_PX -0.866
param set-default CA_ROTOR2_PY 0.5
param set-default CA_ROTOR3_PX -0.866
param set-default CA_ROTOR3_PY -0.5
param set-default CA_ROTOR3_KM -0.05
param set-default CA_ROTOR4_PX 0
param set-default CA_ROTOR4_PY -1
param set-default CA_ROTOR5_PX 0.866
param set-default CA_ROTOR5_PY -0.5
param set-default CA_ROTOR5_KM -0.05
param set-default HIL_ACT_FUNC1 101
param set-default HIL_ACT_FUNC2 102
param set-default HIL_ACT_FUNC3 103
param set-default HIL_ACT_FUNC4 104
param set-default HIL_ACT_FUNC5 105
param set-default HIL_ACT_FUNC6 106
@@ -49,7 +49,6 @@ if(CONFIG_MODULES_SIMULATION_PWM_OUT_SIM)
1101_rc_plane_sih.hil
1102_tailsitter_duo_sih.hil
1103_standard_vtol_sih.hil
1105_rc_hexa_x_sih.hil
)
if(CONFIG_MODULES_ROVER_ACKERMANN)
px4_add_romfs_files(
+46 -101
View File
@@ -31,20 +31,11 @@ set PARAM_FILE ""
set PARAM_BACKUP_FILE ""
set RC_INPUT_ARGS ""
set STORAGE_AVAILABLE no
set STORAGE_CHECK yes
set SDCARD_EXT_PATH /fs/microsd/ext_autostart
set SDCARD_FORMAT no
set STARTUP_TUNE 1
set VEHICLE_TYPE none
# Fine-grained feature gates.
set USE_HARDFAULT_LOG no
set USE_EXTERNAL_AIRFRAMES no
set USE_PARAM_BACKUPS no
set USE_PARAM_IMPORT_DEBUG no
set USE_TASK_WATCHDOG no
set USE_ALT_UPDATE_DIRS no
# Airframe parameter versioning
# Value set to 1 by default but can optionally be overridden in the airframe configuration startup script.
# Airframe maintainers can ensure a reset to the airframe defaults during an update by increasing by one.
@@ -57,81 +48,53 @@ set PARAM_DEFAULTS_VER 1
ver all
#
# Optional early board init: rc.board_early
# Can be used for setting env vars for rcS.
# Try to mount the microSD card.
#
set BOARD_RC_EARLY ${R}etc/init.d/rc.board_early
if [ -f $BOARD_RC_EARLY ]
if [ -b "/dev/mmcsd0" ]
then
. $BOARD_RC_EARLY
fi
unset BOARD_RC_EARLY
#
# Try to mount/check storage (rc.board_early can disable this).
#
if [ $STORAGE_CHECK = yes ]
then
#
# Try to mount the microSD card.
#
if [ -b "/dev/mmcsd0" ]
if mount -t vfat /dev/mmcsd0 /fs/microsd
then
if mount -t vfat /dev/mmcsd0 /fs/microsd
if [ -f "/fs/microsd/.format" ]
then
if [ -f "/fs/microsd/.format" ]
then
echo "INFO [init] format /dev/mmcsd0 requested (/fs/microsd/.format)"
set SDCARD_FORMAT yes
rm /fs/microsd/.format
umount /fs/microsd
echo "INFO [init] format /dev/mmcsd0 requested (/fs/microsd/.format)"
set SDCARD_FORMAT yes
rm /fs/microsd/.format
umount /fs/microsd
else
set STORAGE_AVAILABLE yes
fi
fi
if [ $STORAGE_AVAILABLE = no -o $SDCARD_FORMAT = yes ]
then
echo "INFO [init] formatting /dev/mmcsd0"
set STARTUP_TUNE 15 # tune 15 = SD_ERROR (overridden to SD_INIT if format + mount succeeds)
if mkfatfs -F 32 /dev/mmcsd0
then
echo "INFO [init] card formatted"
if mount -t vfat /dev/mmcsd0 /fs/microsd
then
set STORAGE_AVAILABLE yes
set STARTUP_TUNE 14 # tune 14 = SD_INIT
else
echo "ERROR [init] card mount failed"
fi
else
echo "ERROR [init] format failed"
fi
fi
else
# Is there a device mounted for storage
if mft query -q -k MTD -s MTD_PARAMETERS -v /mnt/microsd
then
else
set STORAGE_AVAILABLE yes
fi
fi
if [ $STORAGE_AVAILABLE = no -o $SDCARD_FORMAT = yes ]
then
echo "INFO [init] formatting /dev/mmcsd0"
set STARTUP_TUNE 15 # tune 15 = SD_ERROR (overridden to SD_INIT if format + mount succeeds)
if mkfatfs -F 32 /dev/mmcsd0
then
echo "INFO [init] card formatted"
if mount -t vfat /dev/mmcsd0 /fs/microsd
then
set STORAGE_AVAILABLE yes
set STARTUP_TUNE 14 # tune 14 = SD_INIT
else
echo "ERROR [init] card mount failed"
fi
else
echo "ERROR [init] format failed"
fi
fi
else
# Is there a device mounted for storage
if mft query -q -k MTD -s MTD_PARAMETERS -v /mnt/microsd
then
set STORAGE_AVAILABLE yes
fi
fi
if [ $STORAGE_AVAILABLE = yes ]
then
set USE_HARDFAULT_LOG yes
set USE_EXTERNAL_AIRFRAMES yes
set USE_PARAM_BACKUPS yes
set USE_PARAM_IMPORT_DEBUG yes
set USE_ALT_UPDATE_DIRS yes
set PARAM_FILE /fs/microsd/params
set PARAM_BACKUP_FILE "/fs/microsd/parameters_backup.bson"
fi
if [ $USE_HARDFAULT_LOG = yes ]
then
if hardfault_log check
then
@@ -141,15 +104,7 @@ then
hardfault_log reset
fi
fi
fi
if [ $USE_TASK_WATCHDOG = yes ]
then
task_watchdog start
fi
if [ $USE_ALT_UPDATE_DIRS = yes ]
then
# Check for an update of the ext_autostart folder, and replace the old one with it
if [ -e /fs/microsd/ext_autostart_new ]
then
@@ -157,6 +112,9 @@ then
rm -r $SDCARD_EXT_PATH
mv /fs/microsd/ext_autostart_new $SDCARD_EXT_PATH
fi
set PARAM_FILE /fs/microsd/params
set PARAM_BACKUP_FILE "/fs/microsd/parameters_backup.bson"
fi
#
@@ -197,11 +155,8 @@ else
if [ -d "/fs/microsd" ]
then
if [ $USE_PARAM_IMPORT_DEBUG = yes ]
then
# save copy of the failed param file for debugging
cp $PARAM_FILE /fs/microsd/param_import_fail.bson
fi
# try to make a backup copy
cp $PARAM_FILE /fs/microsd/param_import_fail.bson
# try importing from backup file
if [ -f $PARAM_BACKUP_FILE ]
@@ -219,14 +174,11 @@ else
param status
if [ $USE_PARAM_IMPORT_DEBUG = yes ]
then
dmesg >> /fs/microsd/param_import_fail.txt &
fi
dmesg >> /fs/microsd/param_import_fail.txt &
fi
fi
if [ $USE_PARAM_BACKUPS = yes ]
if [ $STORAGE_AVAILABLE = yes ]
then
param select-backup $PARAM_BACKUP_FILE
fi
@@ -282,12 +234,12 @@ else
if [ ${VEHICLE_TYPE} = none ]
then
# Run external airframe script on SD card or EEPROM-backed storage
if [ $USE_EXTERNAL_AIRFRAMES = yes ]
# Run external airframe script on SD card
if [ $STORAGE_AVAILABLE = yes ]
then
. ${R}etc/init.d/rc.autostart_ext
else
echo "ERROR [init] no external airframe storage, skipping"
echo "ERROR [init] SD not mounted, skipping external airframe"
fi
fi
@@ -727,16 +679,9 @@ unset PARAM_BACKUP_FILE
unset PARAM_DEFAULTS_VER
unset RC_INPUT_ARGS
unset STORAGE_AVAILABLE
unset STORAGE_CHECK
unset SDCARD_EXT_PATH
unset SDCARD_FORMAT
unset STARTUP_TUNE
unset USE_HARDFAULT_LOG
unset USE_EXTERNAL_AIRFRAMES
unset USE_PARAM_BACKUPS
unset USE_PARAM_IMPORT_DEBUG
unset USE_TASK_WATCHDOG
unset USE_ALT_UPDATE_DIRS
unset VEHICLE_TYPE
#
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#! /bin/bash
# exit when any command fails
set -e
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
if [[ $# -eq 0 ]] ; then
exit 0
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
# Flash PX4 to a device running AuterionOS in the local network
if [ "$1" == "-h" ] || [ "$1" == "--help" ] || [ $# -lt 2 ]; then
echo "Usage: $0 -f <firmware.px4|.elf> [-c <configuration_dir>] -d <IP/Device> [-u <user>] [-p <ssh_port>] [--revert]"
-172
View File
@@ -1,172 +0,0 @@
#!/usr/bin/env python3
"""
Shared GitHub REST helpers for PX4 CI scripts.
This module is imported by the PR poster scripts under Tools/ci/. It is
NOT an executable entry point; do not run it directly.
Provides:
- fail(msg) terminates the caller with a clear error
- GitHubClient(token) thin stdlib-only GitHub REST client with
single-request and paginated helpers
Python stdlib only. No third-party dependencies.
History: extracted from Tools/ci/pr-comment-poster.py so that
pr-comment-poster.py and pr-review-poster.py share the same HTTP plumbing
without duplicating ~100 lines of request/pagination/error-handling code.
"""
import json
import sys
import typing
import urllib.error
import urllib.request
GITHUB_API = 'https://api.github.com'
DEFAULT_USER_AGENT = 'px4-ci'
API_VERSION = '2022-11-28'
def fail(msg: str) -> typing.NoReturn:
"""Print an error to stderr and exit with status 1.
Annotated NoReturn so static checkers understand control does not
continue past a fail() call.
"""
print('error: {}'.format(msg), file=sys.stderr)
sys.exit(1)
def _parse_next_link(link_header):
"""Return the URL for rel="next" from an RFC 5988 Link header, or None.
The Link header is comma-separated entries of the form:
<https://...?page=2>; rel="next", <https://...?page=5>; rel="last"
We walk each entry and return the URL of the one whose rel attribute is
"next". Accept single-quoted rel values for robustness even though
GitHub always emits double quotes.
"""
if not link_header:
return None
for part in link_header.split(','):
segs = part.strip().split(';')
if len(segs) < 2:
continue
url_seg = segs[0].strip()
if not (url_seg.startswith('<') and url_seg.endswith('>')):
continue
url = url_seg[1:-1]
for attr in segs[1:]:
attr = attr.strip()
if attr == 'rel="next"' or attr == "rel='next'":
return url
return None
class GitHubClient:
"""Minimal GitHub REST client backed by the Python stdlib.
Each instance holds a token and a user-agent so callers do not have to
thread them through every call. Methods return parsed JSON (or None for
empty responses) and raise RuntimeError with the server response body on
HTTP errors, so CI logs show what the API actually objected to.
Usage:
client = GitHubClient(token, user_agent='px4-pr-comment-poster')
body, headers = client.request('GET', 'repos/{o}/{r}/pulls/123')
for item in client.paginated('repos/{o}/{r}/pulls/123/reviews'):
...
"""
def __init__(self, token, user_agent=DEFAULT_USER_AGENT):
if not token:
raise ValueError('GitHub token is required')
self._token = token
self._user_agent = user_agent
def request(self, method, path_or_url, json_body=None):
"""GET/POST/PATCH/PUT/DELETE a single API path or absolute URL.
`path_or_url` may be either a relative API path (e.g.
"repos/PX4/PX4-Autopilot/pulls/123") or an absolute URL such as the
next-page URL returned from paginated results. Relative paths are
prefixed with the GitHub API base.
Returns (parsed_json_or_none, headers_dict). Raises RuntimeError
on HTTP or transport errors.
"""
url = self._resolve(path_or_url)
return self._do_request(method, url, json_body)
def paginated(self, path, per_page=100):
"""GET a path and follow rel="next" Link headers.
Yields items from each page's JSON array. Bumps per_page to 100
(GitHub's max) so large result sets take fewer round-trips.
Raises RuntimeError if any page response is not a JSON array.
"""
url = self._resolve(path)
sep = '&' if '?' in url else '?'
url = '{}{}per_page={}'.format(url, sep, per_page)
while url is not None:
body, headers = self._do_request('GET', url, None)
if body is None:
return
if not isinstance(body, list):
raise RuntimeError(
'expected JSON array from {}, got {}'.format(
url, type(body).__name__))
for item in body:
yield item
url = _parse_next_link(headers.get('Link'))
def _resolve(self, path_or_url):
if path_or_url.startswith('http://') or path_or_url.startswith('https://'):
return path_or_url
return '{}/{}'.format(GITHUB_API.rstrip('/'), path_or_url.lstrip('/'))
def _do_request(self, method, url, json_body):
data = None
headers = {
'Authorization': 'Bearer {}'.format(self._token),
'Accept': 'application/vnd.github+json',
# Pin the API version so GitHub deprecations don't silently
# change the response shape under us.
'X-GitHub-Api-Version': API_VERSION,
'User-Agent': self._user_agent,
}
if json_body is not None:
data = json.dumps(json_body).encode('utf-8')
headers['Content-Type'] = 'application/json; charset=utf-8'
req = urllib.request.Request(
url, data=data, method=method, headers=headers)
try:
with urllib.request.urlopen(req) as resp:
raw = resp.read()
# HTTPMessage is case-insensitive on lookup but its items()
# preserves the original case. GitHub sends "Link" with a
# capital L, which is what _parse_next_link expects.
resp_headers = dict(resp.headers.items())
if not raw:
return None, resp_headers
return json.loads(raw.decode('utf-8')), resp_headers
except urllib.error.HTTPError as e:
# GitHub error bodies are JSON with a "message" field and often
# a "documentation_url". Dump the raw body into the exception so
# the CI log shows exactly what the API objected to. A bare
# "HTTP 422" tells us nothing useful.
try:
err_body = e.read().decode('utf-8', errors='replace')
except Exception:
err_body = '(no body)'
raise RuntimeError(
'GitHub API {} {} failed: HTTP {} {}\n{}'.format(
method, url, e.code, e.reason, err_body))
except urllib.error.URLError as e:
# Network layer failure (DNS, TLS, connection reset). No HTTP
# response to parse; just surface the transport reason.
raise RuntimeError(
'GitHub API {} {} failed: {}'.format(method, url, e.reason))
-71
View File
@@ -1,71 +0,0 @@
# Build All Targets CI Configuration
#
# Controls board grouping, cache sizes, runner specs, and seeder targets
# for the build_all_targets workflow. Forks can customize this file to
# adjust for their infrastructure (e.g., lower cache sizes for GitHub's
# 10GB cache limit, fewer CPU cores for smaller runners).
# Container images
containers:
default: "ghcr.io/px4/px4-dev:v1.17.0-rc2"
voxl2: "ghcr.io/px4/px4-dev-voxl2:v1.7"
# Runner specs
runners:
seeder_cpu: 8
matrix_cpu: 4
# Default ccache max-size for build groups
cache:
default_size: "400M"
# Per-chip overrides for groups with many diverse boards
chip_sizes:
stm32h7: "800M"
stm32f4: "800M"
stm32f7: "800M"
imxrt: "800M"
# Board grouping limits
grouping:
# Max targets per group, tuned for ~10 min wall-clock with warm cache
chip_split_limits:
stm32h7: 10
stm32f7: 12
stm32f4: 20
stm32f1: 39
imxrt: 12
kinetis: 14
s32k: 17
rp2040: 10
special: 10
native: 17
default_split_limit: 12
# Minimum targets for a manufacturer to get a named group
lower_limit: 3
# If last chunk has fewer targets than this, merge into previous chunk
merge_back_threshold: 5
# Labels that isolate builds into the "special" group
special_labels:
- lto
- protected
# NXP chip families are pooled under "nxp-{chip}" regardless of board directory
nxp_chip_families:
- imxrt
- kinetis
- s32k
# Seeder targets: one representative build per chip family
seeders:
stm32h7: "px4_fmu-v6x_default"
stm32f7: "px4_fmu-v5_default"
stm32f4: "px4_fmu-v4_default"
stm32f1: "px4_io-v2_default"
imxrt: "nxp_mr-tropic_default"
kinetis: "nxp_fmuk66-v3_default"
s32k: "nxp_mr-canhubk3_default"
rp2040: "raspberrypi_pico_default"
special: "px4_fmu-v6x_default"
native: "px4_sitl_default"
voxl2: "modalai_voxl2_default"
+3 -1
View File
@@ -1,8 +1,9 @@
#!/usr/bin/env bash
#!/bin/bash
# This script is meant to be used by the build_all.yml workflow in a github runner
# Please only modify if you know what you are doing
set -e
echo "### :clock1: Build Times" >> $GITHUB_STEP_SUMMARY
targets=$1
for target in ${targets//,/ }
do
@@ -13,5 +14,6 @@ do
diff=$(($stop-$start))
build_time="$(($diff /60/60))h $(($diff /60))m $(($diff % 60))s elapsed"
echo -e "\033[0;32mBuild Time: [$build_time]"
echo "* **$target** - $build_time" >> $GITHUB_STEP_SUMMARY
echo "::endgroup::"
done
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#! /bin/bash
# Copy a git diff between two commits if msg versioning is added
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
-111
View File
@@ -1,111 +0,0 @@
#!/usr/bin/env python3
"""
Filter a git diff for consumption by clang-tidy-diff.
Produces a unified diff containing only files that clang-tidy can
actually analyze against the current compilation database:
- C/C++ source files (.c, .cpp, .cc, .cxx, .m, .mm) must be present
in compile_commands.json. Files absent from the database are test
files, excluded code, or platform-specific sources that were not
compiled. Feeding them to clang-tidy-diff produces spurious
"header not found" errors (gtest/gtest.h in particular).
- Header files (.h, .hpp, .hxx) always pass through. clang-tidy
analyzes header changes via the TUs that include them; there is
no separate TU for a header to match against the database.
- All other files (CMakeLists.txt, .yml, .md, etc.) are dropped.
Output is a unified diff suitable for piping into clang-tidy-diff.py.
If nothing remains, the output file is empty.
Used by .github/workflows/clang-tidy.yml as a pre-filter for the
`pr-review` artifact producer. Python stdlib only.
"""
import argparse
import json
import os
import subprocess
import sys
SOURCE_EXTS = {'.c', '.cpp', '.cc', '.cxx', '.m', '.mm'}
HEADER_EXTS = {'.h', '.hpp', '.hxx'}
def load_db_files(build_dir):
"""Return the set of source paths (repo-relative) in compile_commands.json."""
path = os.path.join(build_dir, 'compile_commands.json')
with open(path) as f:
db = json.load(f)
root = os.path.abspath('.')
prefix = root + os.sep
paths = set()
for entry in db:
p = entry.get('file', '')
if p.startswith(prefix):
paths.add(p[len(prefix):])
else:
# Relative or external path; record as-is
paths.add(p)
return paths
def changed_files(base_ref):
out = subprocess.check_output(
['git', 'diff', '--name-only', '{}...HEAD'.format(base_ref)],
text=True,
)
return [line.strip() for line in out.splitlines() if line.strip()]
def keep_file(path, db_files):
"""Decide whether to keep this path in the filtered diff."""
ext = os.path.splitext(path)[1].lower()
if ext in HEADER_EXTS:
return True
if ext in SOURCE_EXTS:
return path in db_files
return False
def filtered_diff(base_ref, keep_paths):
if not keep_paths:
return ''
cmd = ['git', 'diff', '-U0', '{}...HEAD'.format(base_ref), '--'] + sorted(keep_paths)
return subprocess.check_output(cmd, text=True)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--build-dir', required=True,
help='CMake build dir containing compile_commands.json')
parser.add_argument('--base-ref', required=True,
help='Git ref to diff against (e.g. origin/main)')
parser.add_argument('--out', required=True,
help='Output path for the filtered unified diff')
args = parser.parse_args()
db_files = load_db_files(args.build_dir)
changed = changed_files(args.base_ref)
keep = [p for p in changed if keep_file(p, db_files)]
dropped = [p for p in changed if p not in keep]
print('clang-tidy-diff-filter: kept {} of {} changed files'.format(
len(keep), len(changed)))
if dropped:
print(' dropped (not in compile_commands.json or not source/header):')
for p in dropped:
print(' {}'.format(p))
diff = filtered_diff(args.base_ref, keep)
with open(args.out, 'w') as f:
f.write(diff)
return 0
if __name__ == '__main__':
sys.exit(main())
-539
View File
@@ -1,539 +0,0 @@
#!/usr/bin/env python3
#
# clang-tidy-fixes-to-review.py
#
# Producer-side helper that converts a clang-tidy fixes.yml file into a
# pr-review artifact (manifest.json + comments.json) suitable for
# Tools/ci/pr-review-poster.py.
#
# This script runs inside the clang-tidy job's px4-dev container so it can
# read the source tree directly and look up byte offsets in the original
# files. The output it writes is a fully-baked array of review comments;
# the poster never reads source files or fixes.yml.
#
# ----------------------------------------------------------------------------
# ATTRIBUTION
# ----------------------------------------------------------------------------
# This script reuses the diagnostic-to-review-comment translation logic
# from platisd/clang-tidy-pr-comments. The original work is:
#
# MIT License
#
# Copyright (c) 2021 Dimitris Platis
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be included
# in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#
# Adapted parts:
# - get_diff_line_ranges_per_file() and its inner change_to_line_range()
# - generate_review_comments() and its nested helpers
# (get_line_by_offset, validate_warning_applicability,
# calculate_replacements_diff, markdown, markdown_url,
# diagnostic_name_visual, generate_single_comment)
# - reorder_diagnostics()
#
# Removed parts (handled by Tools/ci/pr-review-poster.py instead):
# - post_review_comments / dismiss_change_requests / resolve_conversations
# - the original argparse main and the requests-based HTTP layer
#
# Adaptation notes:
# - The HTTP layer is rewritten on top of Tools/ci/_github_helpers.py so
# this script does not depend on the third-party `requests` package.
# - Conversation resolution (the GraphQL path) is intentionally dropped
# for v1; revisit if it turns out to be missed.
# - Clang-Tidy 8 upconvert is preserved verbatim.
#
# ----------------------------------------------------------------------------
# Bounded assumptions (documented for future maintainers):
# - Source files are UTF-8 (we read them as latin_1, matching clang-tidy's
# own byte-offset model, and the offsets we surface are line counts)
# - Source files use LF line endings
# - Malformed entries in fixes.yml are skipped with a warning rather than
# crashing the job
#
# Dependencies: pyyaml + Tools/ci/_github_helpers.py.
# pyyaml is preinstalled in the px4-dev container; this script is intended
# to run there, not on bare ubuntu-latest.
"""Convert a clang-tidy fixes.yml into a pr-review artifact."""
import argparse
import difflib
import json
import os
import posixpath
import re
import sys
import urllib.parse
import yaml
import _github_helpers
from _github_helpers import fail as _fail
# Markers used inside the per-comment body to call out severity. Plain
# strings rather than emojis to keep the file emoji-free per project
# preferences; the rendered Markdown is unaffected.
SINGLE_COMMENT_MARKERS = {
'Error': '**[error]**',
'Warning': '**[warning]**',
'Remark': '**[remark]**',
'fallback': '**[note]**',
}
# ---------------------------------------------------------------------------
# Diff-range parsing (adapted from platisd)
# ---------------------------------------------------------------------------
def get_diff_line_ranges_per_file(pr_files):
"""Return a dict mapping each PR file path to a list of line ranges
(the +new-side hunks) parsed from its patch."""
def change_to_line_range(change):
split_change = change.split(',')
start = int(split_change[0])
size = int(split_change[1]) if len(split_change) > 1 else 1
return range(start, start + size)
result = {}
for pr_file in pr_files:
# Removed binary files etc. have no patch section.
if 'patch' not in pr_file:
continue
file_name = pr_file['filename']
# Match lines like '@@ -101,8 +102,11 @@'
git_line_tags = re.findall(
r'^@@ -.*? +.*? @@', pr_file['patch'], re.MULTILINE)
changes = [
tag.replace('@@', '').strip().split()[1].replace('+', '')
for tag in git_line_tags
]
result[file_name] = [
change_to_line_range(change) for change in changes
]
return result
def fetch_pull_request_files(client, repo, pr_number):
"""Yield file metadata objects for each file modified by the PR."""
path = 'repos/{}/pulls/{}/files'.format(repo, pr_number)
for entry in client.paginated(path):
yield entry
# ---------------------------------------------------------------------------
# Diagnostic ordering (adapted from platisd)
# ---------------------------------------------------------------------------
def reorder_diagnostics(diags):
"""Return diagnostics ordered Error -> Warning -> Remark -> other."""
errors = [d for d in diags if d.get('Level') == 'Error']
warnings = [d for d in diags if d.get('Level') == 'Warning']
remarks = [d for d in diags if d.get('Level') == 'Remark']
others = [
d for d in diags
if d.get('Level') not in {'Error', 'Warning', 'Remark'}
]
if others:
print(
'warning: some fixes have an unexpected Level (not Error, '
'Warning, or Remark)', file=sys.stderr)
return errors + warnings + remarks + others
# ---------------------------------------------------------------------------
# Comment generation (adapted from platisd)
# ---------------------------------------------------------------------------
def generate_review_comments(clang_tidy_fixes, repository_root,
diff_line_ranges_per_file,
single_comment_markers):
"""Yield review comment dicts for each clang-tidy diagnostic that
intersects the PR diff."""
def get_line_by_offset(file_path, offset):
# Clang-Tidy doesn't support multibyte encodings and measures
# offsets in bytes; latin_1 makes byte offsets and string offsets
# equivalent.
with open(repository_root + file_path, encoding='latin_1') as fh:
source = fh.read()
return source[:offset].count('\n') + 1
def validate_warning_applicability(file_path, start_line_num, end_line_num):
assert end_line_num >= start_line_num
for line_range in diff_line_ranges_per_file[file_path]:
assert line_range.step == 1
if (line_range.start <= start_line_num
and end_line_num < line_range.stop):
return True
return False
def calculate_replacements_diff(file_path, replacements):
# Apply replacements in reverse order so subsequent offsets do not
# shift.
replacements.sort(key=lambda item: (-item['Offset']))
with open(repository_root + file_path, encoding='latin_1') as fh:
source = fh.read()
changed = source
for replacement in replacements:
changed = (
changed[:replacement['Offset']]
+ replacement['ReplacementText']
+ changed[replacement['Offset'] + replacement['Length']:]
)
return difflib.Differ().compare(
source.splitlines(keepends=True),
changed.splitlines(keepends=True),
)
def markdown(s):
md_chars = '\\`*_{}[]<>()#+-.!|'
def escape_chars(s):
for ch in md_chars:
s = s.replace(ch, '\\' + ch)
return s
def unescape_chars(s):
for ch in md_chars:
s = s.replace('\\' + ch, ch)
return s
s = escape_chars(s)
s = re.sub(
"'([^']*)'",
lambda m: '`` ' + unescape_chars(m.group(1)) + ' ``',
s,
)
return s
def markdown_url(label, url):
return '[{}]({})'.format(label, url)
def diagnostic_name_visual(diagnostic_name):
visual = '**{}**'.format(markdown(diagnostic_name))
try:
first_dash_idx = diagnostic_name.index('-')
except ValueError:
return visual
namespace = urllib.parse.quote_plus(diagnostic_name[:first_dash_idx])
check_name = urllib.parse.quote_plus(
diagnostic_name[first_dash_idx + 1:])
return markdown_url(
visual,
'https://clang.llvm.org/extra/clang-tidy/checks/{}/{}.html'.format(
namespace, check_name),
)
def generate_single_comment(file_path, start_line_num, end_line_num,
name, message, single_comment_marker,
replacement_text=None):
result = {
'path': file_path,
'line': end_line_num,
'side': 'RIGHT',
'body': '{} {} {}\n{}'.format(
single_comment_marker,
diagnostic_name_visual(name),
single_comment_marker,
markdown(message),
),
}
if start_line_num != end_line_num:
result['start_line'] = start_line_num
result['start_side'] = 'RIGHT'
if replacement_text is not None:
if not replacement_text or replacement_text[-1] != '\n':
replacement_text += '\n'
result['body'] += '\n```suggestion\n{}```'.format(replacement_text)
return result
for diag in clang_tidy_fixes['Diagnostics']:
# Upconvert clang-tidy 8 format to 9+
if 'DiagnosticMessage' not in diag:
diag['DiagnosticMessage'] = {
'FileOffset': diag.get('FileOffset'),
'FilePath': diag.get('FilePath'),
'Message': diag.get('Message'),
'Replacements': diag.get('Replacements', []),
}
diag_message = diag['DiagnosticMessage']
diag_message['FilePath'] = posixpath.normpath(
(diag_message.get('FilePath') or '').replace(repository_root, ''))
for replacement in diag_message.get('Replacements') or []:
replacement['FilePath'] = posixpath.normpath(
replacement['FilePath'].replace(repository_root, ''))
diag_name = diag.get('DiagnosticName', '<unknown>')
diag_message_msg = diag_message.get('Message', '')
level = diag.get('Level', 'Warning')
single_comment_marker = single_comment_markers.get(
level, single_comment_markers['fallback'])
replacements = diag_message.get('Replacements') or []
if not replacements:
file_path = diag_message['FilePath']
offset = diag_message.get('FileOffset')
if offset is None:
print('warning: skipping {!r}: missing FileOffset'.format(
diag_name), file=sys.stderr)
continue
if file_path not in diff_line_ranges_per_file:
print(
"'{}' for {} does not apply to the files changed in "
'this PR'.format(diag_name, file_path))
continue
try:
line_num = get_line_by_offset(file_path, offset)
except (OSError, ValueError) as e:
print('warning: skipping {!r} on {}: {}'.format(
diag_name, file_path, e), file=sys.stderr)
continue
print("Processing '{}' at line {} of {}...".format(
diag_name, line_num, file_path))
if validate_warning_applicability(file_path, line_num, line_num):
yield generate_single_comment(
file_path,
line_num,
line_num,
diag_name,
diag_message_msg,
single_comment_marker=single_comment_marker,
)
else:
print('This warning does not apply to the lines changed '
'in this PR')
else:
for file_path in {item['FilePath'] for item in replacements}:
if file_path not in diff_line_ranges_per_file:
print(
"'{}' for {} does not apply to the files changed "
'in this PR'.format(diag_name, file_path))
continue
line_num = 1
start_line_num = None
end_line_num = None
replacement_text = None
try:
diff_iter = calculate_replacements_diff(
file_path,
[r for r in replacements if r['FilePath'] == file_path],
)
except (OSError, ValueError) as e:
print('warning: skipping {!r} on {}: {}'.format(
diag_name, file_path, e), file=sys.stderr)
continue
for line in diff_iter:
# Comment line, ignore.
if line.startswith('? '):
continue
# A '-' line is the start or continuation of a region
# to replace.
if line.startswith('- '):
if start_line_num is None:
start_line_num = line_num
end_line_num = line_num
else:
end_line_num = line_num
if replacement_text is None:
replacement_text = ''
line_num += 1
# A '+' line is part of the replacement text.
elif line.startswith('+ '):
if replacement_text is None:
replacement_text = line[2:]
else:
replacement_text += line[2:]
# A context line marks the end of a replacement region.
elif line.startswith(' '):
if replacement_text is not None:
if start_line_num is None:
# Pure addition: synthesize a one-line
# range and append the context line to
# the replacement.
start_line_num = line_num
end_line_num = line_num
replacement_text += line[2:]
print("Processing '{}' at lines {}-{} of {}...".format(
diag_name, start_line_num, end_line_num, file_path))
if validate_warning_applicability(
file_path, start_line_num, end_line_num):
yield generate_single_comment(
file_path,
start_line_num,
end_line_num,
diag_name,
diag_message_msg,
single_comment_marker=single_comment_marker,
replacement_text=replacement_text,
)
else:
print(
'This warning does not apply to the '
'lines changed in this PR')
start_line_num = None
end_line_num = None
replacement_text = None
line_num += 1
else:
# Unknown difflib prefix; skip rather than abort.
print('warning: unexpected diff prefix {!r}; '
'skipping diagnostic'.format(line[:2]),
file=sys.stderr)
break
# End of file with a pending replacement region.
if replacement_text is not None and start_line_num is not None:
print("Processing '{}' at lines {}-{} of {}...".format(
diag_name, start_line_num, end_line_num, file_path))
if validate_warning_applicability(
file_path, start_line_num, end_line_num):
yield generate_single_comment(
file_path,
start_line_num,
end_line_num,
diag_name,
diag_message_msg,
single_comment_marker=single_comment_marker,
replacement_text=replacement_text,
)
else:
print('This warning does not apply to the lines '
'changed in this PR')
# ---------------------------------------------------------------------------
# Entry point
# ---------------------------------------------------------------------------
def main(argv=None):
parser = argparse.ArgumentParser(
description='Convert a clang-tidy fixes.yml into a pr-review '
'artifact (manifest.json + comments.json).',
)
parser.add_argument('--fixes', required=True,
help='Path to fixes.yml from clang-tidy')
parser.add_argument('--repo-root', required=True,
help='Path to the repository root containing the '
'source files referenced by fixes.yml')
parser.add_argument('--repo', required=True,
help='owner/name of the repository')
parser.add_argument('--pr-number', required=True, type=int,
help='Pull request number')
parser.add_argument('--commit-sha', required=True,
help='40-char hex commit SHA the review will pin to')
parser.add_argument('--out-dir', required=True,
help='Directory to write manifest.json and '
'comments.json')
parser.add_argument(
'--marker',
default='<!-- pr-review-poster:clang-tidy -->',
help='Marker string embedded in the review body so the poster '
'can find and dismiss stale runs')
parser.add_argument(
'--event',
default='REQUEST_CHANGES',
choices=('COMMENT', 'REQUEST_CHANGES'),
help='GitHub review event type')
parser.add_argument(
'--summary', default='',
help='Optional review summary text appended to the review body')
args = parser.parse_args(argv)
if args.pr_number <= 0:
_fail('--pr-number must be > 0')
if not re.match(r'^[0-9a-f]{40}$', args.commit_sha):
_fail('--commit-sha must be a 40-char lowercase hex string')
token = os.environ.get('GITHUB_TOKEN')
if not token:
_fail('GITHUB_TOKEN is not set')
# Normalize the repo root with a trailing slash so the platisd-style
# str.replace() trick still strips it cleanly.
repo_root = args.repo_root
if not repo_root.endswith(os.sep):
repo_root = repo_root + os.sep
os.makedirs(args.out_dir, exist_ok=True)
client = _github_helpers.GitHubClient(token, user_agent='px4-clang-tidy-fixes-to-review')
print('Fetching PR file list from GitHub...')
pr_files = list(fetch_pull_request_files(client, args.repo, args.pr_number))
diff_line_ranges_per_file = get_diff_line_ranges_per_file(pr_files)
print('Loading clang-tidy fixes from {}...'.format(args.fixes))
if not os.path.isfile(args.fixes):
# No fixes file means clang-tidy ran cleanly. Emit an empty
# comments.json so the poster can short-circuit.
comments = []
else:
with open(args.fixes, encoding='utf-8') as fh:
clang_tidy_fixes = yaml.safe_load(fh)
if (not clang_tidy_fixes
or 'Diagnostics' not in clang_tidy_fixes
or not clang_tidy_fixes['Diagnostics']):
comments = []
else:
clang_tidy_fixes['Diagnostics'] = reorder_diagnostics(
clang_tidy_fixes['Diagnostics'])
comments = list(generate_review_comments(
clang_tidy_fixes,
repo_root,
diff_line_ranges_per_file,
single_comment_markers=SINGLE_COMMENT_MARKERS,
))
print('Generated {} review comment(s)'.format(len(comments)))
manifest = {
'pr_number': args.pr_number,
'marker': args.marker,
'event': args.event,
'commit_sha': args.commit_sha,
}
if args.summary:
manifest['summary'] = args.summary
manifest_path = os.path.join(args.out_dir, 'manifest.json')
comments_path = os.path.join(args.out_dir, 'comments.json')
with open(manifest_path, 'w', encoding='utf-8') as fh:
json.dump(manifest, fh, indent=2)
fh.write('\n')
with open(comments_path, 'w', encoding='utf-8') as fh:
json.dump(comments, fh, indent=2)
fh.write('\n')
print('Wrote {} and {}'.format(manifest_path, comments_path))
return 0
if __name__ == '__main__':
sys.exit(main())
+87 -360
View File
@@ -16,7 +16,6 @@ kconf.warn_assign_override = False
kconf.warn_assign_redun = False
source_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')
boards_dir = os.path.join(source_dir, '..', 'boards')
parser = argparse.ArgumentParser(description='Generate build targets')
@@ -27,8 +26,6 @@ parser.add_argument('-p', '--pretty', dest='pretty', action='store_true',
parser.add_argument('-g', '--groups', dest='group', action='store_true',
help='Groups targets')
parser.add_argument('-f', '--filter', dest='filter', help='comma separated list of build target name prefixes to include instead of all e.g. "px4_fmu-v5_"')
parser.add_argument('-s', '--seeders', dest='seeders', action='store_true',
help='Output seeder matrix JSON (one entry per chip family)')
args = parser.parse_args()
verbose = args.verbose
@@ -38,14 +35,8 @@ if args.filter:
for target in args.filter.split(','):
target_filter.append(target)
# Load CI configuration from YAML
import yaml
ci_config_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'build_all_config.yml')
with open(ci_config_path) as f:
ci_config = yaml.safe_load(f)
default_container = ci_config['containers']['default']
voxl2_container = ci_config['containers']['voxl2']
default_container = 'ghcr.io/px4/px4-dev:v1.16.0-rc1-258-g0369abd556'
voxl2_container = 'ghcr.io/px4/px4-dev-voxl2:v1.5'
build_configs = []
grouped_targets = {}
excluded_boards = ['px4_ros2', 'espressif_esp32'] # TODO: fix and enable
@@ -65,71 +56,6 @@ excluded_labels = [
'uavcanv1', # TODO: fix and enable
]
# Labels that mark isolated/special builds (poor cache reuse with normal builds)
special_labels = ci_config.get('special_labels', ['lto', 'protected'])
def detect_chip_family(manufacturer_name, board_name, label):
"""Detect the chip family for a board by reading its NuttX defconfig.
Returns a chip family string used for cache grouping:
stm32h7, stm32f7, stm32f4, stm32f1, imxrt, kinetis, s32k, rp2040, native, special
"""
# Special labels get their own group regardless of chip
if label in special_labels:
return 'special'
board_path = os.path.join(boards_dir, manufacturer_name, board_name)
nsh_defconfig = os.path.join(board_path, 'nuttx-config', 'nsh', 'defconfig')
if not os.path.exists(nsh_defconfig):
# Try bootloader defconfig as fallback
bl_defconfig = os.path.join(board_path, 'nuttx-config', 'bootloader', 'defconfig')
if os.path.exists(bl_defconfig):
nsh_defconfig = bl_defconfig
else:
return 'native'
arch_chip = None
specific_chip = None
with open(nsh_defconfig) as f:
for line in f:
line = line.strip()
if line.startswith('CONFIG_ARCH_CHIP='):
arch_chip = line.split('=')[1].strip('"')
elif line.startswith('CONFIG_ARCH_CHIP_STM32F') and line.endswith('=y'):
specific_chip = line.split('=')[0].replace('CONFIG_ARCH_CHIP_', '')
if arch_chip is None:
return 'native'
# Direct matches for chips that have unique CONFIG_ARCH_CHIP values
if arch_chip == 'stm32h7':
return 'stm32h7'
elif arch_chip == 'stm32f7':
return 'stm32f7'
elif arch_chip == 'imxrt':
return 'imxrt'
elif arch_chip == 'kinetis':
return 'kinetis'
elif arch_chip.startswith('s32k'):
return 's32k'
elif arch_chip == 'rp2040':
return 'rp2040'
elif arch_chip == 'stm32':
# Disambiguate STM32 sub-families using specific chip define
if specific_chip:
if specific_chip.startswith('STM32F1'):
return 'stm32f1'
elif specific_chip.startswith('STM32F4'):
return 'stm32f4'
else:
return 'stm32f4' # Default STM32 to F4
return 'stm32f4'
else:
return 'native'
target_chip_families = {} # target_name -> chip_family mapping
github_action_config = { 'include': build_configs }
extra_args = {}
if args.pretty:
@@ -140,21 +66,11 @@ def chunks(arr, size):
for i in range(0, len(arr), size):
yield arr[i:i + size]
MERGE_BACK_THRESHOLD = 5
def chunks_merged(arr, size):
"""Split array into chunks, merging the last chunk back if it's too small."""
result = list(chunks(arr, size))
if len(result) > 1 and len(result[-1]) < MERGE_BACK_THRESHOLD:
result[-2] = result[-2] + result[-1]
result.pop()
return result
def comma_targets(targets):
# turns array of targets into a comma split string
return ",".join(targets)
def process_target(px4board_file, target_name, manufacturer_name=None, board_dir_name=None, label=None):
def process_target(px4board_file, target_name):
# reads through the board file and grabs
# useful information for building
ret = None
@@ -191,16 +107,6 @@ def process_target(px4board_file, target_name, manufacturer_name=None, board_dir
if board_name in board_container_overrides:
container = board_container_overrides[board_name]
# Detect chip family for cache grouping
chip_family = 'native'
if manufacturer_name and board_dir_name:
if platform == 'nuttx':
chip_family = detect_chip_family(manufacturer_name, board_dir_name, label or '')
elif board_name in board_container_overrides or platform in platform_container_overrides:
chip_family = 'native' # voxl2/qurt targets
else:
chip_family = 'native'
# Boards with container overrides get their own group
if board_name in board_container_overrides or platform in platform_container_overrides:
group = 'voxl2'
@@ -218,7 +124,7 @@ def process_target(px4board_file, target_name, manufacturer_name=None, board_dir
else:
if verbose: print(f'unmatched platform: {platform}')
ret = {'target': target_name, 'container': container, 'chip_family': chip_family}
ret = {'target': target_name, 'container': container}
if(args.group):
ret['arch'] = group
@@ -241,8 +147,6 @@ grouped_targets['base']['container'] = default_container
grouped_targets['base']['manufacturers'] = {}
grouped_targets['base']['manufacturers']['px4'] = []
grouped_targets['base']['manufacturers']['px4'] += metadata_targets
for mt in metadata_targets:
target_chip_families[mt] = 'native'
for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), key=lambda e: e.name):
if not manufacturer.is_dir():
@@ -273,10 +177,7 @@ for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), ke
if label in excluded_labels:
if verbose: print(f'excluding label {label} ({target_name})')
continue
target = process_target(files.path, target_name,
manufacturer_name=manufacturer.name,
board_dir_name=board.name,
label=label)
target = process_target(files.path, target_name)
if (args.group and target is not None):
if (target['arch'] not in grouped_targets):
grouped_targets[target['arch']] = {}
@@ -285,7 +186,6 @@ for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), ke
if(manufacturer.name not in grouped_targets[target['arch']]['manufacturers']):
grouped_targets[target['arch']]['manufacturers'][manufacturer.name] = []
grouped_targets[target['arch']]['manufacturers'][manufacturer.name].append(target_name)
target_chip_families[target_name] = target['chip_family']
if target is not None:
build_configs.append(target)
@@ -346,9 +246,6 @@ for manufacturer in sorted(os.scandir(os.path.join(source_dir, '../boards')), ke
if manufacturer.name not in grouped_targets[group]['manufacturers']:
grouped_targets[group]['manufacturers'][manufacturer.name] = []
grouped_targets[group]['manufacturers'][manufacturer.name].append(deb_target)
# Inherit chip_family from the default target
default_chip = target_chip_families.get(default_target, 'native')
target_chip_families[deb_target] = default_chip
build_configs.append(target_entry)
if(verbose):
@@ -364,227 +261,109 @@ if(verbose):
print("===================")
if (args.group):
# Group targets by chip family for better ccache reuse.
# Targets sharing the same MCU family (e.g. stm32h7) benefit from
# a shared ccache seed since they compile the same NuttX kernel and HAL.
#
# Grouping strategy:
# 1. Collect all targets per (arch, chip_family, manufacturer)
# 2. Within each chip_family, large manufacturers get their own groups
# named "{manufacturer}-{chip_family}[-N]"
# 3. Small manufacturers are merged into "misc-{chip_family}[-N]"
# 4. Special groups: "special" (lto/protected/allyes), "io" (stm32f1),
# "voxl2-0" (unchanged)
# 5. Non-NuttX groups: "base-N", "aarch64-N", "armhf-N" (unchanged)
# if we are using this script for grouping builds
# we loop trough the manufacturers list and split their targets
# if a manufacturer has more than a LIMIT of boards then we split that
# into sub groups such as "arch-manufacturer name-index"
# example:
# nuttx-px4-0
# nuttx-px4-1
# nuttx-px4-2
# nuttx-ark-0
# nuttx-ark-1
# if the manufacturer doesn't have more targets than LIMIT then we add
# them to a generic group with the following structure "arch-index"
# example:
# nuttx-0
# nuttx-1
final_groups = []
# Load grouping and cache config
grouping_config = ci_config.get('grouping', {})
CHIP_SPLIT_LIMITS = grouping_config.get('chip_split_limits', {})
DEFAULT_SPLIT_LIMIT = grouping_config.get('default_split_limit', 12)
LOWER_LIMIT = grouping_config.get('lower_limit', 3)
cache_config = ci_config.get('cache', {})
DEFAULT_CACHE_SIZE = cache_config.get('default_size', '400M')
CHIP_CACHE_SIZES = cache_config.get('chip_sizes', {})
last_man = ''
last_arch = ''
SPLIT_LIMIT = 10
LOWER_LIMIT = 5
if(verbose):
print(f'=:Architectures: [{grouped_targets.keys()}]')
for arch in grouped_targets:
runner = 'x64'
# armhf and aarch64 Linux boards need the arm64 container image
# which ships the arm-linux-gnueabihf and aarch64-linux-gnu cross compilers
# (the x64 container image does not include them)
if arch in ('armhf', 'aarch64'):
runner = 'arm64'
runner = 'x64' if arch in ('nuttx', 'voxl2') else 'arm64'
if(verbose):
print(f'=:Processing: [{arch}]')
if arch == 'nuttx':
# Re-bucket NuttX targets by chip_family then manufacturer
chip_man_buckets = {} # (chip_family, manufacturer) -> [target_names]
for man in grouped_targets[arch]['manufacturers']:
for target in grouped_targets[arch]['manufacturers'][man]:
chip = target_chip_families.get(target, 'native')
key = (chip, man)
if key not in chip_man_buckets:
chip_man_buckets[key] = []
chip_man_buckets[key].append(target)
# Collect all chip families present
chip_families_seen = sorted(set(k[0] for k in chip_man_buckets.keys()))
for chip in chip_families_seen:
SPLIT_LIMIT = CHIP_SPLIT_LIMITS.get(chip, DEFAULT_SPLIT_LIMIT)
# Special naming for certain chip families
if chip == 'special':
chip_label = 'special'
elif chip == 'stm32f1':
chip_label = 'io'
elif chip == 'rp2040':
chip_label = 'special' # rp2040 goes into special group
else:
chip_label = chip
# Gather all (manufacturer -> targets) for this chip family
# NXP chip families (imxrt, kinetis, s32k) pool all manufacturers
# under "nxp" since all boards use NXP silicon regardless of
# which directory they live in (e.g., px4/fmu-v6xrt is imxrt).
nxp_chips = tuple(ci_config.get('nxp_chip_families', ['imxrt', 'kinetis', 's32k']))
man_targets = {}
for (c, m), targets in chip_man_buckets.items():
if c == chip:
man_key = 'nxp' if chip in nxp_chips else m
if man_key not in man_targets:
man_targets[man_key] = []
man_targets[man_key].extend(targets)
# Merge rp2040 targets into a flat list for the special group
if chip in ('special', 'rp2040'):
all_targets = []
for m in sorted(man_targets.keys()):
all_targets.extend(man_targets[m])
# These get added to the special bucket below
# We'll handle after the chip loop
continue
temp_group = []
for man in grouped_targets[arch]['manufacturers']:
if(verbose):
print(f'=:Processing: [{arch}][{man}]')
man_len = len(grouped_targets[arch]['manufacturers'][man])
if(man_len > LOWER_LIMIT and man_len < (SPLIT_LIMIT + 1)):
# Manufacturers can have their own group
if(verbose):
print(f'=:Processing chip_family: [{chip}] ({chip_label})')
# Split into large-manufacturer groups and misc groups
# For NXP-exclusive chip families, always use the nxp name
# regardless of target count (there's no other manufacturer to pool with)
force_named = chip in nxp_chips
temp_group = [] # small manufacturers pooled here
for man in sorted(man_targets.keys()):
man_len = len(man_targets[man])
if (force_named or man_len > LOWER_LIMIT) and man_len <= SPLIT_LIMIT:
group_name = f"{man}-{chip_label}"
if(verbose):
print(f'=: [{man}][{man_len}] -> {group_name}')
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(man_targets[man]),
"arch": arch,
"chip_family": chip,
"runner": runner,
"group": group_name,
"len": man_len,
})
elif man_len > SPLIT_LIMIT:
chunk_counter = 0
for chunk in chunks_merged(man_targets[man], SPLIT_LIMIT):
group_name = f"{man}-{chip_label}-{chunk_counter}"
if(verbose):
print(f'=: [{man}][{man_len}] -> {group_name} ({len(chunk)})')
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(chunk),
"arch": arch,
"chip_family": chip,
"runner": runner,
"group": group_name,
"len": len(chunk),
})
chunk_counter += 1
else:
if(verbose):
print(f'=: [{man}][{man_len}] -> misc pool')
temp_group.extend(man_targets[man])
# Emit misc groups for small manufacturers
if temp_group:
misc_chunks = chunks_merged(temp_group, SPLIT_LIMIT)
num_misc_chunks = len(misc_chunks)
chunk_counter = 0
for chunk in misc_chunks:
if num_misc_chunks == 1:
group_name = f"misc-{chip_label}"
else:
group_name = f"misc-{chip_label}-{chunk_counter}"
if(verbose):
print(f'=: [misc][{len(chunk)}] -> {group_name}')
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(chunk),
"arch": arch,
"chip_family": chip,
"runner": runner,
"group": group_name,
"len": len(chunk),
})
chunk_counter += 1
# Now handle special + rp2040 targets
SPLIT_LIMIT = CHIP_SPLIT_LIMITS.get('special', DEFAULT_SPLIT_LIMIT)
special_targets = []
for (c, m), targets in chip_man_buckets.items():
if c in ('special', 'rp2040'):
special_targets.extend(targets)
if special_targets:
chunk_counter = 0
for chunk in chunks_merged(special_targets, SPLIT_LIMIT):
if len(special_targets) <= SPLIT_LIMIT:
group_name = 'special'
else:
group_name = f'special-{chunk_counter}'
if(verbose):
print(f'=: [special][{len(chunk)}] -> {group_name}')
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(chunk),
"arch": arch,
"chip_family": "special",
"runner": runner,
"group": group_name,
"len": len(chunk),
})
chunk_counter += 1
elif arch == 'voxl2':
# VOXL2 stays as its own group
all_targets = []
for man in grouped_targets[arch]['manufacturers']:
all_targets.extend(grouped_targets[arch]['manufacturers'][man])
if all_targets:
print(f'=:Processing: [{arch}][{man}][{man_len}]==Manufacturers can have their own group')
group_name = arch + "-" + man
targets = comma_targets(grouped_targets[arch]['manufacturers'][man])
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(all_targets),
"targets": targets,
"arch": arch,
"chip_family": "native",
"runner": runner,
"group": "voxl2-0",
"len": len(all_targets),
"group": group_name,
"len": len(grouped_targets[arch]['manufacturers'][man])
})
else:
# Non-NuttX groups (base, aarch64, armhf) - keep simple grouping
SPLIT_LIMIT = CHIP_SPLIT_LIMITS.get('native', DEFAULT_SPLIT_LIMIT)
all_targets = []
for man in grouped_targets[arch]['manufacturers']:
all_targets.extend(grouped_targets[arch]['manufacturers'][man])
if all_targets:
elif(man_len >= (SPLIT_LIMIT + 1)):
# Split big man groups into subgroups
# example: Pixhawk
if(verbose):
print(f'=:Processing: [{arch}][{man}][{man_len}]==Manufacturers has multiple own groups')
chunk_limit = SPLIT_LIMIT
chunk_counter = 0
for chunk in chunks_merged(all_targets, SPLIT_LIMIT):
if len(all_targets) <= SPLIT_LIMIT:
group_name = f"{arch}-0"
else:
group_name = f"{arch}-{chunk_counter}"
for chunk in chunks(grouped_targets[arch]['manufacturers'][man], chunk_limit):
group_name = arch + "-" + man + "-" + str(chunk_counter)
targets = comma_targets(chunk)
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": comma_targets(chunk),
"targets": targets,
"arch": arch,
"chip_family": "native",
"runner": runner,
"group": group_name,
"len": len(chunk),
})
chunk_counter += 1
else:
if(verbose):
print(f'=:Processing: [{arch}][{man}][{man_len}]==Manufacturers too small group with others')
temp_group.extend(grouped_targets[arch]['manufacturers'][man])
# Add cache_size to each group based on chip family
for g in final_groups:
g['cache_size'] = CHIP_CACHE_SIZES.get(g['chip_family'], DEFAULT_CACHE_SIZE)
temp_len = len(temp_group)
chunk_counter = 0
if(temp_len > 0 and temp_len < (SPLIT_LIMIT + 1)):
if(verbose):
print(f'=:Processing: [{arch}][orphan][{temp_len}]==Leftover arch can have their own group')
group_name = arch + "-" + str(chunk_counter)
targets = comma_targets(temp_group)
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": targets,
"arch": arch,
"runner": runner,
"group": group_name,
"len": temp_len
})
elif(temp_len >= (SPLIT_LIMIT + 1)):
# Split big man groups into subgroups
# example: Pixhawk
if(verbose):
print(f'=:Processing: [{arch}][orphan][{temp_len}]==Leftover arch can has multpile group')
chunk_limit = SPLIT_LIMIT
chunk_counter = 0
for chunk in chunks(temp_group, chunk_limit):
group_name = arch + "-" + str(chunk_counter)
targets = comma_targets(chunk)
final_groups.append({
"container": grouped_targets[arch]['container'],
"targets": targets,
"arch": arch,
"runner": runner,
"group": group_name,
"len": len(chunk),
})
chunk_counter += 1
if(verbose):
import pprint
print("================")
@@ -596,58 +375,6 @@ if (args.group):
print("= JSON output =")
print("===============")
if args.seeders:
# Generate one seeder entry per chip family present in the groups.
# Each seeder builds a representative target to warm the ccache for
# all groups sharing that chip family.
seeder_targets = ci_config.get('seeders', {})
seeder_containers = {
'native': default_container,
}
# Determine which chip families actually have groups
active_families = set()
for g in final_groups:
cf = g['chip_family']
active_families.add(cf)
# voxl2 gets its own seeder with a different container
if g['group'].startswith('voxl2'):
active_families.add('voxl2')
seeders = []
for cf in sorted(active_families):
if cf == 'special':
continue # special group seeds from stm32h7
if cf == 'voxl2':
seeders.append({
'chip_family': 'voxl2',
'target': 'modalai_voxl2_default',
'container': voxl2_container,
'runner': 'x64',
})
elif cf == 'native':
# One seeder per runner arch that has native groups (exclude voxl2
# which has its own seeder with a different container)
native_runners = set()
for g in final_groups:
if g['chip_family'] == 'native' and not g['group'].startswith('voxl2'):
native_runners.add(g['runner'])
for r in sorted(native_runners):
seeders.append({
'chip_family': 'native',
'target': seeder_targets['native'],
'container': default_container,
'runner': r,
})
else:
seeders.append({
'chip_family': cf,
'target': seeder_targets.get(cf, seeder_targets['stm32h7']),
'container': seeder_containers.get(cf, default_container),
'runner': 'x64',
})
print(json.dumps({ "include": seeders }, **extra_args))
else:
print(json.dumps({ "include": final_groups }, **extra_args))
print(json.dumps({ "include": final_groups }, **extra_args))
else:
print(json.dumps(github_action_config, **extra_args))
-607
View File
@@ -1,607 +0,0 @@
#!/usr/bin/env python3
"""Generate SPDX 2.3 JSON SBOM for a PX4 firmware build.
Produces one SBOM per board target containing:
- PX4 firmware as the primary package
- Git submodules as CONTAINS dependencies
- Python build requirements as BUILD_DEPENDENCY_OF packages
- Board-specific modules as CONTAINS packages
Requires PyYAML (pyyaml) for loading license overrides.
"""
import argparse
import configparser
import json
import re
import subprocess
import uuid
from datetime import datetime, timezone
from pathlib import Path
import yaml
# Ordered most-specific first: all keywords must appear for a match.
LICENSE_PATTERNS = [
# Copyleft licenses first (more specific keywords prevent false matches)
("GPL-3.0-only", ["GNU GENERAL PUBLIC LICENSE", "Version 3"]),
("GPL-2.0-only", ["GNU GENERAL PUBLIC LICENSE", "Version 2"]),
("LGPL-3.0-only", ["GNU LESSER GENERAL PUBLIC LICENSE", "Version 3"]),
("LGPL-2.1-only", ["GNU Lesser General Public License", "Version 2.1"]),
("AGPL-3.0-only", ["GNU AFFERO GENERAL PUBLIC LICENSE", "Version 3"]),
# Permissive licenses
("Apache-2.0", ["Apache License", "Version 2.0"]),
("MIT", ["Permission is hereby granted"]),
("BSD-3-Clause", ["Redistribution and use", "Neither the name"]),
("BSD-2-Clause", ["Redistribution and use", "THIS SOFTWARE IS PROVIDED"]),
("ISC", ["Permission to use, copy, modify, and/or distribute"]),
("EPL-2.0", ["Eclipse Public License", "2.0"]),
("Unlicense", ["The Unlicense", "unlicense.org"]),
]
COPYLEFT_LICENSES = {
"GPL-2.0-only", "GPL-3.0-only",
"LGPL-2.1-only", "LGPL-3.0-only",
"AGPL-3.0-only",
}
def load_license_overrides(source_dir):
"""Load license overrides and comments from YAML config file.
Returns (overrides, comments) dicts mapping submodule path to values.
Falls back to empty dicts if the file is missing.
"""
yaml_path = source_dir / "Tools" / "ci" / "license-overrides.yaml"
if not yaml_path.exists():
return {}, {}
with open(yaml_path) as f:
data = yaml.safe_load(f)
overrides = {}
comments = {}
for path, entry in (data.get("overrides") or {}).items():
overrides[path] = entry["license"]
if "comment" in entry:
comments[path] = entry["comment"]
return overrides, comments
LICENSE_FILENAMES = ["LICENSE", "LICENSE.md", "LICENSE.txt", "LICENCE", "LICENCE.md", "COPYING", "COPYING.md"]
def detect_license(submodule_dir):
"""Auto-detect SPDX license ID from LICENSE/COPYING file in a directory.
Reads the first 100 lines of the first license file found and matches
keywords against LICENSE_PATTERNS. Returns 'NOASSERTION' if no file
is found or no pattern matches.
"""
for fname in LICENSE_FILENAMES:
license_file = submodule_dir / fname
if license_file.is_file():
try:
lines = license_file.read_text(errors="replace").splitlines()[:100]
text = "\n".join(lines)
except OSError:
continue
text_upper = text.upper()
for spdx_id_val, keywords in LICENSE_PATTERNS:
if all(kw.upper() in text_upper for kw in keywords):
return spdx_id_val
return "NOASSERTION"
return "NOASSERTION"
def get_submodule_license(source_dir, sub_path, license_overrides):
"""Return the SPDX license for a submodule: override > auto-detect."""
if sub_path in license_overrides:
return license_overrides[sub_path]
return detect_license(source_dir / sub_path)
def spdx_id(name: str) -> str:
"""Convert a name to a valid SPDX identifier (letters, digits, dots, hyphens)."""
return re.sub(r"[^a-zA-Z0-9.\-]", "-", name)
def parse_gitmodules(source_dir):
"""Parse .gitmodules and return list of {name, path, url}."""
gitmodules_path = source_dir / ".gitmodules"
if not gitmodules_path.exists():
return []
config = configparser.ConfigParser()
config.read(str(gitmodules_path))
submodules = []
for section in config.sections():
if section.startswith("submodule "):
name = section.split('"')[1] if '"' in section else section.split(" ", 1)[1]
path = config.get(section, "path", fallback="")
url = config.get(section, "url", fallback="")
submodules.append({"name": name, "path": path, "url": url})
return submodules
def get_submodule_commits(source_dir):
"""Get commit hashes for all submodules via git ls-tree -r (works without init)."""
try:
result = subprocess.run(
["git", "ls-tree", "-r", "HEAD"],
cwd=str(source_dir),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=True,
)
except (subprocess.CalledProcessError, FileNotFoundError):
return {}
commits = {}
for line in result.stdout.splitlines():
parts = line.split()
if len(parts) >= 4 and parts[1] == "commit":
commits[parts[3]] = parts[2]
return commits
def get_git_info(source_dir: Path) -> dict:
"""Get PX4 git version and hash."""
info = {"version": "unknown", "hash": "unknown"}
try:
result = subprocess.run(
["git", "describe", "--always", "--tags", "--dirty"],
cwd=str(source_dir),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=True,
)
info["version"] = result.stdout.strip()
except (subprocess.CalledProcessError, FileNotFoundError):
pass
try:
result = subprocess.run(
["git", "rev-parse", "HEAD"],
cwd=str(source_dir),
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=True,
)
info["hash"] = result.stdout.strip()
except (subprocess.CalledProcessError, FileNotFoundError):
pass
return info
def parse_requirements(requirements_path):
"""Parse pip requirements.txt into list of {name, version_spec}."""
if not requirements_path.exists():
return []
deps = []
for line in requirements_path.read_text().splitlines():
line = line.strip()
if not line or line.startswith("#") or line.startswith("-"):
continue
# Split on version specifiers
match = re.match(r"^([a-zA-Z0-9_\-]+)(.*)?$", line)
if match:
deps.append({
"name": match.group(1),
"version_spec": match.group(2).strip() if match.group(2) else "",
})
return deps
def read_module_list(modules_file, source_dir):
"""Read board-specific module list from file.
Paths may be absolute; they are converted to relative paths under src/.
Duplicates are removed while preserving order.
"""
if not modules_file or not modules_file.exists():
return []
seen = set()
modules = []
source_str = str(source_dir.resolve()) + "/"
for line in modules_file.read_text().splitlines():
path = line.strip()
if not path or path.startswith("#"):
continue
# Convert absolute path to relative
if path.startswith(source_str):
path = path[len(source_str):]
if path not in seen:
seen.add(path)
modules.append(path)
return modules
def make_purl(pkg_type: str, namespace: str, name: str, version: str = "") -> str:
"""Construct a Package URL (purl)."""
purl = f"pkg:{pkg_type}/{namespace}/{name}"
if version:
purl += f"@{version}"
return purl
def extract_git_host_org_repo(url):
"""Extract host type, org, and repo from a git URL.
Returns (host, org, repo) where host is 'github', 'gitlab', or ''.
"""
match = re.search(r"github\.com[:/]([^/]+)/([^/]+?)(?:\.git)?$", url)
if match:
return "github", match.group(1), match.group(2)
match = re.search(r"gitlab\.com[:/](.+?)/([^/]+?)(?:\.git)?$", url)
if match:
return "gitlab", match.group(1), match.group(2)
return "", "", ""
def generate_sbom(source_dir, board, modules_file, compiler, platform=""):
"""Generate a complete SPDX 2.3 JSON document."""
license_overrides, license_comments = load_license_overrides(source_dir)
git_info = get_git_info(source_dir)
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
# Deterministic namespace using UUID5 from git hash + board
ns_seed = f"{git_info['hash']}:{board}"
doc_namespace = f"https://spdx.org/spdxdocs/{board}-{uuid.uuid5(uuid.NAMESPACE_URL, ns_seed)}"
doc = {
"spdxVersion": "SPDX-2.3",
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": f"PX4 Firmware SBOM for {board}",
"documentNamespace": doc_namespace,
"creationInfo": {
"created": timestamp,
"creators": [
"Tool: px4-generate-sbom",
"Organization: Dronecode Foundation",
],
"licenseListVersion": "3.22",
},
"packages": [],
"relationships": [],
}
# Primary package: PX4 firmware
primary_spdx_id = f"SPDXRef-PX4-{spdx_id(board)}"
doc["packages"].append({
"SPDXID": primary_spdx_id,
"name": board,
"versionInfo": git_info["version"],
"packageFileName": f"{board}.px4",
"supplier": "Organization: Dronecode Foundation",
"downloadLocation": "https://github.com/PX4/PX4-Autopilot",
"filesAnalyzed": False,
"primaryPackagePurpose": "FIRMWARE",
"licenseConcluded": "BSD-3-Clause",
"licenseDeclared": "BSD-3-Clause",
"copyrightText": "Copyright (c) PX4 Development Team",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": make_purl(
"github", "PX4", "PX4-Autopilot", git_info["version"]
),
}
],
})
doc["relationships"].append({
"spdxElementId": "SPDXRef-DOCUMENT",
"relationshipType": "DESCRIBES",
"relatedSpdxElement": primary_spdx_id,
})
# Git submodules (filtered to those relevant to this board's modules)
submodules = parse_gitmodules(source_dir)
submodule_commits = get_submodule_commits(source_dir)
modules = read_module_list(modules_file, source_dir)
def submodule_is_relevant(sub_path):
"""A submodule is relevant if any board module path overlaps with it."""
# NuttX platform submodules are only relevant for NuttX builds
if sub_path.startswith("platforms/nuttx/"):
return platform in ("nuttx", "")
if not modules:
return True # no module list means include all
# Other platform submodules are always relevant
if sub_path.startswith("platforms/"):
return True
for mod in modules:
# Module is under this submodule, or submodule is under a module
if mod.startswith(sub_path + "/") or sub_path.startswith(mod + "/"):
return True
return False
for sub in submodules:
if not submodule_is_relevant(sub["path"]):
continue
sub_path = sub["path"]
sub_path_id = sub_path.replace("/", "-")
sub_spdx_id = f"SPDXRef-Submodule-{spdx_id(sub_path_id)}"
commit = submodule_commits.get(sub_path, "unknown")
license_id = get_submodule_license(source_dir, sub_path, license_overrides)
host, org, repo = extract_git_host_org_repo(sub["url"])
download = sub["url"] if sub["url"] else "NOASSERTION"
# Use repo name from URL for human-readable name, fall back to last path component
display_name = repo if repo else sub_path.rsplit("/", 1)[-1]
pkg = {
"SPDXID": sub_spdx_id,
"name": display_name,
"versionInfo": commit,
"supplier": f"Organization: {org}" if org else "NOASSERTION",
"downloadLocation": download,
"filesAnalyzed": False,
"licenseConcluded": license_id,
"licenseDeclared": license_id,
"copyrightText": "NOASSERTION",
}
comment = license_comments.get(sub_path)
if comment:
pkg["licenseComments"] = comment
if host and org and repo:
pkg["externalRefs"] = [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": make_purl(host, org, repo, commit),
}
]
doc["packages"].append(pkg)
doc["relationships"].append({
"spdxElementId": primary_spdx_id,
"relationshipType": "CONTAINS",
"relatedSpdxElement": sub_spdx_id,
})
# Python build dependencies
requirements_path = source_dir / "Tools" / "setup" / "requirements.txt"
py_deps = parse_requirements(requirements_path)
for dep in py_deps:
dep_name = dep["name"]
dep_spdx_id = f"SPDXRef-PyDep-{spdx_id(dep_name)}"
version_str = dep["version_spec"] if dep["version_spec"] else "NOASSERTION"
doc["packages"].append({
"SPDXID": dep_spdx_id,
"name": dep_name,
"versionInfo": version_str,
"supplier": "NOASSERTION",
"downloadLocation": f"https://pypi.org/project/{dep_name}/",
"filesAnalyzed": False,
"primaryPackagePurpose": "APPLICATION",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
"externalRefs": [
{
"referenceCategory": "PACKAGE-MANAGER",
"referenceType": "purl",
"referenceLocator": f"pkg:pypi/{dep_name}",
}
],
})
doc["relationships"].append({
"spdxElementId": dep_spdx_id,
"relationshipType": "BUILD_DEPENDENCY_OF",
"relatedSpdxElement": primary_spdx_id,
})
# Board-specific modules (already read above for submodule filtering)
for mod in modules:
mod_path_id = mod.replace("/", "-")
mod_spdx_id = f"SPDXRef-Module-{spdx_id(mod_path_id)}"
# Derive short name: strip leading src/ for readability
display_name = mod
if display_name.startswith("src/"):
display_name = display_name[4:]
doc["packages"].append({
"SPDXID": mod_spdx_id,
"name": display_name,
"versionInfo": git_info["version"],
"supplier": "Organization: Dronecode Foundation",
"downloadLocation": "https://github.com/PX4/PX4-Autopilot",
"filesAnalyzed": False,
"licenseConcluded": "BSD-3-Clause",
"licenseDeclared": "BSD-3-Clause",
"copyrightText": "NOASSERTION",
})
doc["relationships"].append({
"spdxElementId": primary_spdx_id,
"relationshipType": "CONTAINS",
"relatedSpdxElement": mod_spdx_id,
})
# Compiler as a build tool
if compiler:
compiler_spdx_id = f"SPDXRef-Compiler-{spdx_id(compiler)}"
doc["packages"].append({
"SPDXID": compiler_spdx_id,
"name": compiler,
"versionInfo": "NOASSERTION",
"supplier": "NOASSERTION",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": False,
"primaryPackagePurpose": "APPLICATION",
"licenseConcluded": "NOASSERTION",
"licenseDeclared": "NOASSERTION",
"copyrightText": "NOASSERTION",
})
doc["relationships"].append({
"spdxElementId": compiler_spdx_id,
"relationshipType": "BUILD_TOOL_OF",
"relatedSpdxElement": primary_spdx_id,
})
return doc
def verify_licenses(source_dir):
"""Verify license detection for all submodules. Returns exit code."""
license_overrides, _ = load_license_overrides(source_dir)
submodules = parse_gitmodules(source_dir)
if not submodules:
print("No submodules found in .gitmodules")
return 1
has_noassertion = False
print(f"{'Submodule Path':<65} {'Detected':<16} {'Override':<16} {'Final'}")
print("-" * 115)
for sub in submodules:
sub_path = sub["path"]
sub_dir = source_dir / sub_path
checked_out = sub_dir.is_dir() and any(sub_dir.iterdir())
has_explicit_override = sub_path in license_overrides
if not checked_out:
detected = "(not checked out)"
override = license_overrides.get(sub_path, "")
final = override if override else "NOASSERTION"
else:
detected = detect_license(sub_dir)
override = license_overrides.get(sub_path, "")
final = override if override else detected
if final == "NOASSERTION" and has_explicit_override:
# Explicitly acknowledged in overrides file — not a failure
marker = " (acknowledged)"
elif final == "NOASSERTION" and checked_out:
has_noassertion = True
marker = " <-- UNRESOLVED"
elif final == "NOASSERTION" and not checked_out:
marker = " (skipped)"
else:
marker = ""
print(f"{sub_path:<65} {str(detected):<16} {str(override) if override else '':<16} {final}{marker}")
# Copyleft warning (informational, not a failure)
copyleft_found = []
for sub in submodules:
sub_path = sub["path"]
sub_dir = source_dir / sub_path
checked_out = sub_dir.is_dir() and any(sub_dir.iterdir())
override = license_overrides.get(sub_path, "")
if checked_out:
final_lic = override if override else detect_license(sub_dir)
else:
final_lic = override if override else "NOASSERTION"
for cl in COPYLEFT_LICENSES:
if cl in final_lic:
copyleft_found.append((sub_path, final_lic))
break
print()
if copyleft_found:
print("Copyleft licenses detected (informational):")
for path, lic in copyleft_found:
print(f" {path}: {lic}")
print()
if has_noassertion:
print("FAIL: Some submodules have unresolved licenses. "
"Add an entry to Tools/ci/license-overrides.yaml or check the LICENSE file.")
return 1
print("OK: All submodules have a resolved license.")
return 0
def main():
parser = argparse.ArgumentParser(
description="Generate SPDX 2.3 JSON SBOM for PX4 firmware"
)
parser.add_argument(
"--source-dir",
type=Path,
default=Path.cwd(),
help="PX4 source directory (default: cwd)",
)
parser.add_argument(
"--verify-licenses",
action="store_true",
help="Verify license detection for all submodules and exit",
)
parser.add_argument(
"--board",
default=None,
help="Board target name (e.g. px4_fmu-v5x_default)",
)
parser.add_argument(
"--modules-file",
type=Path,
default=None,
help="Path to config_module_list.txt",
)
parser.add_argument(
"--compiler",
default="",
help="Compiler identifier (e.g. arm-none-eabi-gcc)",
)
parser.add_argument(
"--platform",
default="",
help="PX4 platform (nuttx, posix, qurt). Filters platform-specific submodules.",
)
parser.add_argument(
"--output",
type=Path,
default=None,
help="Output SBOM file path",
)
args = parser.parse_args()
if args.verify_licenses:
raise SystemExit(verify_licenses(args.source_dir))
if not args.board:
parser.error("--board is required when not using --verify-licenses")
if not args.output:
parser.error("--output is required when not using --verify-licenses")
sbom = generate_sbom(
source_dir=args.source_dir,
board=args.board,
modules_file=args.modules_file,
compiler=args.compiler,
platform=args.platform,
)
args.output.parent.mkdir(parents=True, exist_ok=True)
with open(args.output, "w") as f:
json.dump(sbom, f, indent=2)
f.write("\n")
pkg_count = len(sbom["packages"])
print(f"SBOM generated: {args.output} ({pkg_count} packages)")
if __name__ == "__main__":
main()
-163
View File
@@ -1,163 +0,0 @@
#!/usr/bin/env python3
"""Inspect a PX4 SPDX SBOM file.
Usage:
inspect_sbom.py <sbom.spdx.json> # full summary
inspect_sbom.py <sbom.spdx.json> search <term> # search packages by name
inspect_sbom.py <sbom.spdx.json> ntia # NTIA minimum elements check
inspect_sbom.py <sbom.spdx.json> licenses # license summary
inspect_sbom.py <sbom.spdx.json> list <type> # list packages (Submodule|PyDep|Module|all)
"""
import json
import sys
from collections import Counter
from pathlib import Path
def load(path):
return json.loads(Path(path).read_text())
def pkg_type(pkg):
spdx_id = pkg["SPDXID"]
for prefix in ("Submodule", "PyDep", "Module", "Compiler", "PX4"):
if f"-{prefix}-" in spdx_id or spdx_id.startswith(f"SPDXRef-{prefix}"):
return prefix
return "Other"
def summary(doc):
print(f"spdxVersion: {doc['spdxVersion']}")
print(f"name: {doc['name']}")
print(f"namespace: {doc['documentNamespace']}")
print(f"created: {doc['creationInfo']['created']}")
print(f"creators: {', '.join(doc['creationInfo']['creators'])}")
print()
types = Counter(pkg_type(p) for p in doc["packages"])
print(f"Packages: {len(doc['packages'])}")
for t, c in types.most_common():
print(f" {t}: {c}")
print()
rc = Counter(r["relationshipType"] for r in doc["relationships"])
print(f"Relationships: {len(doc['relationships'])}")
for t, n in rc.most_common():
print(f" {t}: {n}")
print()
primary = doc["packages"][0]
print(f"Primary package:")
print(f" name: {primary['name']}")
print(f" version: {primary['versionInfo']}")
print(f" purpose: {primary.get('primaryPackagePurpose', 'N/A')}")
print(f" license: {primary['licenseDeclared']}")
print()
noassert = [
p["name"]
for p in doc["packages"]
if pkg_type(p) == "Submodule" and p["licenseDeclared"] == "NOASSERTION"
]
if noassert:
print(f"WARNING: {len(noassert)} submodules with NOASSERTION license:")
for n in noassert:
print(f" - {n}")
else:
print("All submodule licenses mapped")
print(f"\nFile size: {Path(sys.argv[1]).stat().st_size // 1024}KB")
def search(doc, term):
term = term.lower()
found = [p for p in doc["packages"] if term in p["name"].lower()]
if not found:
print(f"No packages matching '{term}'")
return
print(f"Found {len(found)} packages matching '{term}':\n")
for p in found:
print(json.dumps(p, indent=2))
print()
def ntia_check(doc):
required = ["SPDXID", "name", "versionInfo", "supplier", "downloadLocation"]
missing = []
for p in doc["packages"]:
for f in required:
if f not in p or p[f] in ("", None):
missing.append((p["name"], f))
if missing:
print(f"FAIL: {len(missing)} missing fields:")
for name, field in missing:
print(f" {name}: missing {field}")
else:
print(f"PASS: All {len(doc['packages'])} packages have required fields")
print(f"\nCreators: {doc['creationInfo']['creators']}")
print(f"Timestamp: {doc['creationInfo']['created']}")
rels = [r for r in doc["relationships"] if r["relationshipType"] == "DESCRIBES"]
print(f"DESCRIBES relationships: {len(rels)}")
return len(missing) == 0
def licenses(doc):
by_license = {}
for p in doc["packages"]:
lic = p.get("licenseDeclared", "NOASSERTION")
by_license.setdefault(lic, []).append(p["name"])
for lic in sorted(by_license.keys()):
names = by_license[lic]
print(f"\n{lic} ({len(names)}):")
for n in sorted(names):
print(f" {n}")
def list_packages(doc, filter_type):
filter_type = filter_type.lower()
for p in sorted(doc["packages"], key=lambda x: x["name"]):
t = pkg_type(p)
if filter_type != "all" and t.lower() != filter_type:
continue
lic = p.get("licenseDeclared", "?")
ver = p["versionInfo"][:20] if len(p["versionInfo"]) > 20 else p["versionInfo"]
print(f" {t:10s} {p['name']:50s} {ver:20s} {lic}")
def main():
if len(sys.argv) < 2:
print(__doc__)
sys.exit(1)
doc = load(sys.argv[1])
cmd = sys.argv[2] if len(sys.argv) > 2 else "summary"
if cmd == "summary":
summary(doc)
elif cmd == "search":
if len(sys.argv) < 4:
print("Usage: inspect_sbom.py <file> search <term>")
sys.exit(1)
search(doc, sys.argv[3])
elif cmd == "ntia":
if not ntia_check(doc):
sys.exit(1)
elif cmd == "licenses":
licenses(doc)
elif cmd == "list":
filter_type = sys.argv[3] if len(sys.argv) > 3 else "all"
list_packages(doc, filter_type)
else:
print(f"Unknown command: {cmd}")
print(__doc__)
sys.exit(1)
if __name__ == "__main__":
main()
-62
View File
@@ -1,62 +0,0 @@
# SPDX license overrides for submodules where auto-detection fails or is wrong.
# Each entry maps a submodule path to its SPDX license identifier and an
# optional comment explaining why the override exists.
#
# Run `python3 Tools/ci/generate_sbom.py --verify-licenses` to validate.
overrides:
src/modules/mavlink/mavlink:
license: "LGPL-3.0-only AND MIT"
comment: "Generator is LGPL-3.0; PX4 ships only MIT-licensed generated headers."
Tools/simulation/gazebo-classic/sitl_gazebo-classic:
license: "BSD-3-Clause"
comment: >-
PX4 org project. No LICENSE file in repo; source files carry
BSD-3-Clause headers consistent with the PX4 project license.
src/lib/cdrstream/cyclonedds:
license: "EPL-2.0 OR BSD-3-Clause"
comment: >-
Dual-licensed. PX4 elects BSD-3-Clause.
No board currently enables CONFIG_LIB_CDRSTREAM.
src/lib/cdrstream/rosidl:
license: "Apache-2.0"
src/lib/crypto/monocypher:
license: "BSD-2-Clause OR CC0-1.0"
comment: >-
Dual-licensed. LICENCE.md offers BSD-2-Clause with CC0-1.0 as
public domain fallback.
src/lib/crypto/libtomcrypt:
license: "Unlicense"
comment: "Public domain dedication. Functionally equivalent to Unlicense."
src/lib/crypto/libtommath:
license: "Unlicense"
comment: "Public domain dedication. Functionally equivalent to Unlicense."
platforms/nuttx/NuttX/nuttx:
license: "Apache-2.0"
comment: >-
Composite LICENSE (6652 lines) includes BSD/MIT/ISC sub-components.
Primary license is Apache-2.0. NOTICE file contains FAT LFN patent warnings.
platforms/nuttx/NuttX/apps:
license: "Apache-2.0"
boards/modalai/voxl2/libfc-sensor-api:
license: "NOASSERTION"
comment: >-
No LICENSE file in repo. README describes it as public interface
for proprietary sensor library.
boards/modalai/voxl2/src/lib/mpa/libmodal-json:
license: "LGPL-3.0-only"
comment: "LGPL-3.0 weak copyleft. Used via header includes in VOXL2 mpa library."
boards/modalai/voxl2/src/lib/mpa/libmodal-pipe:
license: "LGPL-3.0-only"
comment: "LGPL-3.0 weak copyleft. Used via header includes in VOXL2 mpa library."
+1 -6
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
mkdir artifacts
cp **/**/*.px4 artifacts/ 2>/dev/null || true
@@ -29,11 +29,6 @@ for build_dir_path in build/*/ ; do
# Events
mkdir -p artifacts/$build_dir/events/
cp $build_dir_path/events/all_events.json.xz artifacts/$build_dir/events/ 2>/dev/null || true
# Also copy to top level: firmware advertises the metadata URI without the events/ subdirectory
# (see src/lib/component_information/CMakeLists.txt comp_metadata_events_uri_board)
cp $build_dir_path/events/all_events.json.xz artifacts/$build_dir/ 2>/dev/null || true
# SBOM
cp $build_dir_path/*.sbom.spdx.json artifacts/$build_dir/ 2>/dev/null || true
ls -la artifacts/$build_dir
echo "----------"
done
-288
View File
@@ -1,288 +0,0 @@
#!/usr/bin/env python3
"""
PR comment poster for analysis workflows.
This script is invoked from the `PR Comment Poster` workflow which runs on
`workflow_run` in the base repository context. It consumes a `pr-comment`
artifact produced by an upstream analysis job (clang-tidy, flash_analysis,
etc.) and posts or updates a sticky PR comment via the GitHub REST API.
Artifact contract (directory passed on the command line):
manifest.json
{
"pr_number": 12345, (required, int > 0)
"marker": "<!-- pr-comment-poster:flash-analysis -->", (required, printable ASCII)
"mode": "upsert" (optional, default "upsert")
}
body.md
Markdown comment body, posted verbatim. Must be non-empty and
<= 60000 bytes (GitHub's hard limit is 65535, we cap under).
Security: this script is run in a write-token context from a workflow that
MUST NOT check out PR code. Both manifest.json and body.md are treated as
opaque data. The marker is validated to printable ASCII only before use.
Subcommands:
validate <dir> Validate that <dir> contains a conforming manifest + body.
post <dir> Validate, then upsert a sticky comment on the target PR.
Requires env GITHUB_TOKEN and GITHUB_REPOSITORY.
Python stdlib only. No third-party dependencies.
"""
import argparse
import json
import os
import sys
import _github_helpers
from _github_helpers import fail as _fail
# GitHub hard limit is 65535 bytes. Cap well under to leave headroom for
# the appended marker line and any future wrapping.
MAX_BODY_BYTES = 60000
# Marker length bounds. 1..200 is plenty for an HTML comment tag such as
# "<!-- pr-comment-poster:flash-analysis -->".
MARKER_MIN_LEN = 1
MARKER_MAX_LEN = 200
ACCEPTED_MODES = ('upsert',)
USER_AGENT = 'px4-pr-comment-poster'
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
def _is_printable_ascii(s):
# Space (0x20) through tilde (0x7E) inclusive.
return all(0x20 <= ord(ch) <= 0x7E for ch in s)
def validate_marker(marker):
"""Validate the marker string.
The marker is printable ASCII only and bounded in length. The original
shell implementation also rejected quotes, backticks, and backslashes
because the value flowed through jq and shell contexts. Now that Python
owns the handling (the value is only ever used as a substring match in
comment bodies and as a literal string in JSON request payloads that
urllib serialises for us) those characters are safe. We keep the
printable-ASCII and length rules as a belt-and-braces sanity check.
"""
if not isinstance(marker, str):
_fail('marker must be a string')
n = len(marker)
if n < MARKER_MIN_LEN or n > MARKER_MAX_LEN:
_fail('marker length out of range ({}..{}): {}'.format(
MARKER_MIN_LEN, MARKER_MAX_LEN, n))
if not _is_printable_ascii(marker):
_fail('marker contains non-printable or non-ASCII character')
def validate_manifest(directory):
"""Validate <directory>/manifest.json and <directory>/body.md.
Returns a dict with keys: pr_number (int), marker (str), mode (str),
body (str, verbatim contents of body.md).
"""
manifest_path = os.path.join(directory, 'manifest.json')
body_path = os.path.join(directory, 'body.md')
if not os.path.isfile(manifest_path):
_fail('manifest.json missing at {}'.format(manifest_path))
if not os.path.isfile(body_path):
_fail('body.md missing at {}'.format(body_path))
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, json.JSONDecodeError) as e:
_fail('manifest.json is not valid JSON: {}'.format(e))
if not isinstance(manifest, dict):
_fail('manifest.json must be a JSON object')
pr_number = manifest.get('pr_number')
# bool is a subclass of int in Python, so isinstance(True, int) is True.
# Reject bools explicitly so "true"/"false" in the manifest doesn't silently
# validate as 1/0 and then either fail upstream or poke the wrong PR.
if not isinstance(pr_number, int) or isinstance(pr_number, bool):
_fail('pr_number must be an integer')
if pr_number <= 0:
_fail('pr_number must be > 0 (got {})'.format(pr_number))
marker = manifest.get('marker')
validate_marker(marker)
mode = manifest.get('mode', 'upsert')
if mode not in ACCEPTED_MODES:
_fail('unsupported mode {!r} (accepted: {})'.format(
mode, ', '.join(ACCEPTED_MODES)))
# Read as bytes first so the size check is an honest byte count (matching
# GitHub's own 65535-byte comment limit) before we pay the cost of decoding.
try:
with open(body_path, 'rb') as f:
body_bytes = f.read()
except OSError as e:
_fail('could not read body.md: {}'.format(e))
if len(body_bytes) == 0:
_fail('body.md is empty')
if len(body_bytes) > MAX_BODY_BYTES:
_fail('body.md too large: {} bytes (max {})'.format(
len(body_bytes), MAX_BODY_BYTES))
# Require UTF-8 up front so a producer that wrote a garbage encoding fails
# here rather than later inside json.dumps with a less obvious traceback.
try:
body = body_bytes.decode('utf-8')
except UnicodeDecodeError as e:
_fail('body.md is not valid UTF-8: {}'.format(e))
return {
'pr_number': pr_number,
'marker': marker,
'mode': mode,
'body': body,
}
# ---------------------------------------------------------------------------
# Comment upsert
# ---------------------------------------------------------------------------
def find_existing_comment_id(client, repo, pr_number, marker):
"""Return the id of the first PR comment whose body contains marker, or None.
PR comments are issue comments in GitHub's data model, so we hit
/issues/{n}/comments rather than /pulls/{n}/comments (the latter only
returns review comments tied to specific code lines, which is not what
we want). The match is a plain substring check against the comment body;
the marker is expected to be an HTML comment that will not accidentally
appear in user-written prose.
"""
path = 'repos/{}/issues/{}/comments'.format(repo, pr_number)
for comment in client.paginated(path):
body = comment.get('body') or ''
if marker in body:
return comment.get('id')
return None
def build_final_body(body, marker):
"""Append the marker to body if not already present.
If the caller already embedded the marker (e.g. inside a hidden HTML
comment anywhere in their body) we leave the body alone; otherwise we
rstrip trailing newlines and append the marker on its own line after a
blank-line separator. Trailing-newline stripping keeps the output from
accumulating extra blank lines every time an existing comment is
re-rendered and re-posted.
"""
if marker in body:
return body
return '{}\n\n{}\n'.format(body.rstrip('\n'), marker)
def upsert_comment(client, repo, pr_number, marker, body):
final_body = build_final_body(body, marker)
existing_id = find_existing_comment_id(client, repo, pr_number, marker)
if existing_id is not None:
print('Updating comment {} on PR #{}'.format(existing_id, pr_number))
client.request(
'PATCH',
'repos/{}/issues/comments/{}'.format(repo, existing_id),
json_body={'body': final_body},
)
else:
print('Creating new comment on PR #{}'.format(pr_number))
client.request(
'POST',
'repos/{}/issues/{}/comments'.format(repo, pr_number),
json_body={'body': final_body},
)
# ---------------------------------------------------------------------------
# Entry points
# ---------------------------------------------------------------------------
def cmd_validate(args):
result = validate_manifest(args.directory)
print('ok: pr_number={} marker_len={} mode={} body_bytes={}'.format(
result['pr_number'],
len(result['marker']),
result['mode'],
len(result['body'].encode('utf-8')),
))
return 0
def cmd_post(args):
result = validate_manifest(args.directory)
# GITHUB_TOKEN is provided by the workflow via env; GITHUB_REPOSITORY is
# auto-set on every Actions runner. Both are required here because a local
# developer running the script directly won't have either unless they
# export them, and we want a clear error in that case.
token = os.environ.get('GITHUB_TOKEN')
if not token:
_fail('GITHUB_TOKEN is not set')
repo = os.environ.get('GITHUB_REPOSITORY')
if not repo:
_fail('GITHUB_REPOSITORY is not set (expected "owner/name")')
# Minimal shape check. If "owner/name" is malformed the subsequent API
# calls would 404 with an unhelpful URL. Fail fast here instead.
if '/' not in repo:
_fail('GITHUB_REPOSITORY must be "owner/name", got {!r}'.format(repo))
try:
client = _github_helpers.GitHubClient(token, user_agent=USER_AGENT)
upsert_comment(
client=client,
repo=repo,
pr_number=result['pr_number'],
marker=result['marker'],
body=result['body'],
)
except RuntimeError as e:
_fail(str(e))
return 0
def main(argv=None):
parser = argparse.ArgumentParser(
description='Validate and post sticky PR comments from CI artifacts.',
)
sub = parser.add_subparsers(dest='command', required=True)
p_validate = sub.add_parser(
'validate',
help='Validate manifest.json and body.md in the given directory.',
)
p_validate.add_argument('directory')
p_validate.set_defaults(func=cmd_validate)
p_post = sub.add_parser(
'post',
help='Validate, then upsert a sticky PR comment. Requires env '
'GITHUB_TOKEN and GITHUB_REPOSITORY.',
)
p_post.add_argument('directory')
p_post.set_defaults(func=cmd_post)
args = parser.parse_args(argv)
return args.func(args)
if __name__ == '__main__':
sys.exit(main())
-481
View File
@@ -1,481 +0,0 @@
#!/usr/bin/env python3
"""
PR review-comment poster for analysis workflows.
Sibling of Tools/ci/pr-comment-poster.py. Where pr-comment-poster.py posts
sticky issue-style PR comments, this script posts line-anchored review
comments on the "Files changed" tab. Use it for tools like clang-tidy that
want to flag specific lines instead of (or in addition to) a rollup
comment.
This script is invoked from the `PR Review Poster` workflow which runs on
`workflow_run` in the base repository context. It consumes a `pr-review`
artifact produced by an upstream analysis job and posts a fresh PR review
via the GitHub REST API, dismissing any stale review the same producer
left on a previous run.
Artifact contract (directory passed on the command line):
manifest.json
{
"pr_number": 12345, (required, int > 0)
"marker": "<!-- pr-review-poster:clang-tidy -->", (required, printable ASCII)
"event": "COMMENT", (required, "COMMENT" only)
"commit_sha": "0123456789abcdef0123456789abcdef01234567",(required, 40 hex chars)
"summary": "Optional review body text" (optional)
}
comments.json
JSON array of line-anchored review comment objects:
[
{"path": "src/foo.cpp", "line": 42, "side": "RIGHT",
"body": "..."},
{"path": "src/bar.hpp", "start_line": 10, "line": 15,
"side": "RIGHT", "start_side": "RIGHT", "body": "..."}
]
Note: `APPROVE` and `REQUEST_CHANGES` events are intentionally NOT
supported. Bots should never approve a pull request, and REQUEST_CHANGES
cannot be dismissed by the GITHUB_TOKEN when branch protection restricts
review dismissals, leading to undismissable spam on every push.
Security: this script is run in a write-token context from a workflow that
MUST NOT check out PR code. Both manifest.json and comments.json are
treated as opaque data. The marker is validated to printable ASCII only
before use, and only reviews authored by github-actions[bot] whose body
contains the marker can be dismissed (a fork cannot spoof either).
Subcommands:
validate <dir> Validate that <dir> contains a conforming manifest +
comments file.
post <dir> Validate, then dismiss any stale matching review and
post a new review on the target PR. Requires env
GITHUB_TOKEN and GITHUB_REPOSITORY.
Python stdlib only. No third-party dependencies.
"""
import argparse
import json
import os
import re
import sys
import time
import _github_helpers
from _github_helpers import fail as _fail
USER_AGENT = 'px4-pr-review-poster'
# Marker length bounds. 1..200 is plenty for an HTML comment tag such as
# "<!-- pr-review-poster:clang-tidy -->".
MARKER_MIN_LEN = 1
MARKER_MAX_LEN = 200
# Cap per-comment body size well under GitHub's hard limit so we leave
# headroom for the wrapping JSON envelope. Empirically GitHub allows ~65535
# bytes per review comment body; 60000 is a safe ceiling.
MAX_COMMENT_BODY_BYTES = 60000
# Cap on number of comments per single review POST. platisd uses 10. The
# value matters because GitHub's review-creation endpoint has a payload
# size limit and review comments occasionally trip an abuse-detection
# threshold when posted in very large batches. Smaller chunks also let us
# spread the work across multiple reviews so a single bad entry only
# fails its own chunk.
COMMENTS_PER_REVIEW = 10
# Sleep between successive review POSTs to stay clear of GitHub's
# secondary rate limits. platisd uses 10s; 5s is enough for our volume
# and cuts user-visible latency.
SLEEP_BETWEEN_CHUNKS_SECONDS = 5
ACCEPTED_EVENTS = ('COMMENT',)
ACCEPTED_SIDES = ('LEFT', 'RIGHT')
COMMIT_SHA_RE = re.compile(r'^[0-9a-f]{40}$')
# The login GitHub assigns to the built-in actions token. Used to filter
# the list of existing reviews so we never touch a human reviewer's review.
BOT_LOGIN = 'github-actions[bot]'
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
def _is_printable_ascii(s):
return all(0x20 <= ord(ch) <= 0x7E for ch in s)
def validate_marker(marker):
"""Validate the marker string. See pr-comment-poster.py for rationale."""
if not isinstance(marker, str):
_fail('marker must be a string')
n = len(marker)
if n < MARKER_MIN_LEN or n > MARKER_MAX_LEN:
_fail('marker length out of range ({}..{}): {}'.format(
MARKER_MIN_LEN, MARKER_MAX_LEN, n))
if not _is_printable_ascii(marker):
_fail('marker contains non-printable or non-ASCII character')
def _validate_comment_entry(idx, entry):
"""Validate a single review-comment entry. Raises via _fail on error."""
if not isinstance(entry, dict):
_fail('comments[{}]: must be an object'.format(idx))
path = entry.get('path')
if not isinstance(path, str) or not path:
_fail('comments[{}].path: required non-empty string'.format(idx))
line = entry.get('line')
if not isinstance(line, int) or isinstance(line, bool) or line <= 0:
_fail('comments[{}].line: required positive integer'.format(idx))
side = entry.get('side', 'RIGHT')
if side not in ACCEPTED_SIDES:
_fail('comments[{}].side: must be one of {} (got {!r})'.format(
idx, ', '.join(ACCEPTED_SIDES), side))
if 'start_line' in entry:
start_line = entry['start_line']
if (not isinstance(start_line, int)
or isinstance(start_line, bool)
or start_line <= 0):
_fail('comments[{}].start_line: must be positive integer'.format(idx))
if start_line >= line:
_fail('comments[{}].start_line ({}) must be < line ({})'.format(
idx, start_line, line))
start_side = entry.get('start_side', side)
if start_side not in ACCEPTED_SIDES:
_fail('comments[{}].start_side: must be one of {}'.format(
idx, ', '.join(ACCEPTED_SIDES)))
body = entry.get('body')
if not isinstance(body, str) or not body:
_fail('comments[{}].body: required non-empty string'.format(idx))
body_bytes = len(body.encode('utf-8'))
if body_bytes > MAX_COMMENT_BODY_BYTES:
_fail('comments[{}].body too large: {} bytes (max {})'.format(
idx, body_bytes, MAX_COMMENT_BODY_BYTES))
def validate_manifest(directory):
"""Validate <directory>/manifest.json and <directory>/comments.json.
Returns a dict with keys: pr_number, marker, event, commit_sha,
summary, comments (list of validated comment dicts).
"""
manifest_path = os.path.join(directory, 'manifest.json')
comments_path = os.path.join(directory, 'comments.json')
if not os.path.isfile(manifest_path):
_fail('manifest.json missing at {}'.format(manifest_path))
if not os.path.isfile(comments_path):
_fail('comments.json missing at {}'.format(comments_path))
try:
with open(manifest_path, 'r', encoding='utf-8') as f:
manifest = json.load(f)
except (OSError, json.JSONDecodeError) as e:
_fail('manifest.json is not valid JSON: {}'.format(e))
if not isinstance(manifest, dict):
_fail('manifest.json must be a JSON object')
pr_number = manifest.get('pr_number')
if not isinstance(pr_number, int) or isinstance(pr_number, bool):
_fail('pr_number must be an integer')
if pr_number <= 0:
_fail('pr_number must be > 0 (got {})'.format(pr_number))
marker = manifest.get('marker')
validate_marker(marker)
event = manifest.get('event')
if event not in ACCEPTED_EVENTS:
_fail('event must be one of {} (got {!r}). APPROVE and '
'REQUEST_CHANGES are intentionally forbidden.'.format(
', '.join(ACCEPTED_EVENTS), event))
commit_sha = manifest.get('commit_sha')
if not isinstance(commit_sha, str) or not COMMIT_SHA_RE.match(commit_sha):
_fail('commit_sha must be a 40-character lowercase hex string')
summary = manifest.get('summary', '')
if summary is None:
summary = ''
if not isinstance(summary, str):
_fail('summary must be a string if present')
try:
with open(comments_path, 'r', encoding='utf-8') as f:
comments = json.load(f)
except (OSError, json.JSONDecodeError) as e:
_fail('comments.json is not valid JSON: {}'.format(e))
if not isinstance(comments, list):
_fail('comments.json must be a JSON array')
for idx, entry in enumerate(comments):
_validate_comment_entry(idx, entry)
return {
'pr_number': pr_number,
'marker': marker,
'event': event,
'commit_sha': commit_sha,
'summary': summary,
'comments': comments,
}
# ---------------------------------------------------------------------------
# Stale-review dismissal
# ---------------------------------------------------------------------------
def find_stale_reviews(client, repo, pr_number, marker):
"""Yield (id, state) for each existing review owned by the bot AND
whose body contains the marker.
Filtering on BOTH author == github-actions[bot] AND marker-in-body is
the security invariant: a fork PR cannot impersonate the bot login,
and a fork PR also cannot inject the marker into a human reviewer's
body without API access.
"""
path = 'repos/{}/pulls/{}/reviews'.format(repo, pr_number)
for review in client.paginated(path):
user = review.get('user') or {}
if user.get('login') != BOT_LOGIN:
continue
body = review.get('body') or ''
if marker not in body:
continue
yield review.get('id'), review.get('state')
def dismiss_stale_reviews(client, repo, pr_number, marker):
"""Dismiss (or, for PENDING reviews, delete) every stale matching review.
Returns the number of reviews that could NOT be dismissed (still active).
"""
dismissal_message = 'Superseded by a newer run'
failed_dismissals = 0
for review_id, state in find_stale_reviews(client, repo, pr_number, marker):
if review_id is None:
continue
if state in ('DISMISSED', 'COMMENTED'):
# Already inert or non-blocking; nothing to do.
continue
if state == 'PENDING':
# PENDING reviews cannot be dismissed; they must be deleted.
print('Deleting pending stale review {}'.format(review_id))
try:
client.request(
'DELETE',
'repos/{}/pulls/{}/reviews/{}'.format(
repo, pr_number, review_id))
except RuntimeError as e:
failed_dismissals += 1
print('warning: failed to delete pending review {}: {}'.format(
review_id, e), file=sys.stderr)
continue
print('Dismissing stale review {} (state={})'.format(review_id, state))
try:
client.request(
'PUT',
'repos/{}/pulls/{}/reviews/{}/dismissals'.format(
repo, pr_number, review_id),
json_body={
'message': dismissal_message,
'event': 'DISMISS',
},
)
except RuntimeError as e:
failed_dismissals += 1
print('warning: failed to dismiss review {}: {}'.format(
review_id, e), file=sys.stderr)
return failed_dismissals
# ---------------------------------------------------------------------------
# Review posting
# ---------------------------------------------------------------------------
def _chunk(lst, n):
"""Yield successive n-sized slices of lst."""
for i in range(0, len(lst), n):
yield lst[i:i + n]
def _build_review_body(marker, summary, chunk_index, chunk_total):
"""Construct the review body text.
Always begins with the marker (so future runs can find and dismiss
this review). Appends a chunk index when the comment set is split
across multiple reviews, and the producer-supplied summary if any.
"""
parts = [marker]
if chunk_total > 1:
parts.append('({}/{})'.format(chunk_index + 1, chunk_total))
if summary:
parts.append('')
parts.append(summary)
return '\n'.join(parts)
def _comment_to_api(entry):
"""Project a validated comment dict to the GitHub API shape."""
api = {
'path': entry['path'],
'line': entry['line'],
'side': entry.get('side', 'RIGHT'),
'body': entry['body'],
}
if 'start_line' in entry:
api['start_line'] = entry['start_line']
api['start_side'] = entry.get('start_side', api['side'])
return api
def post_review(client, repo, pr_number, marker, event, commit_sha, summary,
comments):
"""Post one or more reviews containing the validated comments.
Comments are split into COMMENTS_PER_REVIEW-sized chunks. Each chunk
becomes its own review POST. A failed chunk logs a warning and the
loop continues to the next chunk.
"""
chunks = list(_chunk(comments, COMMENTS_PER_REVIEW))
total = len(chunks)
if total == 0:
print('No comments to post; skipping review creation.')
return
posted_any = False
for idx, chunk in enumerate(chunks):
if idx > 0:
time.sleep(SLEEP_BETWEEN_CHUNKS_SECONDS)
body = _build_review_body(marker, summary, idx, total)
payload = {
'commit_id': commit_sha,
'body': body,
'event': event,
'comments': [_comment_to_api(c) for c in chunk],
}
print('Posting review chunk {}/{} with {} comment(s)'.format(
idx + 1, total, len(chunk)))
try:
client.request(
'POST',
'repos/{}/pulls/{}/reviews'.format(repo, pr_number),
json_body=payload,
)
posted_any = True
except RuntimeError as e:
# Most common cause is HTTP 422: a comment refers to a line
# GitHub does not consider part of the diff. Skip the bad
# chunk and keep going so other findings still get posted.
print('warning: review chunk {}/{} failed: {}'.format(
idx + 1, total, e), file=sys.stderr)
if not posted_any:
# If every single chunk failed, surface that as a hard error so
# the workflow turns red and a human looks at it.
_fail('all review chunks failed to post; see warnings above')
# ---------------------------------------------------------------------------
# Entry points
# ---------------------------------------------------------------------------
def cmd_validate(args):
result = validate_manifest(args.directory)
print(('ok: pr_number={} marker_len={} event={} commit_sha={} '
'comments={} summary_len={}').format(
result['pr_number'],
len(result['marker']),
result['event'],
result['commit_sha'],
len(result['comments']),
len(result['summary']),
))
return 0
def cmd_post(args):
result = validate_manifest(args.directory)
# Empty comment lists short-circuit silently. A producer that ran but
# found nothing to flag should not generate noise on the PR.
if len(result['comments']) == 0:
print('No comments in artifact; nothing to post.')
return 0
token = os.environ.get('GITHUB_TOKEN')
if not token:
_fail('GITHUB_TOKEN is not set')
repo = os.environ.get('GITHUB_REPOSITORY')
if not repo:
_fail('GITHUB_REPOSITORY is not set (expected "owner/name")')
if '/' not in repo:
_fail('GITHUB_REPOSITORY must be "owner/name", got {!r}'.format(repo))
try:
client = _github_helpers.GitHubClient(token, user_agent=USER_AGENT)
undismissed = dismiss_stale_reviews(
client=client,
repo=repo,
pr_number=result['pr_number'],
marker=result['marker'],
)
if undismissed > 0:
print('{} prior review(s) could not be dismissed (likely '
'branch protection).'.format(undismissed))
post_review(
client=client,
repo=repo,
pr_number=result['pr_number'],
marker=result['marker'],
event=result['event'],
commit_sha=result['commit_sha'],
summary=result['summary'],
comments=result['comments'],
)
except RuntimeError as e:
_fail(str(e))
return 0
def main(argv=None):
parser = argparse.ArgumentParser(
description='Validate and post line-anchored PR review comments '
'from CI artifacts.',
)
sub = parser.add_subparsers(dest='command', required=True)
p_validate = sub.add_parser(
'validate',
help='Validate manifest.json and comments.json in the given directory.',
)
p_validate.add_argument('directory')
p_validate.set_defaults(func=cmd_validate)
p_post = sub.add_parser(
'post',
help='Validate, then dismiss any stale review and post a new one. '
'Requires env GITHUB_TOKEN and GITHUB_REPOSITORY.',
)
p_post.add_argument('directory')
p_post.set_defaults(func=cmd_post)
args = parser.parse_args(argv)
return args.func(args)
if __name__ == '__main__':
sys.exit(main())
-147
View File
@@ -1,147 +0,0 @@
#!/usr/bin/env python3
"""
Run clang-tidy incrementally on files changed in a PR.
Usage: run-clang-tidy-pr.py <base-ref>
base-ref: e.g. origin/main
Computes the set of translation units (TUs) affected by the PR diff,
then invokes Tools/run-clang-tidy.py on that subset only.
Exits 0 silently when no C++ files were changed.
"""
import argparse
import json
import os
import subprocess
import sys
EXTENSIONS_CPP = {'.cpp', '.c'}
EXTENSIONS_HDR = {'.hpp', '.h'}
# Manual exclusions from Makefile:508
EXCLUDE_EXTRA = '|'.join([
'src/systemcmds/tests',
'src/examples',
'src/modules/gyro_fft/CMSIS_5',
'src/lib/drivers/smbus',
'src/drivers/gpio',
r'src/modules/commander/failsafe/emscripten',
r'failsafe_test\.dir',
])
def repo_root():
try:
return subprocess.check_output(
['git', 'rev-parse', '--show-toplevel'], text=True).strip()
except subprocess.CalledProcessError:
print('error: not inside a git repository', file=sys.stderr)
sys.exit(1)
def changed_files(base_ref, root):
try:
out = subprocess.check_output(
['git', 'diff', '--name-only', f'{base_ref}...HEAD',
'--', '*.cpp', '*.hpp', '*.h', '*.c'],
text=True, cwd=root).strip()
return out.splitlines() if out else []
except subprocess.CalledProcessError:
print(f'error: could not diff against "{base_ref}"'
'is the ref valid and fetched?', file=sys.stderr)
sys.exit(1)
def submodule_paths(root):
# Returns [] if .gitmodules is absent or has no paths — both valid
try:
out = subprocess.check_output(
['git', 'config', '--file', '.gitmodules',
'--get-regexp', 'path'],
text=True, cwd=root).strip()
return [line.split()[1] for line in out.splitlines()]
except subprocess.CalledProcessError:
return []
def build_exclude(root):
submodules = '|'.join(submodule_paths(root))
return f'{submodules}|{EXCLUDE_EXTRA}' if submodules else EXCLUDE_EXTRA
def load_db(build_dir):
db_path = os.path.join(build_dir, 'compile_commands.json')
if not os.path.isfile(db_path):
print(f'error: {db_path} not found', file=sys.stderr)
print('Run "make px4_sitl_default-clang" first to generate '
'the compilation database', file=sys.stderr)
sys.exit(1)
try:
with open(db_path) as f:
return json.load(f)
except json.JSONDecodeError as e:
print(f'error: compile_commands.json is malformed: {e}', file=sys.stderr)
sys.exit(1)
def find_tus(changed, db, root):
db_files = {e['file'] for e in db}
result = set()
for f in changed:
abs_path = os.path.join(root, f)
ext = os.path.splitext(f)[1]
if ext in EXTENSIONS_CPP:
if abs_path in db_files:
result.add(abs_path)
elif ext in EXTENSIONS_HDR:
hdr = os.path.basename(f)
for e in db:
try:
if hdr in open(e['file']).read():
result.add(e['file'])
except OSError:
pass # file deleted in PR — skip
return sorted(result)
def main():
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('base_ref',
help='Git ref to diff against, e.g. origin/main')
args = parser.parse_args()
root = repo_root()
build_dir = os.path.join(root, 'build', 'px4_sitl_default-clang')
run_tidy = os.path.join(root, 'Tools', 'run-clang-tidy.py')
if not os.path.isfile(run_tidy):
print(f'error: {run_tidy} not found', file=sys.stderr)
sys.exit(1)
changed = changed_files(args.base_ref, root)
if not changed:
print('No C++ files changed — skipping clang-tidy')
sys.exit(0)
db = load_db(build_dir)
tus = find_tus(changed, db, root)
if not tus:
print('No matching TUs in compile_commands.json — skipping clang-tidy')
sys.exit(0)
print(f'Running clang-tidy on {len(tus)} translation unit(s)')
result = subprocess.run(
[sys.executable, run_tidy,
'-header-filter=.*\\.hpp',
'-j0',
f'-exclude={build_exclude(root)}',
'-p', build_dir] + tus
)
sys.exit(result.returncode)
if __name__ == '__main__':
main()
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
# This script runs the fuzz tests from a given binary for a certain amount of time
set -e
-42
View File
@@ -1,42 +0,0 @@
#!/bin/sh
# Rewrite the container's apt sources to point at the AWS regional Ubuntu
# mirror that is local to the runs-on instance.
#
# The default archive.ubuntu.com round-robin sometimes serves out-of-sync
# index files mid-sync, breaking apt-get update with errors like:
# File has unexpected size (25378 != 25381). Mirror sync in progress?
# The Canonical-operated EC2 mirrors are region-local and sync aggressively,
# eliminating that failure mode.
#
# This script is a no-op outside runs-on, so it is safe to call from any CI
# job (forks, self-hosted runners, local docker runs, etc.) without changing
# behavior there.
#
# Usage (from a workflow step running inside the container):
# ./Tools/ci/use_aws_apt_mirror.sh
set -e
if [ -z "$RUNS_ON_AWS_REGION" ]; then
echo "use_aws_apt_mirror: not running on runs-on (RUNS_ON_AWS_REGION unset), skipping"
exit 0
fi
MIRROR="http://${RUNS_ON_AWS_REGION}.ec2.archive.ubuntu.com/ubuntu"
echo "use_aws_apt_mirror: rewriting apt sources to ${MIRROR}"
# Noble (24.04+) uses the deb822 format at /etc/apt/sources.list.d/ubuntu.sources
if [ -f /etc/apt/sources.list.d/ubuntu.sources ]; then
sed -i \
-e "s|http://archive.ubuntu.com/ubuntu|${MIRROR}|g" \
-e "s|http://security.ubuntu.com/ubuntu|${MIRROR}|g" \
/etc/apt/sources.list.d/ubuntu.sources
fi
# Jammy (22.04) and earlier use the legacy /etc/apt/sources.list
if [ -f /etc/apt/sources.list ]; then
sed -i \
-e "s|http://archive.ubuntu.com/ubuntu|${MIRROR}|g" \
-e "s|http://security.ubuntu.com/ubuntu|${MIRROR}|g" \
/etc/apt/sources.list
fi
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#! /bin/bash
# Copy msgs and the message translation node into a ROS workspace directory
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#! /bin/bash
if [ -z ${PX4_DOCKER_REPO+x} ]; then
PX4_DOCKER_REPO="px4io/px4-dev:v1.17.0-beta1"
-84
View File
@@ -1,84 +0,0 @@
# syntax=docker/dockerfile:1
# PX4 SITL Gazebo Harmonic runtime image
# Runs PX4 SITL with Gazebo Harmonic. Supports X11 forwarding for GUI.
#
# Build:
# make px4_sitl_default && cd build/px4_sitl_default && cpack -G DEB && cd ../..
# docker build -f Tools/packaging/Dockerfile.gazebo -t px4io/px4-sitl-gazebo:v1.17.0 build/px4_sitl_default/
#
# Run (headless):
# docker run --rm -it --network host px4io/px4-sitl-gazebo:v1.17.0
#
# Run (X11 GUI):
# xhost +local:docker
# docker run --rm -it --network host \
# -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \
# --gpus all px4io/px4-sitl-gazebo:v1.17.0
FROM ubuntu:24.04 AS extract
COPY px4-gazebo_*.deb /tmp/
RUN apt-get update && apt-get install -y --no-install-recommends binutils \
&& dpkg -x /tmp/px4-gazebo_*.deb /staging \
&& strip /staging/opt/px4-gazebo/bin/px4 \
&& rm -rf /var/lib/apt/lists/*
FROM ubuntu:24.04
LABEL maintainer="PX4 Development Team"
LABEL description="PX4 SITL with Gazebo Harmonic simulation"
ENV DEBIAN_FRONTEND=noninteractive
ENV RUNS_IN_DOCKER=true
# Install Gazebo Harmonic with buildkit cache mounts for apt
# The --mount=type=cache persists /var/cache/apt and /var/lib/apt across builds
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update \
&& apt-get install -y --no-install-recommends \
bc \
ca-certificates \
gnupg \
lsb-release \
wget \
&& wget -q https://packages.osrfoundation.org/gazebo.gpg \
-O /usr/share/keyrings/pkgs-osrf-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/pkgs-osrf-archive-keyring.gpg] http://packages.osrfoundation.org/gazebo/ubuntu-stable $(lsb_release -cs) main" \
> /etc/apt/sources.list.d/gazebo-stable.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends \
gz-harmonic
# Install PX4 files from .deb
COPY --from=extract /staging/opt/px4-gazebo /opt/px4-gazebo
RUN ln -sf /opt/px4-gazebo/bin/px4-gazebo /usr/bin/px4-gazebo
# Create the DART physics engine symlink (avoids needing the -dev package)
RUN GZ_PHYSICS_DIR=$(find /usr/lib -maxdepth 3 -type d -name "engine-plugins" -path "*/gz-physics-7/*" 2>/dev/null | head -1) \
&& if [ -n "$GZ_PHYSICS_DIR" ] && [ -d "$GZ_PHYSICS_DIR" ]; then \
VERSIONED=$(ls "$GZ_PHYSICS_DIR"/libgz-physics*-dartsim-plugin.so.* 2>/dev/null | head -1) \
&& [ -n "$VERSIONED" ] \
&& ln -sf "$(basename "$VERSIONED")" "$GZ_PHYSICS_DIR/libgz-physics-dartsim-plugin.so"; \
fi
# Gazebo resource paths
ENV GZ_SIM_RESOURCE_PATH=/opt/px4-gazebo/share/gz/models:/opt/px4-gazebo/share/gz/worlds
ENV GZ_SIM_SYSTEM_PLUGIN_PATH=/opt/px4-gazebo/lib/gz/plugins
ENV GZ_SIM_SERVER_CONFIG_PATH=/opt/px4-gazebo/share/gz/server.config
ENV PX4_GZ_MODELS=/opt/px4-gazebo/share/gz/models
ENV PX4_GZ_WORLDS=/opt/px4-gazebo/share/gz/worlds
ENV PX4_SIM_MODEL=gz_x500
ENV HOME=/root
# MAVLink, MAVSDK, DDS
EXPOSE 14550/udp 14540/udp 8888/udp
# Platform-adaptive entrypoint: detects Docker Desktop (macOS/Windows) via
# host.docker.internal and configures MAVLink + DDS to target the host.
COPY px4-entrypoint.sh /opt/px4-gazebo/bin/px4-entrypoint.sh
RUN chmod +x /opt/px4-gazebo/bin/px4-entrypoint.sh
WORKDIR /root
ENTRYPOINT ["/opt/px4-gazebo/bin/px4-entrypoint.sh"]
CMD []
-49
View File
@@ -1,49 +0,0 @@
# syntax=docker/dockerfile:1
# PX4 SITL SIH runtime image
# Minimal container that runs PX4 with the SIH physics engine (no Gazebo).
#
# Build:
# make px4_sitl_sih && cd build/px4_sitl_sih && cpack -G DEB && cd ../..
# docker build -f Tools/packaging/Dockerfile.sih -t px4io/px4-sitl:v1.17.0 build/px4_sitl_sih/
#
# Run (Linux):
# docker run --rm -it --network host px4io/px4-sitl:v1.17.0
#
# Run (macOS / Windows):
# docker run --rm -it -p 14550:14550/udp -p 14540:14540/udp -p 19410:19410/udp -p 8888:8888/udp px4io/px4-sitl:v1.17.0
FROM ubuntu:24.04 AS build
COPY px4_*.deb /tmp/
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update \
&& apt-get install -y --no-install-recommends binutils \
&& dpkg -x /tmp/px4_*.deb /staging \
&& strip /staging/opt/px4/bin/px4
FROM ubuntu:24.04
LABEL maintainer="PX4 Development Team"
LABEL description="PX4 SITL with SIH physics (no simulator dependencies)"
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update && apt-get install -y --no-install-recommends bc
COPY --from=build /staging/opt/px4 /opt/px4
RUN ln -sf /opt/px4/bin/px4 /usr/bin/px4
# Platform-adaptive entrypoint: detects Docker Desktop (macOS/Windows) via
# host.docker.internal and configures MAVLink + DDS to target the host.
COPY px4-entrypoint.sh /opt/px4/bin/px4-entrypoint.sh
RUN chmod +x /opt/px4/bin/px4-entrypoint.sh
ENV PX4_SIM_MODEL=sihsim_quadx
ENV HOME=/root
# MAVLink (QGC, MAVSDK), DDS (ROS 2), jMAVSim/viewer display
EXPOSE 14550/udp 14540/udp 19410/udp 8888/udp
WORKDIR /root
ENTRYPOINT ["/opt/px4/bin/px4-entrypoint.sh"]
CMD []
-4
View File
@@ -1,4 +0,0 @@
#!/bin/sh
set -e
ln -sf /opt/px4-gazebo/bin/px4-gazebo /usr/bin/px4-gazebo
exit 0
-6
View File
@@ -1,6 +0,0 @@
#!/bin/sh
set -e
if [ "$1" = "remove" ] || [ "$1" = "purge" ]; then
rm -f /usr/bin/px4-gazebo
fi
exit 0
-35
View File
@@ -1,35 +0,0 @@
#!/bin/sh
# Docker entrypoint for PX4 SITL containers.
#
# On Docker Desktop (macOS/Windows), host.docker.internal resolves to the
# host machine. We detect this and configure MAVLink + DDS to send to the
# host IP instead of localhost (which stays inside the container VM).
#
# On Linux with --network host, host.docker.internal does not resolve and
# PX4 defaults work without modification.
set -e
# Detect install prefix (SIH uses /opt/px4, Gazebo uses /opt/px4-gazebo)
if [ -d /opt/px4-gazebo ]; then
PX4_PREFIX=/opt/px4-gazebo
else
PX4_PREFIX=/opt/px4
fi
# Resolve host.docker.internal to an IPv4 address. mavlink and uxrce_dds_client
# only parse IPv4, and on Docker Desktop for Windows the default `getent hosts`
# lookup can return an IPv6 ULA first, which both modules then reject.
DOCKER_HOST_IP=$(getent ahostsv4 host.docker.internal 2>/dev/null | awk '/STREAM/ {print $1; exit}')
if [ -n "$DOCKER_HOST_IP" ]; then
# MAVLink: replace default target (127.0.0.1) with the Docker host IP
sed -i "s/mavlink start -x -u/mavlink start -x -t $DOCKER_HOST_IP -u/g" \
"$PX4_PREFIX/etc/init.d-posix/px4-rc.mavlink"
# DDS: point uXRCE-DDS client at the host
sed -i "s|uxrce_dds_client start -t udp|uxrce_dds_client start -t udp -h $DOCKER_HOST_IP|" \
"$PX4_PREFIX/etc/init.d-posix/rcS"
fi
exec "$PX4_PREFIX/bin/px4" "$@"
-32
View File
@@ -1,32 +0,0 @@
#!/usr/bin/env bash
# px4-gazebo: Launch PX4 SITL with Gazebo from the installed .deb package
set -e
PX4_GAZEBO_DIR="$(cd "$(dirname "$(readlink -f "$0")")/.." && pwd)"
PX4_BINARY="${PX4_GAZEBO_DIR}/bin/px4"
# Set Gazebo resource paths so gz-sim finds PX4 models, worlds, and plugins.
export PX4_GZ_MODELS="${PX4_GAZEBO_DIR}/share/gz/models"
export PX4_GZ_WORLDS="${PX4_GAZEBO_DIR}/share/gz/worlds"
export PX4_GZ_PLUGINS="${PX4_GAZEBO_DIR}/lib/gz/plugins"
export PX4_GZ_SERVER_CONFIG="${PX4_GAZEBO_DIR}/share/gz/server.config"
export GZ_SIM_RESOURCE_PATH="${GZ_SIM_RESOURCE_PATH}:${PX4_GZ_MODELS}:${PX4_GZ_WORLDS}"
export GZ_SIM_SYSTEM_PLUGIN_PATH="${GZ_SIM_SYSTEM_PLUGIN_PATH}:${PX4_GZ_PLUGINS}"
export GZ_SIM_SERVER_CONFIG_PATH="${PX4_GZ_SERVER_CONFIG}"
# Gazebo's Physics system searches for "gz-physics-dartsim-plugin" which maps
# to the unversioned libgz-physics-dartsim-plugin.so. The runtime package only
# ships versioned .so files; the unversioned symlink lives in the -dev package.
# Create it if missing so Gazebo finds the DART engine without installing -dev.
GZ_PHYSICS_ENGINE_DIR=$(find /usr/lib -maxdepth 3 -type d -name "engine-plugins" -path "*/gz-physics-7/*" 2>/dev/null | head -1)
if [ -n "$GZ_PHYSICS_ENGINE_DIR" ] && [ -d "$GZ_PHYSICS_ENGINE_DIR" ]; then
UNVERSIONED="$GZ_PHYSICS_ENGINE_DIR/libgz-physics-dartsim-plugin.so"
if [ ! -e "$UNVERSIONED" ]; then
VERSIONED=$(ls "$GZ_PHYSICS_ENGINE_DIR"/libgz-physics*-dartsim-plugin.so.* 2>/dev/null | head -1)
if [ -n "$VERSIONED" ]; then
ln -sf "$(basename "$VERSIONED")" "$UNVERSIONED" 2>/dev/null || true
fi
fi
fi
exec "${PX4_BINARY}" "$@"
-4
View File
@@ -1,4 +0,0 @@
#!/bin/sh
set -e
ln -sf /opt/px4/bin/px4 /usr/bin/px4
exit 0
-6
View File
@@ -1,6 +0,0 @@
#!/bin/sh
set -e
if [ "$1" = "remove" ] || [ "$1" = "purge" ]; then
rm -f /usr/bin/px4
fi
exit 0
-144
View File
@@ -1,144 +0,0 @@
#!/usr/bin/env python3
"""
MAVSDK mission test for PX4 SIH SITL in Docker.
Takes off to 100m, flies a short 4-waypoint box mission, then lands.
Validates that the SIH Docker container works end-to-end with MAVSDK.
Prerequisites:
- Docker container running:
docker run --rm --network host px4io/px4-sitl:v1.17.0-alpha1
- pip install mavsdk
- mavsim-viewer running (optional):
/path/to/mavsim-viewer -n 1
Usage:
python3 Tools/packaging/test_sih_mission.py
python3 Tools/packaging/test_sih_mission.py --speed 10 # faster-than-realtime
"""
import asyncio
import argparse
import sys
import time
from mavsdk import System
from mavsdk.mission import MissionItem, MissionPlan
async def run_mission(speed_factor: int = 1):
drone = System()
print(f"Connecting to drone on udp://:14540 ...")
await drone.connect(system_address="udp://:14540")
print("Waiting for drone to connect...")
async for state in drone.core.connection_state():
if state.is_connected:
print(f"Connected (UUID: {state.uuid if hasattr(state, 'uuid') else 'N/A'})")
break
print("Waiting for global position estimate...")
async for health in drone.telemetry.health():
if health.is_global_position_ok and health.is_home_position_ok:
print("Global position OK")
break
# Get home position for reference
async for pos in drone.telemetry.position():
home_lat = pos.latitude_deg
home_lon = pos.longitude_deg
print(f"Home position: {home_lat:.6f}, {home_lon:.6f}")
break
# Build a small box mission at 100m AGL
# ~100m offset in each direction
offset = 0.001 # roughly 111m at equator
mission_items = [
MissionItem(
home_lat + offset, home_lon,
100, 10, True, float('nan'), float('nan'),
MissionItem.CameraAction.NONE,
float('nan'), float('nan'), float('nan'),
float('nan'), float('nan'),
MissionItem.VehicleAction.NONE,
),
MissionItem(
home_lat + offset, home_lon + offset,
100, 10, True, float('nan'), float('nan'),
MissionItem.CameraAction.NONE,
float('nan'), float('nan'), float('nan'),
float('nan'), float('nan'),
MissionItem.VehicleAction.NONE,
),
MissionItem(
home_lat, home_lon + offset,
100, 10, True, float('nan'), float('nan'),
MissionItem.CameraAction.NONE,
float('nan'), float('nan'), float('nan'),
float('nan'), float('nan'),
MissionItem.VehicleAction.NONE,
),
MissionItem(
home_lat, home_lon,
100, 10, True, float('nan'), float('nan'),
MissionItem.CameraAction.NONE,
float('nan'), float('nan'), float('nan'),
float('nan'), float('nan'),
MissionItem.VehicleAction.NONE,
),
]
mission_plan = MissionPlan(mission_items)
print(f"Uploading mission ({len(mission_items)} waypoints, 100m AGL)...")
await drone.mission.upload_mission(mission_plan)
print("Mission uploaded")
print("Arming...")
await drone.action.arm()
print("Armed")
t0 = time.time()
print("Starting mission...")
await drone.mission.start_mission()
# Monitor mission progress
async for progress in drone.mission.mission_progress():
elapsed = time.time() - t0
print(f" [{elapsed:6.1f}s] Waypoint {progress.current}/{progress.total}")
if progress.current == progress.total:
print(f"Mission complete in {elapsed:.1f}s (speed factor: {speed_factor}x)")
break
print("Returning to launch...")
await drone.action.return_to_launch()
# Wait for landing
async for in_air in drone.telemetry.in_air():
if not in_air:
print("Landed")
break
print("Disarming...")
await drone.action.disarm()
print("Test PASSED")
def main():
parser = argparse.ArgumentParser(description="PX4 SIH Docker mission test")
parser.add_argument("--speed", type=int, default=1,
help="PX4_SIM_SPEED_FACTOR (must match container)")
args = parser.parse_args()
try:
asyncio.run(run_mission(args.speed))
except KeyboardInterrupt:
print("\nInterrupted")
sys.exit(1)
except Exception as e:
print(f"Test FAILED: {e}")
sys.exit(1)
if __name__ == "__main__":
main()
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
# Script to run ShellCheck (a static analysis tool for shell scripts) over a
# script directory
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
GREEN='\033[0;32m'
NO_COLOR='\033[0m' # No Color
-7
View File
@@ -79,13 +79,6 @@ if [[ $INSTALL_SIM == "--sim-tools" ]]; then
elif [[ $REINSTALL_FORMULAS == "--reinstall" ]]; then
brew reinstall px4-sim
fi
# jMAVSim requires a JDK (Java 17 LTS recommended)
if ! brew ls --versions openjdk@17 > /dev/null; then
echo "[macos.sh] Installing OpenJDK 17 (required for jMAVSim)"
brew install openjdk@17
sudo ln -sfn $(brew --prefix openjdk@17)/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-17.jdk
fi
fi
echo "[macos.sh] All set! The PX4 Autopilot toolchain was installed."
+1 -1
View File
@@ -23,7 +23,7 @@ pyserial
pyulog>=0.5.0
pyyaml
requests
setuptools>=39.2.0,<=81.0.0
setuptools>=39.2.0
six>=1.12.0
sympy>=1.10.1
toml>=0.9
@@ -1,4 +1,4 @@
#!/usr/bin/env bash
#!/bin/bash
#
# Setup environment to make PX4 visible to Gazebo.
#

Some files were not shown because too many files have changed in this diff Show More