Compare commits

...

7 Commits

Author SHA1 Message Date
Ramon Roche 170b7b4102 feat(manifest): emit variant field from .px4board filename stem
Add an explicit "variant" field to the per-build manifest, sourced from
CMake's ${LABEL} variable which is already derived from the .px4board
filename stem (e.g. multicopter.px4board → "multicopter"). This is the
authoritative source of the variant name: CMake uses it everywhere else
in the tree for board selection.

Previously the variant was implicit in "target" (last underscore-
separated segment), forcing consumers to string-split and assume the
target always follows <board>_<variant>. Now it's a stable, machine-
readable field alongside target and label_pretty.

detect_firmware_category() now takes the variant as its first argument
instead of re-deriving it from target, removing a duplicated parse.

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 22:53:51 -07:00
Ramon Roche 0be3f937bd feat(manifest): add artifact_type discriminator field
Tag every build entry with artifact_type="px4" so the manifest schema
can represent non-.px4 producers in the future (VOXL2 .deb, Linux
tarballs, etc.) without a format_version bump. Purely additive: older
QGC ignores the unknown field, newer consumers branch on it.

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 21:12:46 -07:00
Ramon Roche eacfab99cc fix(manifest): correct px_mkfw shebang order and _merge_manifest logic
Move the shebang to line 1 so direct execution works on Linux; the vim
modeline was sitting above it and silently breaking ./Tools/px_mkfw.py.

Fix _merge_manifest to only merge dict values into dst["hardware"] when
the key is "hardware". The previous structure ran the isinstance check
unconditionally, so any dict-valued field in the fragment would have
been shoved into dst["hardware"] regardless of its key, and could
KeyError if a non-"hardware" dict arrived first. It only worked today
because "hardware" happened to be the only dict in the fragment.

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 21:12:36 -07:00
Ramon Roche 63f059fd47 manifest: add label_pretty and firmware_category fields
Add human-readable labels and firmware classification to the build
manifest so ground stations like QGC can display friendly names and
filter builds by category.

New fields:
- label_pretty: human-readable variant name (e.g. "Multicopter")
- firmware_category: auto-detected classification
  - "vehicle" for multicopter, fixedwing, vtol, rover, uuv, spacecraft
  - "peripheral" for CAN sensor nodes (GPS, flow, mag, etc.)
  - "bootloader" for bootloader/canbootloader
  - "dev" for everything else (default, zenoh, mavlink-dev, etc.)

Peripheral detection uses ROMFSROOT="cannode" which is shared by all
~18 CAN sensor boards across ARK, Holybro, CUAV, Freefly, Matek, NXP.

A build-time warning fires when an unrecognized label falls through to
"dev", so new vehicle types are not silently hidden from end users.

Boards can override via CONFIG_BOARD_FIRMWARE_CATEGORY in .px4board.
CONFIG_BOARD_LABEL_PRETTY set on all px4/fmu-v6x variants as Phase 1.

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 17:37:54 -07:00
Ramon Roche 68c391328a ci: unified firmware manifest for release discovery
Replace the two-tier approach (per-release manifest + top-level index)
with a single unified manifest containing all releases and their builds
inline. This allows QGC to fetch one file to find firmware for any board
across all releases, avoiding the need to fetch many per-release manifests.

Schema uses format_version 2 with releases as a dict keyed by version
for O(1) upserts. The manifest is stored at s3://px4-travis/Firmware/manifest.json
and updated on every CI build (tags + main/stable/beta branches).

Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 17:37:38 -07:00
Ramon Roche 80abad2baa build: bundle manifest to *.px4 file
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 17:37:18 -07:00
Ramon Roche ee056c1536 tools: create firmware manifest
Signed-off-by: Ramon Roche <mrpollo@gmail.com>
2026-04-08 17:36:51 -07:00
22 changed files with 811 additions and 5 deletions
+57
View File
@@ -210,6 +210,10 @@ jobs:
outputs:
uploadlocation: ${{ steps.upload-location.outputs.uploadlocation }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: Tools/manifest
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
@@ -269,4 +273,57 @@ jobs:
artifacts/*.px4
artifacts/*.deb
artifacts/**/*.sbom.spdx.json
manifest.json
name: ${{ steps.upload-location.outputs.uploadlocation }}
# Update the unified firmware manifest with this release
# The manifest lives at s3://px4-travis/Firmware/manifest.json and provides
# a complete index of all releases and builds for tools like QGroundControl
- name: Backup Existing Firmware Manifest
if: startsWith(github.ref, 'refs/tags/v')
run: |
s3_base="https://px4-travis.s3.us-west-1.amazonaws.com/Firmware"
mkdir -p manifest_backup
curl -sf "${s3_base}/manifest.json" -o manifest_backup/manifest.json.backup || echo "No existing manifest to backup"
if [ -f manifest_backup/manifest.json.backup ]; then
echo "Backed up existing firmware manifest ($(wc -c < manifest_backup/manifest.json.backup) bytes)"
fi
- name: Upload Firmware Manifest Backup
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/upload-artifact@v4
with:
name: firmware_manifest_backup_${{ steps.upload-location.outputs.uploadlocation }}
path: manifest_backup/
if-no-files-found: ignore
retention-days: 90
- name: Update Firmware Manifest
if: startsWith(github.ref, 'refs/tags/v')
run: |
version="${{ steps.upload-location.outputs.uploadlocation }}"
git_tag=""
base_url=""
s3_base="https://px4-travis.s3.us-west-1.amazonaws.com/Firmware"
if [[ "$GITHUB_REF" == refs/tags/v* ]]; then
git_tag="${GITHUB_REF#refs/tags/}"
base_url="https://github.com/${{ github.repository }}/releases/download/${git_tag}"
fi
python3 ./Tools/manifest/update_firmware_manifest.py \
--dir artifacts/ \
--version "$version" \
--git-tag "$git_tag" \
--base-url "$base_url" \
--fetch-url "${s3_base}/manifest.json" \
--out manifest.json
- name: Upload Firmware Manifest to S3
if: startsWith(github.ref, 'refs/tags/v')
run: |
aws s3 cp manifest.json s3://px4-travis/Firmware/manifest.json --acl public-read
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: 'us-west-1'
+24
View File
@@ -54,6 +54,30 @@ menu "Toolchain"
string "Architecture"
default ""
config BOARD_LABEL_PRETTY
string "Human-readable label for this build variant"
default ""
help
A short display name for this build variant (e.g. "Multicopter",
"Rover"). Used by ground stations like QGC to show a user-friendly
name instead of raw target strings. Must be set in every .px4board
variant file because non-base variants inherit from default.
config BOARD_FIRMWARE_CATEGORY
string "Firmware category override (vehicle, peripheral, dev, bootloader)"
default ""
help
Override the auto-detected firmware category. Normally inferred
from the build label and board config: vehicle types (multicopter,
fixedwing, etc.) map to "vehicle", bootloader/canbootloader map to
"bootloader", CAN peripheral boards (ROMFSROOT=cannode) map to
"peripheral", and everything else maps to "dev". Set this only
when the auto-detection is wrong for a particular board variant.
IMPORTANT: If you add a new vehicle type with a label not yet in
_VEHICLE_LABELS (in gen_board_manifest_from_defconfig.py), either
add it there or set this to "vehicle" — otherwise the build will
be hidden from end-users in QGC.
config BOARD_LTO
bool "(EXPERIMENTAL) Link Time Optimization (LTO)"
default n
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
import argparse, json, os, re, sys
from typing import Dict
_VEHICLE_LABELS = frozenset({
"multicopter", "fixedwing", "vtol", "rover", "uuv", "spacecraft",
})
_BOOTLOADER_LABELS = frozenset({
"bootloader", "canbootloader",
})
def parse_defconfig(path: str) -> Dict[str, str]:
d: Dict[str, str] = {}
if not path or not os.path.exists(path):
return d
with open(path, "r", encoding="utf-8", errors="ignore") as f:
for raw in f:
line = raw.strip()
if not line or not line.startswith("CONFIG_") or "=" not in line or line.startswith("#"):
continue
k, v = line.split("=", 1)
v = v.strip()
if len(v) >= 2 and v[0] == '"' and v[-1] == '"':
v = v[1:-1]
d[k.strip()] = v
return d
def norm_hex(s: str) -> str:
if not s:
return ""
s = s.strip()
if s.lower().startswith("0x"):
return s.lower()
try:
return f"0x{int(s, 0):04x}"
except Exception:
return s
def detect_chip(defcfg: Dict[str,str]) -> str:
for k, v in defcfg.items():
if k.startswith("CONFIG_ARCH_CHIP_") and v == "y":
return k[len("CONFIG_ARCH_CHIP_"):].lower().replace("_", "")
s = defcfg.get("CONFIG_ARCH_CHIP", "")
return s.lower().replace("_", "") if s else ""
def detect_firmware_category(variant: str, defcfg: Dict[str, str], target: str = "") -> str:
"""Infer firmware_category from the variant label and defconfig.
Detection order:
1. bootloader / canbootloader variants → "bootloader"
2. Known vehicle variants → "vehicle"
3. ROMFSROOT == "cannode" (CAN peripheral boards) → "peripheral"
4. Everything else → "dev"
If you are adding a NEW vehicle type (e.g. "balloon"), you must EITHER:
1. Add the label to _VEHICLE_LABELS in this file, OR
2. Set CONFIG_BOARD_FIRMWARE_CATEGORY="vehicle" in the .px4board file
Otherwise the build will be classified as "dev" and hidden from
end-users in ground stations like QGroundControl.
"""
if variant in _BOOTLOADER_LABELS:
return "bootloader"
if variant in _VEHICLE_LABELS:
return "vehicle"
# CAN peripheral boards (sensors, GPS, flow, etc.) use cannode ROMFS
romfsroot = defcfg.get("CONFIG_BOARD_ROMFSROOT", "")
if romfsroot == "cannode":
return "peripheral"
if variant not in ("default", ""):
print(f"WARNING: variant '{variant}' (target '{target}') is not a known "
f"vehicle type — defaulting firmware_category to 'dev'. "
f"If this is a vehicle type, add it to _VEHICLE_LABELS in "
f"{__file__} or set CONFIG_BOARD_FIRMWARE_CATEGORY in the "
f".px4board file.", file=sys.stderr)
return "dev"
def pick(preferred: str, fallback_key: str, defcfg: Dict[str, str]) -> str:
return preferred if preferred else defcfg.get(fallback_key, "")
def main():
ap = argparse.ArgumentParser(description="Generate board manifest (prefer CMake-passed overrides, fallback to defconfig).")
ap.add_argument("--defconfig", required=False, help="Path to defconfig (fallback only)")
# explicit overrides coming from CMake
ap.add_argument("--manufacturer", default="")
ap.add_argument("--productstr", default="")
ap.add_argument("--target", default="")
ap.add_argument("--name", default="")
ap.add_argument("--variant", default="",
help="Build variant label (the .px4board filename stem, e.g. 'multicopter')")
ap.add_argument("--arch", default="")
ap.add_argument("--chip", default="")
ap.add_argument("--vid", default="")
ap.add_argument("--pid", default="")
ap.add_argument("--label-pretty", default="")
ap.add_argument("--firmware-category", default="")
ap.add_argument("--out", help="Write to file instead of stdout")
args = ap.parse_args()
defcfg = parse_defconfig(args.defconfig) if args.defconfig else {}
manufacturer = pick(args.manufacturer, "CONFIG_BOARD_MANUFACTURER", defcfg)
productstr = pick(args.productstr, "CONFIG_BOARD_PRODUCTSTR", defcfg)
target = args.target or ""
name = args.name or ""
variant = (args.variant or "").lower()
arch = (pick(args.arch, "CONFIG_ARCH", defcfg)).lower()
chip = args.chip or detect_chip(defcfg)
vid = norm_hex(pick(args.vid, "CONFIG_CDCACM_VENDORID", defcfg))
pid = norm_hex(pick(args.pid, "CONFIG_CDCACM_PRODUCTID", defcfg))
label_pretty = pick(args.label_pretty, "CONFIG_BOARD_LABEL_PRETTY", defcfg)
firmware_cat = pick(args.firmware_category,"CONFIG_BOARD_FIRMWARE_CATEGORY",defcfg)
if not firmware_cat:
firmware_cat = detect_firmware_category(variant, defcfg, target=target)
manifest = {
"name": name,
"target": target,
"variant": variant,
"label_pretty": label_pretty,
"firmware_category": firmware_cat,
"manufacturer": manufacturer,
"hardware": {
"architecture": arch,
"vendor_id": vid,
"product_id": pid,
"chip": chip,
"productstr": productstr
}
}
if args.out:
out_dir = os.path.dirname(args.out)
if out_dir:
os.makedirs(out_dir, exist_ok=True)
with open(args.out, "w", encoding="utf-8") as f:
json.dump(manifest, f, indent=2)
f.write("\n")
else:
json.dump(manifest, sys.stdout, indent=2)
sys.stdout.write("\n")
return 0
if __name__ == "__main__":
sys.exit(main())
+367
View File
@@ -0,0 +1,367 @@
#!/usr/bin/env python3
############################################################################
#
# Copyright (c) 2025 PX4 Development Team. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in
# the documentation and/or other materials provided with the
# distribution.
# 3. Neither the name PX4 nor the names of its contributors may be
# used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
# OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
#
############################################################################
"""
PX4 Unified Firmware Manifest
Scans a directory for *.px4 firmware files, extracts their embedded metadata,
and upserts the release into a single unified manifest JSON file containing
all releases with all firmware variants inline.
The manifest is stored at s3://px4-travis/Firmware/manifest.json and provides
a complete index of all available firmware for tools like QGroundControl.
"""
import argparse
import json
import re
import sys
import time
from pathlib import Path
from typing import Any, Dict, List, Optional
from urllib.request import urlopen
from urllib.error import URLError
def extract_px4_metadata(px4_path: Path) -> Optional[Dict[str, Any]]:
"""
Extract metadata from a .px4 firmware file.
The .px4 file is a JSON-encoded object containing firmware metadata
and a compressed binary image.
"""
try:
with open(px4_path, "r", encoding="utf-8") as f:
data = json.load(f)
# Verify this is a valid firmware file by checking required fields
# Different boards use different magic strings (PX4FWv1, PX4FWv2, ARKFWv1, etc.)
if "magic" not in data or "board_id" not in data or "image" not in data:
print(f"Warning: {px4_path.name} missing required fields, skipping", file=sys.stderr)
return None
# Extract relevant metadata (exclude the large binary data)
metadata = {
"filename": px4_path.name,
"board_id": data.get("board_id", 0),
"board_revision": data.get("board_revision", 0),
"version": data.get("version", ""),
"git_identity": data.get("git_identity", ""),
"git_hash": data.get("git_hash", ""),
"build_time": data.get("build_time", 0),
"image_size": data.get("image_size", 0),
"sha256sum": data.get("sha256sum", ""),
"mav_autopilot": data.get("mav_autopilot", 12),
}
# Include the board manifest if present
if "manifest" in data and data["manifest"]:
metadata["manifest"] = data["manifest"]
# Additive: discriminator so future non-.px4 producers (VOXL2 .deb,
# Linux tarballs, etc.) can coexist in the same manifest. Older QGC
# ignores unknown fields, new consumers branch on this.
metadata["artifact_type"] = "px4"
return metadata
except json.JSONDecodeError as e:
print(f"Warning: Failed to parse {px4_path.name}: {e}", file=sys.stderr)
return None
except Exception as e:
print(f"Warning: Error reading {px4_path.name}: {e}", file=sys.stderr)
return None
def determine_channel(version: str) -> str:
"""
Determine the release channel from version string.
- stable: vX.Y.Z (no suffix)
- beta: vX.Y.Z-beta*, vX.Y.Z-rc*
- dev: vX.Y.Z-alpha*, vX.Y.Z-dev*, or any other suffix
"""
version_lower = version.lower()
if re.match(r"^v?\d+\.\d+\.\d+$", version):
return "stable"
elif "-beta" in version_lower or "-rc" in version_lower:
return "beta"
else:
return "dev"
def parse_version_tuple(version: str) -> tuple:
"""
Parse version string into a tuple for sorting.
Returns (major, minor, patch, prerelease_type, prerelease_num)
"""
# Remove 'v' prefix if present
v = version.lstrip("v")
# Match version pattern
match = re.match(r"(\d+)\.(\d+)\.(\d+)(?:-([a-zA-Z]+)(\d+)?)?", v)
if not match:
return (0, 0, 0, "zzz", 0) # Unknown versions sort last
major, minor, patch = int(match.group(1)), int(match.group(2)), int(match.group(3))
prerelease_type = match.group(4) or ""
prerelease_num = int(match.group(5)) if match.group(5) else 0
# Stable releases (no prerelease) should sort after prereleases
# Use empty string to sort after alpha/beta/rc
if not prerelease_type:
prerelease_type = "zzz" # Sorts after alpha, beta, rc
return (major, minor, patch, prerelease_type.lower(), prerelease_num)
def fetch_existing_manifest(url: str) -> Optional[Dict[str, Any]]:
"""
Fetch the existing firmware manifest from a URL.
Returns None if the manifest doesn't exist or can't be fetched.
"""
try:
with urlopen(url, timeout=30) as response:
return json.loads(response.read().decode("utf-8"))
except URLError as e:
print(f"Note: Could not fetch existing manifest from {url}: {e}", file=sys.stderr)
return None
except json.JSONDecodeError as e:
print(f"Warning: Invalid JSON in existing manifest: {e}", file=sys.stderr)
return None
def load_existing_manifest(path: Path) -> Optional[Dict[str, Any]]:
"""
Load the existing firmware manifest from a local file.
Returns None if the file doesn't exist.
"""
if not path.exists():
return None
try:
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
except json.JSONDecodeError as e:
print(f"Warning: Invalid JSON in existing manifest file: {e}", file=sys.stderr)
return None
def create_empty_manifest() -> Dict[str, Any]:
"""Create a new empty unified firmware manifest."""
return {
"format_version": 2,
"updated_at": int(time.time()),
"description": "PX4 Firmware Manifest",
"releases": {},
}
def scan_builds(px4_dir: Path, base_url: str = "") -> List[Dict[str, Any]]:
"""
Scan a directory for .px4 files and extract metadata from each.
Returns a sorted list of build metadata dicts.
"""
builds: List[Dict[str, Any]] = []
px4_files = sorted(px4_dir.glob("*.px4"))
if not px4_files:
print(f"Warning: No .px4 files found in {px4_dir}", file=sys.stderr)
for px4_path in px4_files:
metadata = extract_px4_metadata(px4_path)
if metadata:
if base_url:
metadata["url"] = base_url.rstrip("/") + "/" + metadata["filename"]
builds.append(metadata)
# Sort builds by target name for consistent ordering
builds.sort(key=lambda b: b.get("manifest", {}).get("target", b["filename"]))
return builds
def update_latest_pointers(manifest: Dict[str, Any]) -> None:
"""
Recompute latest_stable, latest_beta, and latest_dev from the releases dict.
Modifies manifest in place.
"""
channels: Dict[str, List[str]] = {"stable": [], "beta": [], "dev": []}
for version, release in manifest["releases"].items():
ch = release.get("channel", "dev")
if ch in channels:
channels[ch].append(version)
for ch, versions in channels.items():
if versions:
versions.sort(key=parse_version_tuple, reverse=True)
manifest[f"latest_{ch}"] = versions[0]
elif f"latest_{ch}" in manifest:
del manifest[f"latest_{ch}"]
def update_manifest(
manifest: Dict[str, Any],
version: str,
git_tag: str,
builds: List[Dict[str, Any]],
) -> Dict[str, Any]:
"""
Upsert a release entry into the unified manifest with all builds inline.
"""
channel = determine_channel(version)
release_date = time.strftime("%Y-%m-%d")
manifest["releases"][version] = {
"git_tag": git_tag,
"release_date": release_date,
"channel": channel,
"build_count": len(builds),
"generated_at": int(time.time()),
"builds": builds,
}
manifest["updated_at"] = int(time.time())
update_latest_pointers(manifest)
return manifest
def main() -> int:
parser = argparse.ArgumentParser(
description="Update the unified PX4 firmware manifest with a new release.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Generate manifest from local .px4 files (fresh start)
%(prog)s --dir ./artifacts --version v1.15.0 --git-tag v1.15.0 --out manifest.json
# Update existing manifest from S3 with a new release
%(prog)s --dir ./artifacts --version v1.15.0 --git-tag v1.15.0 \\
--base-url https://github.com/PX4/PX4-Autopilot/releases/download/v1.15.0 \\
--fetch-url https://px4-travis.s3.us-west-1.amazonaws.com/Firmware/manifest.json \\
--out manifest.json
# Update an existing local manifest file
%(prog)s --dir ./artifacts --version master --existing manifest.json --out manifest.json
""",
)
parser.add_argument(
"--dir",
type=Path,
required=True,
help="Directory containing .px4 files to scan",
)
parser.add_argument(
"--version",
required=True,
help="Release version / key (e.g., v1.15.0, master)",
)
parser.add_argument(
"--git-tag",
default="",
help="Git tag for this release (empty for branch builds)",
)
parser.add_argument(
"--base-url",
default="",
help="Base URL prefix for firmware download URLs",
)
parser.add_argument(
"--fetch-url",
default="",
help="URL to fetch existing manifest from (e.g., S3 public URL)",
)
parser.add_argument(
"--existing",
type=Path,
help="Path to existing local manifest file to update",
)
parser.add_argument(
"--out",
type=Path,
required=True,
help="Output file path for updated manifest",
)
args = parser.parse_args()
if not args.dir.is_dir():
print(f"Error: {args.dir} is not a directory", file=sys.stderr)
return 1
# Load or create manifest
manifest = None
if args.existing:
manifest = load_existing_manifest(args.existing)
if manifest:
print(f"Loaded existing manifest from {args.existing}", file=sys.stderr)
if manifest is None and args.fetch_url:
manifest = fetch_existing_manifest(args.fetch_url)
if manifest:
print(f"Fetched existing manifest from {args.fetch_url}", file=sys.stderr)
if manifest is None:
print("Creating new firmware manifest", file=sys.stderr)
manifest = create_empty_manifest()
# Scan .px4 files
builds = scan_builds(args.dir, base_url=args.base_url)
# Upsert release into manifest
manifest = update_manifest(
manifest=manifest,
version=args.version,
git_tag=args.git_tag,
builds=builds,
)
# Write output
args.out.parent.mkdir(parents=True, exist_ok=True)
with open(args.out, "w", encoding="utf-8") as f:
json.dump(manifest, f, indent=2)
f.write("\n")
release = manifest["releases"][args.version]
print(
f"Updated firmware manifest: {args.out} "
f"({len(manifest['releases'])} releases, "
f"{release['build_count']} builds for {args.version})",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
sys.exit(main())
+31 -3
View File
@@ -1,4 +1,5 @@
#!/usr/bin/env python3
# vim: set noexpandtab tabstop=4 shiftwidth=4:
############################################################################
#
# Copyright (C) 2012, 2013 PX4 Development Team. All rights reserved.
@@ -46,6 +47,7 @@ import os
import zlib
import time
import subprocess
import hashlib
#
# Construct a basic firmware description
@@ -63,8 +65,18 @@ def mkdesc():
proto['build_time'] = 0
proto['image'] = bytes()
proto['image_size'] = 0
proto['sha256sum'] = ""
return proto
def _merge_manifest(dst, src):
if not isinstance(src, dict):
return
for k, v in src.items():
if k == "hardware" and isinstance(v, dict):
dst.setdefault("hardware", {}).update(v)
else:
dst[k] = v
# Parse commandline
parser = argparse.ArgumentParser(description="Firmware generator for the PX autopilot system.")
parser.add_argument("--prototype", action="store", help="read a prototype description from a file")
@@ -77,6 +89,7 @@ parser.add_argument("--git_identity", action="store", help="the working director
parser.add_argument("--parameter_xml", action="store", help="the parameters.xml file")
parser.add_argument("--airframe_xml", action="store", help="the airframes.xml file")
parser.add_argument("--image", action="store", help="the firmware image")
parser.add_argument("--manifest_json", action="append", help="path to manifest JSON fragment to merge")
args = parser.parse_args()
# Fetch the firmware descriptor prototype if specified
@@ -87,6 +100,9 @@ if args.prototype != None:
else:
desc = mkdesc()
desc.setdefault("manifest_version", 1)
desc.setdefault("manifest", {})
desc['build_time'] = int(time.time())
if args.board_id != None:
@@ -120,8 +136,20 @@ if args.airframe_xml != None:
desc['airframe_xml'] = base64.b64encode(zlib.compress(bytes,9)).decode('utf-8')
if args.image != None:
f = open(args.image, "rb")
bytes = f.read()
desc['image_size'] = len(bytes)
desc['image'] = base64.b64encode(zlib.compress(bytes,9)).decode('utf-8')
raw_image = f.read()
f.close()
desc['image_size'] = len(raw_image)
sha256sum = hashlib.sha256(raw_image).hexdigest()
desc['sha256sum'] = sha256sum
desc['image'] = base64.b64encode(zlib.compress(raw_image, 9)).decode('utf-8')
# merge manifest
manifest_inputs = args.manifest_json or []
if isinstance(manifest_inputs, str):
manifest_inputs = [manifest_inputs]
for p in manifest_inputs:
with open(p, "r", encoding="utf-8") as f:
frag = json.load(f)
_merge_manifest(desc["manifest"], frag)
print(json.dumps(desc, indent=4))
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_TOOLCHAIN="arm-none-eabi"
CONFIG_BOARD_ARCHITECTURE="cortex-m7"
CONFIG_BOARD_LABEL_PRETTY="Bootloader"
CONFIG_BOARD_ROMFSROOT=""
+1
View File
@@ -1,5 +1,6 @@
CONFIG_BOARD_TOOLCHAIN="arm-none-eabi"
CONFIG_BOARD_ARCHITECTURE="cortex-m7"
CONFIG_BOARD_LABEL_PRETTY="Default"
CONFIG_BOARD_ETHERNET=y
CONFIG_BOARD_SERIAL_GPS1="/dev/ttyS0"
CONFIG_BOARD_SERIAL_GPS2="/dev/ttyS7"
@@ -1 +1,2 @@
CONFIG_BOARD_LABEL_PRETTY="Flash Analysis"
CONFIG_BOARD_LINKER_PREFIX="flash-analysis"
+1
View File
@@ -1,2 +1,3 @@
CONFIG_BOARD_LABEL_PRETTY="MAVLink Dev"
CONFIG_MAVLINK_DIALECT="development"
CONFIG_MODULES_UXRCE_DDS_CLIENT=n
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_LABEL_PRETTY="Multicopter"
CONFIG_COMMON_DIFFERENTIAL_PRESSURE=n
CONFIG_MODE_NAVIGATOR_VTOL_TAKEOFF=n
CONFIG_MODULES_AIRSPEED_SELECTOR=n
@@ -1,5 +1,6 @@
CONFIG_BOARD_TOOLCHAIN="arm-none-eabi"
CONFIG_BOARD_ARCHITECTURE="cortex-m7"
CONFIG_BOARD_LABEL_PRETTY="Performance Test"
CONFIG_BOARD_ETHERNET=y
CONFIG_BOARD_SERIAL_GPS1="/dev/ttyS0"
CONFIG_BOARD_SERIAL_GPS2="/dev/ttyS7"
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_LABEL_PRETTY="Rover"
CONFIG_MODULES_AIRSPEED_SELECTOR=n
CONFIG_MODULES_FLIGHT_MODE_MANAGER=n
CONFIG_MODULES_FW_ATT_CONTROL=n
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_LABEL_PRETTY="Spacecraft"
CONFIG_BOARD_PWM_FREQ=100000
CONFIG_MODULES_AIRSPEED_SELECTOR=n
CONFIG_MODULES_FLIGHT_MODE_MANAGER=n
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_LABEL_PRETTY="UUV"
CONFIG_MODULES_AIRSPEED_SELECTOR=n
CONFIG_MODULES_FLIGHT_MODE_MANAGER=n
CONFIG_MODULES_FW_ATT_CONTROL=n
+1
View File
@@ -1,3 +1,4 @@
CONFIG_BOARD_LABEL_PRETTY="Zenoh"
# CONFIG_BOARD_UAVCAN_TIMER_OVERRIDE is not set
CONFIG_DRIVERS_UAVCAN=n
CONFIG_MODULES_UXRCE_DDS_CLIENT=n
+1
View File
@@ -481,6 +481,7 @@
- [PX4 Board Configuration (kconfig)](hardware/porting_guide_config.md)
- [NuttX Board Porting Guide](hardware/porting_guide_nuttx.md)
- [Board Firmware Packaging (.deb)](hardware/board_packaging.md)
- [Firmware Manifest & Metadata](hardware/firmware_manifest.md)
- [Serial Port Mapping](hardware/serial_port_mapping.md)
- [Airframes](dev_airframes/index.md)
- [Adding a New Airframe](dev_airframes/adding_a_new_frame.md)
+1
View File
@@ -478,6 +478,7 @@
- [Flight Controller Porting Guide](/hardware/porting_guide.md)
- [PX4 Board Configuration (kconfig)](/hardware/porting_guide_config.md)
- [NuttX Board Porting Guide](/hardware/porting_guide_nuttx.md)
- [Firmware Manifest & Metadata](/hardware/firmware_manifest.md)
- [Serial Port Mapping](/hardware/serial_port_mapping.md)
- [Airframes](/dev_airframes/index.md)
- [Adding a New Airframe](/dev_airframes/adding_a_new_frame.md)
+9
View File
@@ -78,6 +78,15 @@ If _QGroundControl_ installs the FMUv2 target (see console during installation),
You can update it by following the instructions in [Bootloader update > FMUv2 Bootloader Update](../advanced_config/bootloader_update.md#fmuv2-bootloader-update).
## Firmware Variants
PX4 boards may provide multiple firmware variants for different vehicle types (multicopter, fixed-wing, rover, etc.).
QGroundControl shows vehicle firmware by default.
CAN peripheral firmware (for sensor nodes like GPS, optical flow, magnetometer, etc.) is shown in a separate section.
Developer builds (such as `default` or `zenoh`) are available in advanced mode.
For technical details on how firmware variants are classified and discovered, see [Firmware Manifest & Metadata](../hardware/firmware_manifest.md).
## Further Information
- [QGroundControl User Guide > Firmware](https://docs.qgroundcontrol.com/master/en/qgc-user-guide/setup_view/firmware.html).
+3
View File
@@ -113,6 +113,9 @@ Contact PX4 board maintainers at [boards@px4.io](mailto:boards@px4.io) and reque
2. The assignment of REV and VER ID resistor values.
3. If the board supports USB: Either request the assignment of a USB VID and PID or provide the USB VID and PID.
4. Set `CONFIG_BOARD_LABEL_PRETTY` in all `.px4board` variant files so that ground stations can display a human-readable name for each build variant.
See [Firmware Manifest & Metadata](firmware_manifest.md) for details on firmware categories and how QGC discovers firmware.
Integrate the board according to the board porting release process described in the [porting guide](../hardware/porting_guide.md)
:::warning
+110
View File
@@ -0,0 +1,110 @@
# Firmware Manifest & Metadata
Each PX4 NuttX build produces a **manifest** — a JSON metadata object that describes the firmware target, board hardware, and build variant.
This manifest is embedded in the `.px4` firmware file and aggregated into a unified release manifest consumed by ground stations like QGroundControl for firmware discovery and selection.
## Manifest Schema
```json
{
"name": "px4_fmu-v6x",
"target": "px4_fmu-v6x_multicopter",
"label_pretty": "Multicopter",
"firmware_category": "vehicle",
"manufacturer": "Holybro",
"hardware": {
"architecture": "cortex-m7",
"vendor_id": "0x1234",
"product_id": "0x5678",
"chip": "stm32h753ii",
"productstr": "Pixhawk 6X"
}
}
```
### Field Descriptions
| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Board name without variant label (e.g. `px4_fmu-v6x`) |
| `target` | string | Full build target including variant (e.g. `px4_fmu-v6x_multicopter`) |
| `label_pretty` | string | Human-readable variant name shown in ground stations (e.g. "Multicopter") |
| `firmware_category` | string | Build classification: `vehicle`, `peripheral`, `dev`, or `bootloader` |
| `manufacturer` | string | Board manufacturer name |
| `hardware` | object | Hardware details (architecture, USB IDs, chip, product string) |
## Firmware Categories
The `firmware_category` field classifies each build for ground station filtering:
| Value | Description | Examples | Ground Station Behavior |
|-------|-------------|----------|------------------------|
| `vehicle` | Production firmware for a vehicle type | multicopter, fixedwing, vtol, rover, uuv, spacecraft | Shown to users (primary) |
| `peripheral` | Firmware for CAN sensor nodes and peripherals | ark/can-gps, holybro/can-gps-v1, ark/can-flow, ark/mag | Shown in a dedicated peripheral/sensor section |
| `dev` | Developer/engineering builds | default, zenoh, mavlink-dev, flash-analysis, performance-test | Hidden by default, advanced mode only |
| `bootloader` | Bootloader binaries | bootloader, canbootloader | Never shown to end users |
### Auto-Detection
The firmware category is automatically inferred from the build label (the last segment of the target string after the final `_`):
- Labels `multicopter`, `fixedwing`, `vtol`, `rover`, `uuv`, `spacecraft``vehicle`
- Labels `bootloader`, `canbootloader``bootloader`
- Boards with `CONFIG_BOARD_ROMFSROOT="cannode"` (CAN sensor peripherals) → `peripheral`
- Everything else (`default`, `zenoh`, `mavlink-dev`, etc.) → `dev`
The known vehicle labels are maintained in `_VEHICLE_LABELS` in `Tools/manifest/gen_board_manifest_from_defconfig.py`.
Peripheral detection uses the `cannode` ROMFS root, which is shared by all CAN sensor boards (ARK, Holybro, CUAV, Freefly, Matek, NXP, etc.).
A build-time warning is emitted to stderr when an unrecognized label (other than `default`) falls through to `dev`, so that new vehicle types are not silently hidden from end users.
### Adding a New Vehicle Type
If you are adding a new vehicle type (e.g. `balloon`), you must do **one** of the following — otherwise the build is silently classified as `dev` and hidden from end users in QGroundControl:
1. **Add the label to `_VEHICLE_LABELS`** in `Tools/manifest/gen_board_manifest_from_defconfig.py` — this is the preferred approach when the vehicle type applies across multiple boards.
2. **Set `CONFIG_BOARD_FIRMWARE_CATEGORY="vehicle"`** in the `.px4board` file — use this for one-off overrides or when a board variant doesn't follow the standard naming.
### Overriding Auto-Detection
If a board variant needs a non-standard classification for any reason, set `CONFIG_BOARD_FIRMWARE_CATEGORY` in the `.px4board` file:
```
CONFIG_BOARD_FIRMWARE_CATEGORY="vehicle"
```
This override takes precedence over auto-detection.
## Pretty Labels (`label_pretty`)
The `label_pretty` field provides a human-readable name for each build variant.
Ground stations display this instead of raw target strings like `px4_fmu-v6x_default`.
### Setting `label_pretty`
Set `CONFIG_BOARD_LABEL_PRETTY` in each `.px4board` file:
```
CONFIG_BOARD_LABEL_PRETTY="Multicopter"
```
### Kconfig Inheritance Caveat
Non-base variants (e.g. `multicopter.px4board`, `rover.px4board`) are merged on top of `default.px4board`.
If `default.px4board` sets `CONFIG_BOARD_LABEL_PRETTY="Default"` and a variant file does not override it, the variant inherits the "Default" label — which is incorrect.
**Every variant file must set its own `CONFIG_BOARD_LABEL_PRETTY`.**
See [PX4 Board Configuration > Kconfig Label Inheritance](porting_guide_config.md#px4-kconfig-label-inheritance) for more details on how variant configs inherit from defaults.
## Build Pipeline
The manifest flows through three stages:
1. **`gen_board_manifest_from_defconfig.py`** — Generates per-build `manifest.json` from CMake variables and defconfig values, including auto-detection of `firmware_category`
2. **`px_mkfw.py`** — Bundles the manifest into the `.px4` firmware file
3. **`update_firmware_manifest.py`** — Aggregates individual `.px4` manifests into a unified release manifest for ground station consumption
## Backward Compatibility
- Old `.px4` files without `label_pretty` / `firmware_category`: ground stations fall back to raw label and filename matching (existing behavior)
- These are additive fields — no `format_version` bump is needed
+24
View File
@@ -45,6 +45,30 @@ When changing the `cyphal.px4board` it only stores the delta of the Kconfig keys
When modifying a Kconfig key in `default.px4board` it will be modified in all derivative configurations of the same board that had the same config as well.
:::
## Board Metadata Fields
PX4 board files can include metadata used by ground stations for firmware discovery and display.
### `CONFIG_BOARD_LABEL_PRETTY`
A human-readable name for the build variant (e.g. `"Multicopter"`, `"Rover"`).
Ground stations like QGroundControl display this instead of raw target strings.
```
CONFIG_BOARD_LABEL_PRETTY="Multicopter"
```
::: warning
Due to [Kconfig label inheritance](#px4-kconfig-label-inheritance), every variant file must set its own `CONFIG_BOARD_LABEL_PRETTY`.
If omitted, a variant inherits the value from `default.px4board`, which is typically wrong.
:::
### `CONFIG_BOARD_FIRMWARE_CATEGORY`
Optional override for the auto-detected firmware category (`vehicle`, `peripheral`, `dev`, or `bootloader`).
Normally this is inferred from the build label and does not need to be set.
See [Firmware Manifest & Metadata](firmware_manifest.md) for details on the auto-detection rules and when to use this override.
## PX4 Menuconfig Setup
The [menuconfig](https://pypi.org/project/kconfiglib/#menuconfig-interfaces) tool is used to modify the PX4 board configuration, adding/removing modules, drivers, and other features.
+30 -2
View File
@@ -422,9 +422,34 @@ endif()
# create .px4 with parameter and airframe metadata
if (TARGET parameters_xml AND TARGET airframes_xml)
string(REPLACE ".elf" ".px4" fw_package ${PX4_BINARY_DIR}/${FW_NAME})
# Generate manifest object
set(MANIFEST_JSON ${PX4_BINARY_DIR}/manifest.json)
add_custom_command(
OUTPUT ${MANIFEST_JSON}
COMMAND
${PYTHON_EXECUTABLE} ${PX4_SOURCE_DIR}/Tools/manifest/gen_board_manifest_from_defconfig.py
--defconfig ${NUTTX_DEFCONFIG}
--manufacturer "${CONFIG_CDCACM_VENDORSTR}"
--productstr "${CONFIG_CDCACM_PRODUCTSTR}"
--target "${PX4_CONFIG}"
--arch "${CONFIG_ARCH}"
--name "${PX4_BOARD_NAME}"
--variant "${LABEL}"
--chip "${CONFIG_ARCH_CHIP}"
--vid "${CONFIG_CDCACM_VENDORID}"
--pid "${CONFIG_CDCACM_PRODUCTID}"
--label-pretty "${CONFIG_BOARD_LABEL_PRETTY}"
--firmware-category "${CONFIG_BOARD_FIRMWARE_CATEGORY}"
--out ${MANIFEST_JSON}
DEPENDS
${PX4_SOURCE_DIR}/Tools/manifest/gen_board_manifest_from_defconfig.py
${NUTTX_DEFCONFIG}
COMMENT "Generating board specific manifest from defconfig"
VERBATIM
)
add_custom_command(
OUTPUT ${fw_package}
COMMAND
@@ -433,11 +458,14 @@ if (TARGET parameters_xml AND TARGET airframes_xml)
--git_identity ${PX4_SOURCE_DIR}
--parameter_xml ${PX4_BINARY_DIR}/parameters.xml
--airframe_xml ${PX4_BINARY_DIR}/airframes.xml
--image ${PX4_BINARY_DIR}/${PX4_CONFIG}.bin > ${fw_package}
--image ${PX4_BINARY_DIR}/${PX4_CONFIG}.bin
--manifest_json ${MANIFEST_JSON}
> ${fw_package}
DEPENDS
${PX4_BINARY_DIR}/${PX4_CONFIG}.bin
airframes_xml
parameters_xml
${MANIFEST_JSON}
COMMENT "Creating ${fw_package}"
WORKING_DIRECTORY ${PX4_BINARY_DIR}
)