Replaces the legacy pure-Python lookup table CRC32 implementation with the built-in `zlib.crc32`.
The previous implementation relied on a manual loop over bytes, which was inefficient for large firmware files (taking ~0.5s for 2MB on modern CPUs). The new implementation reduces this to ~1ms.
Implementation details:
- Removed the hardcoded `crctab` array to clean up the code.
- Adjusted `zlib` initial state (0xFFFFFFFF) and final XOR operations to ensure bit-perfect compatibility with the specific CRC32 variant expected by the PX4 bootloader.
Benchmark (2MB firmware):
- Legacy: ~0.48s
- zlib: ~0.001s
When the px_uploader.py is used scripted, so without an interactive
shell, the progress bar doesn't update. Therefore, I suggest to print a
new line instead of a carriage return for the non-interactive shell
case.
Includes:
- Remove some of the outdated Python2 checks and compatibility.
- Try not catch all exceptions but only the expected ones. Otherwise,
this makes it really hard to debug if anything unexpected actually
goes wrong.
- Make use of fstrings.
- Make output slightly prettier.
Signed-off-by: Julian Oes <julian@oes.ch>
This adds a new protocol extension which allows to get the bootloader
version.
The bootloader version is different from the bootloader protocol
revision which has stabilized at 5 and is not easy to update unless a
bootloader is actually breaking the protocol. The reason being that both
the Python script as well as the uploader used in QGC will not attempt
to load firmware if they don't know the bootloader version, so it could
basically be considered a "breaking" protocol revision.
Signed-off-by: Julian Oes <julian@oes.ch>
If the STM32H7 fails to program or erase a full chunk of 256 bytes, the
ECC check will trigger a busfault when trying to read from it.
To speed up erasing and optimize wear, we read before erasing to check
if it actually needs erasing. That's when a busfault happens and the
erase time outs.
The workaround is to add an option to do a full erase without check.
Credit goes to:
https://github.com/ArduPilot/ardupilot/pull/22090
And the protocol option added to the bootloader is the same as for
ArduPilot, so compatible.
Signed-off-by: Julian Oes <julian@oes.ch>
The windowed mode has been added for Windows targets originally. It gets very often incorrectly detected and slows down flashing considerably. This even applies to serial links. We are disabling it now in most circumstances.
The 9 seconds to erase a board probably still come from the FMU-v1 and
Pixhawks with only 1 MB flash. By now, many targets have 2 MB flash and
take a bit longer to erase. Therefore, we can increase the estimated
time a bit and don't need to resort to the timeout notice.
Before, the pyserial check would fail before it could check for the
VERSION that the Python 2.7 version contains. This fixes it to check
for the VERSION independently.
This is a workaround for the write timeout that we have seen for some
host computers trying to flash the firmware.
We don't know the root cause of the problem but we do observed the
following:
- For blocking writes with timeout (Pyserial write_timeout=0.5):
write() throws SerialTimeoutException. In systrace we see that the
select() call after write waiting for the write to be finished hangs
and finally times out.
- For blocking writes without timeout (Pyserial write_timeout=None):
write() hangs indefinitely. In systrace we see that the
select() call after write waiting for the write to be finished hangs.
- For non-blocking writes:
write() works but flush() hangs. In systrace we see that
ioctl(fd, TCSBRK, 1) which is (correctly) triggered by termios tcdrain
hangs.
Inspecting USB traffic using usbmon, we can see that the data which is
written actually seems to be sent and looking at responses from the
Pixhawk bootloader and the timings it looks like all the data has
arrived.
This workaround uses non-blocking writes without flushing and this
seemed to prevent the issue from happening so far.
Debugging was done in collaboration with Beat Küng and David Sidrane.
If we dont explicitly check for pyserial, we can have the case where the
import works but the Serial object creation fails. However, we don't see
this because we have this huge try/catch block which swallows
everything.
to fix Cygwin upload. It failed silently but when catching it prints
"non-standard baudrates are not supported on this platform".
Discussion about platform independet FTDI detection is in issue #10429.
Using the port name is platform dependent. It may give not
provide the correct feedback. The port can be ttyS, or ftdi
or CDC/ACM. While it is true that buadrate does not matter on
CDC/ACM. It is better to give more information about what the
code is doing before filtering by the port name.
The correct determination of Windowed mode is critical to
maximise the speed on a USB based upload.
This commit bases the detection of a Serial (FTDI) on the
fact a CDC ACM port (USB) does not really have a baud rate.
We bump the baud rate to 233% of the requested baud rate to
see if the SYNC is acked. If it is Acked it must be a USB
port and Windowed mode is turned off. This removes unnessary
baud rate based delays from the proframing logic. If it is a
real Serial port getSync will not get the ACK and Windowed
mode is turned on.
The __getSync was costing about 16Ms per call.
The commit uses a window based approch allowing
the SYNC,<results> to be read all at one time.
and delaying for programing based on transport
time + 1 Ms;
THe improvment at 2Mbps is >4 minutes to ~37
seconds
This fixes a problem where the pyserial write call gets stuck.
It happens on a specific Fedora 28 system with internal USB ports as
well as USB hubs.
It is not clear why the problem is resolved but it is clearly
reproducible that with a timeout of 0, the write can get stuck and with
a timeout > 0 it works every time.
The exception added as part of this commit makes sense but has never
been triggered in my testing.
If the PX4FMUv2 board does not have the v5 uploader, it's not possible
to check the silicon revision to make sure it's safe to upload the
firmware which is more than 1 MB. However, if the user is sure the
silicon revision is not affected by this errata, he can upload the
firmware using px4_uploader.py with --force argument. This commit adds
`force-upload` make target to do it more convenient way.
Signed-off-by: Andrei Korigodski <akorigod@gmail.com>
In the Cygwin environment the native Windows serial COM# ports get mapped
to /dev/ttyS# for POSIX compatibility. While # is one number lower inside
the environment than the COM port number because it's 0 indexed instead
of 1.
I added the necessary handling to all the dependent parts I found which
allows uploading to /dev/ttyS# when the cygwin platform is detected.
Now the usual "make px4fmu-v4 upload" and
"./Tools/upload.sh build/px4fmu-v4_default/px4fmu-v4_default.px4" work.
- Move check to proper location, out of the try catch block for OTP.
- Add Pixhawk specific check to notify users that want to flash
px4fmu-v3_default on Pixhawks with older v4 bootloaders that do not
support the silicon errata check.
This fixes the error below when using Python3:
File "Tools/px_uploader.py", line 128, in
__init__
self.image.append('\xff')
TypeError: an integer is required
