305 Commits

Author SHA1 Message Date
Beat Küng
4c73ac3805 uorb: use single byte for internal types in o_fields metadata
Reduces flash usage by ~9KB.
2021-10-20 08:10:05 +02:00
Jukka Laitinen
9a4ef709ca Add parameters for configuring the logfile encryption
SDLOG_ALGORITHM for selecting the crypto algorithm
SDLOG_KEY for selecting the key in keystore to use with the algorithm.
  for symmetric algorithms which are currently supported, this is just a free
  slot to be used, the key is generated at logging start
SDLOG_EXCH_KEY for selecting an RSA2048 key for encrypting the SDLOG_KEY

Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
2021-09-27 17:45:07 +02:00
Jukka Laitinen
9472f79307 Take crypto interfaces into use in logger, to encrypt ulog data
- Generate a symmetric encryption key and a nonce value
- Use a public key in keystore to encrypt the symmteric key
- Write the encrypted key and the nonce value to disk into .ulgk, name matching with the encrypted log file
- use quick stream encryption to crypt the .ulg file contents

Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
2021-09-27 17:45:07 +02:00
Beat Küng
0a1ae37c1a logger: switch to events 2021-09-11 14:08:41 -04:00
Beat Küng
38eca0c64b logger: start session & log index at 100
To avoid leading zeros, as the events interface does not support that (yet)
2021-09-11 14:08:41 -04:00
Jukka Laitinen
733b1e9e67 Increase logger stack size for 64 bit targets
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
2021-08-24 10:16:27 -04:00
Beat Küng
c0285611ec fix logger: turn off lockstep when starting log streaming
Lockstep created deadlocks between logger and mavlink, due to logger
waiting for mavlink and mavlink waiting for lockstep.
2021-08-12 10:09:11 -04:00
Beat Küng
f88f224fe6 logger: avoid data bursts by distributing slow subscription updates over time
There's an increasing amount of slow logged topics at 1-2Hz, which were all
updated in the same logger iteration, leading to data bursts. For log
streaming this started to exceed uart buffer sizes. By distributing updates
more equal over time those bursts are removed, reducing buffer size
requirements.

Tests showed during steady state a reduction of maximum topic updates per
iteration from 40 down to 17.
Also the SD log buffer fill level is more constant.
2021-08-12 10:09:11 -04:00
Beat Küng
e1ac6fe297 logger: send post-flight perf data when stopping mavlink log streaming
- send out VEHICLE_CMD_RESULT_IN_PROGRESS
- delete the ulog object when receiving the ACK from logger, instead of
  the stop command
2021-08-12 10:09:11 -04:00
TSC21
e39e3b3418 events interface: chance uORB 'sequence' to 'event_sequence'
The reason is that sequence is an IDL reserved keyword, which results on failure when parsing with FastRTPSGen.
2021-07-09 13:24:13 +02:00
Beat Küng
38f3b8d356 mavlink & system: add events interface
- sending protocol
- uorb event message & template methods for argument packing
- libevents submodule to send common events and handle json files
- cmake maintains a list of all (PX4) source files for the current build
  (PX4 modules + libs), which is used to extract event metadata and
  generate a json file
2021-07-07 21:38:09 -04:00
David Sidrane
5faa116681 logger:Use inttypes 2021-06-16 17:07:47 +02:00
Beat Küng
c3985709e4 version: add optional oem version tag (ext/oem-<version>) and log it 2021-03-24 10:08:41 -04:00
Beat Küng
af8c3215f2 logger: add airframe and system-wide default parameters
- adds new ulog message & compatibility flag
- separately add airframe and system-wide defaults
- log only non-volatile defaults that are different from the current value
- additional size is ~3KB for 100 params
2021-03-12 13:44:10 +01:00
FengShun
eac9a6b68b Replace PublicationQueued with Publication to automatically configure ORB_QUEUE_LENGTH 2020-10-26 13:01:13 +01:00
Daniel Agar
023f6d3983
NuttX cpuload monitoring optimization
- Nuttx only process all suspend & resume scheduling notes when top is running, otherwise only track IDLE
 - convert cpuload and print load to c++
 - delete unused fields from print_load struct
 - update hrt_store_absolute_time (previous unused)
2020-09-05 14:35:50 -04:00
Daniel Agar
7b46efaa6b logger: record message gaps 2020-07-29 13:36:22 -04:00
Beat Küng
fe9a971f0e logger: use subscription interval also in polling mode
Now that replay uses lockstep scheduling we don't need that work-around
anymore.
2020-06-23 11:53:55 -04:00
Beat Küng
a3d1c94aa4 logger: reduce poll timeout to 20ms
Just to ensure even if the topic is not updated, logger will still update
regularly.
2020-06-23 11:53:55 -04:00
Beat Küng
71dcf8d619 sitl: use lockstep components API
- avoids the need for ekf2_timestamp publications by q and lpe
- adds logger to the lockstep cycle and makes it poll on ekf2_timestamps
  or vehicle_attitude. This avoids dropped samples (required for replay).
2020-06-23 11:53:55 -04:00
Matthias Grob
e9eae1bd76 Refactor: Name manual_control_setpoint the same way everywhere 2020-06-22 12:06:27 -04:00
Beat Küng
0fcb873636 logger: add raw FIFO high-rate IMU logging profiles
Minimum requirement to use: set IMU_GYRO_RATEMAX to 400.

Logging rate of a single topic: ~85 KB/s. If multiple should be logged,
a really good SD card has to be used.
2020-06-09 09:58:15 -04:00
Daniel Agar
2a2d9e9eba logger: keep typical log path within a single mavlink message 2020-05-06 08:31:40 +02:00
Daniel Agar
466b5db36f
uORB::Publication improvements and cleanup (#14784)
- create common uORB::PublicationBase
 - uORB::PublicationQueued types are now type aliases
 - ORB_PRIO use enum type everywhere to avoid accidental misuse
 - PX4Accelerometer/PX4Gyroscope/etc driver libs explicitly advertise on construction, unadvertise on destruction. This is a workaround for any potential issues that might appear when accel/gyro cdev and uORB indexing doesn't align.
2020-05-04 11:09:30 -04:00
Beat Küng
4e0441ab0b logger: add data format version tag 2020-04-07 09:59:12 -04:00
Beat Küng
40af5b0fbe logger: fix initial subscribing to topics
write_all_add_logged_msg() requires at least some subscriptions to be added
initially, as this message is used to separate ULog header from data.
2020-03-20 11:21:32 +01:00
BazookaJoe1900
5e343b104e
logger: using ModuleParams (#14316)
and some cleanups on main()
2020-03-13 11:31:43 +01:00
Daniel Agar
9585055e9e
uORB: add bitset for faster orb_exists check and remove uORB::Subscription lazy subscribe hack/optimization
- add PX4 bitset and atomic_bitset with testing
 - add uORB::Subscription constructor to take ORB_ID enum
 - move orb test messages into msg/
2020-03-11 09:06:33 -04:00
bazooka joe
887d846f7a remove one space from string sent with the logger filename
now the full file name is sent, and not missing the last seconds digit
2020-03-08 12:07:51 +01:00
Daniel Agar
2a0fe169e6 logger: disable logger_status publiation in replay builds
- this is disabled in replay builds to ensure all data in ekf2 replay logs only contains the same time range, otherwise the plots can be unreadable using common tools
 - fixes #14230
2020-02-26 18:44:20 +11:00
Daniel Agar
2015cc2b39 logger: restore buffer statistics update
- this was accidentally dropped during a rebase of #14050
2020-02-10 00:17:28 -05:00
Daniel Agar
931a3f2684
logger: publish initial logger_status orb message 2020-01-29 17:29:30 -05:00
tzai
fb0bf26787 logger: refactor start_stop_logging
Cleans up function by reducing duplicate code and local variable count

Signed-off-by: Tal Zaitsev <tal@corvus-robotics.com>
2020-01-13 13:19:47 +05:30
tzai
b9d85c7176 logger: Refactor new boot_bat logic into separate function
Implements suggested fixes.

Signed-off-by: Tal Zaitsev <tal@corvus-robotics.com>
2020-01-11 22:27:00 +05:30
tzai
1c2f95697c logger: Add support for logging only on bat power
Will not start logging from boot if battery is not connected

Signed-off-by: Tal Zaitsev <tal@corvus-robotics.com>
2020-01-11 22:27:00 +05:30
Beat Küng
e10a4c5002 refactor logger: move subscription initialization into separate class 2019-12-17 21:22:30 -08:00
Beat Küng
738ceab0ee Logger: dynamic number of subscriptions
Keep a fixed-size array of 250 requested topics on the stack, then allocate
an array with LoggerSubscription with the exact required size.
2019-12-17 21:22:30 -08:00
Beat Küng
2757647897 refactor logger: reduce required size of WrittenFormats
by keeping only recursive topics in there. The rest is checked by looking
at previous _subscriptions.

Reduces stack size requirements when increasing MAX_TOPICS_NUM.
2019-12-17 21:22:30 -08:00
JaeyoungLim
94e05362e1 Make cellular status logging default 2019-12-14 21:56:12 +11:00
kamilritz
b73c80428e ECL: Clean velPos logging, deprecate ekf2_innovations msg 2019-12-05 11:29:29 +01:00
Daniel Agar
1a395fb6d9 ST ISM330DLC IMU driver 2019-12-03 16:16:49 -05:00
JaeyoungLim
6bd4273b9c mavlink: handle cellular_status messages for logging 2019-11-13 09:19:02 -05:00
Beat Küng
3198610f85 src/platforms: move all headers to platforms/common/include/px4_platform_common
and remove the px4_ prefix, except for px4_config.h.

command to update includes:
for k in app.h atomic.h cli.h console_buffer.h defines.h getopt.h i2c.h init.h log.h micro_hal.h module.h module_params.h param.h param_macros.h posix.h sem.h sem.hpp shmem.h shutdown.h tasks.h time.h workqueue.h; do for i in $(grep -rl 'include <px4_'$k src platforms boards); do sed -i 's/#include <px4_'$k'/#include <px4_platform_common\/'$k/ $i; done; done
for in $(grep -rl 'include <px4_config.h' src platforms boards); do sed -i 's/#include <px4_config.h/#include <px4_platform_common\/px4_config.h'/ $i; done

Transitional headers for submodules are added (px4_{defines,log,time}.h)
2019-10-30 11:48:47 +01:00
Daniel Agar
a7f330075a clang-tidy: enable hicpp-braces-around-statements and fix 2019-10-28 10:50:31 -04:00
Jaeyoung-Lim
29987a4f80 Address comments 2019-10-27 11:47:10 +00:00
Jaeyoung-Lim
58d646e33d Log CPU loads 2019-10-27 11:47:10 +00:00
Daniel Agar
c8e59c4e39 parameter_update use uORB::Subscription consistently 2019-09-29 10:49:03 -04:00
kritz
7427768e70 ECL reference frame alignment fix (#12771)
* Fix EKF frame alignemen in ECL

* Remove empty lines

* Add initalization for ev_odom

* Only use yaw covariance for angErr

* Improve frame naming in comments

* Use copyTo

* Add aligned as suffix

* Add missing vehicle_visual_odometry_aligned
2019-09-23 14:24:52 -04:00
Silvan Fuhrer
ff6577ce5f EKF and AirspeedSelector: publish multiple wind estimate topic instances and log them all
Signed-off-by: Silvan Fuhrer <silvan@auterion.com>
2019-09-01 14:05:06 -04:00
Beat Küng
7f6a9e587f logger: fix multiple format definitions for multi-instance topics
With https://github.com/PX4/Firmware/pull/12123 all multi-instance topics
lead to repeated format definitions in the ULog file.
2019-08-21 10:47:31 -04:00