83 Commits

Author SHA1 Message Date
Julian Oes
1e88939605 mavsdk_tests: report speed factor every second
This helps in debugging slow CI.
2021-04-06 11:35:18 -04:00
Julian Oes
2703a34900 mavsdk_tests: add link back to github action 2021-02-17 18:13:14 +01:00
Julian Oes
64d0ce43b6 mavsdk_tests: upload to logs.px4.io in test_runner
By doing the upload directly inside the mavsdk_test_runner.py we have
more context such as the model, case, etc.
2021-02-16 17:07:21 +01:00
Julian Oes
e7651c94f6 mavsdk_tests: use custom rootfs, reset it
Instead of messing with existing params, use a separate rootfs for
tests, as suggested by @bkueng.
2020-09-16 12:51:56 -04:00
Julian Oes
c26b769cc0 mavsdk_tests: add missing type annotation 2020-09-16 12:51:56 -04:00
Julian Oes
630a6f3810 mavsdk_tests: clear params between test cases 2020-09-16 12:51:56 -04:00
Julian Oes
a05ad9c64c mavsdk_tests: add wildcards for case filtering 2020-09-16 12:51:56 -04:00
Julian Oes
a1bf1bdd9d mavsdk_tests: prevent exception if log_fd is None 2020-05-29 11:46:14 +02:00
Julian Oes
eba1cf08f8 mavsdk_tests: also use sleep workaround locally
However, don't sleep as long as in CI.
2020-05-29 11:46:14 +02:00
Julian Oes
76750fc8a6 mavsdk_tests: collect buffered up stdout output
Somehow only subprocess.stdout.readline() works at a time. In order not
to miss out on some of the stdout output, we need to collect it all at
the end. Also, we can stop using readline() for processes that have quit
already.
2020-05-29 11:46:14 +02:00
kritz
33dc0fd146
Add vision velocity integration test (#14818) 2020-05-05 11:14:30 +02:00
Julian Oes
9764c32b55 mavsdk_tests: only apply workaround in GitHub CI 2020-04-08 16:56:52 +02:00
Julian Oes
54fe3c5fd5 mavsdk_tests: remove waitforgz helper, add sleep
This is an attempt to workaround the gz spawn timeout in CI.
Let's just hard-code a sleep before spawning the model.
2020-04-08 16:56:52 +02:00
Julian Oes
28f4dc10ae mavsdk_tests: make sure all log output is printed
This fixes the issue where the last lines of the log output was not
printed in case of error or on the verbose setting. This meant that
essentially the actual test error was not printed.

The fix involves two parts:
1. Firstly collect the output again even if a process has exited.
2. Collect all lines at once and not one line per iteration.
2020-04-03 11:45:35 +02:00
Julian Oes
0edc2aa334 mavsdk_tests: don't stall on timeout 2020-03-31 14:37:37 +02:00
Julian Oes
29c102b205 mavsdk_tests: wait for gz to run for model spawn 2020-03-23 10:55:13 +01:00
JaeyoungLim
b1d3b95ebf Add gzmdoelspawning to mavsdk tests 2020-03-23 10:55:13 +01:00
Julian Oes
6d654cb50b mavsdk_tests: improve usage text
These are improvements based on the review by @hamishwillee
in https://github.com/PX4/Devguide/pull/982.
2020-03-19 08:53:21 +01:00
Julian Oes
89599f345f mavsdk_tests: c++ test_runner is now mavsdk_tests 2020-03-18 17:57:41 +00:00
Julian Oes
4905ac5e2c mavsdk_tests: add filter for cases 2020-03-18 17:57:41 +00:00
Julian Oes
79387428c6 mavsdk_tests: fix missing space after number 2020-03-18 17:57:41 +00:00
Julian Oes
3f1990b083 mavsdk_tests: print where to find logfiles 2020-03-18 17:57:41 +00:00
Julian Oes
188177cef6 mavsdk_tests: print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes
ab2186f126 mavsdk_tests: improve and fix colors/result 2020-03-18 17:57:41 +00:00
Julian Oes
b1ceef5b93 mavsdk_tests: add combined log 2020-03-18 17:57:41 +00:00
Julian Oes
814d79cb32 mavsdk_tests: improve log file folder structure 2020-03-18 17:57:41 +00:00
Julian Oes
8283d19682 mavsdk_tests: print overall results 2020-03-18 17:57:41 +00:00
Julian Oes
a208129921 mavsdk_tests: satisfy mypy type checks 2020-03-18 17:57:41 +00:00
Julian Oes
12252c97c5 mavsdk_tests: further argument cleanup, some types 2020-03-18 17:57:41 +00:00
Julian Oes
ff64d87ae6 mavsdk_tests: import naming, easier coloring 2020-03-18 17:57:41 +00:00
Julian Oes
11a28840f3 mavsdk_tests: split up Python script
Otherwise, the script will get a tangled mess.
2020-03-18 17:57:41 +00:00
Julian Oes
b281d87b4a mavsdk_tests: initialize Tester class in ctor 2020-03-18 17:57:41 +00:00
Julian Oes
e2c80e546d mavsdk_tests: properly exit on Ctrl+C
With threads we need to manually take care of it, otherwise it gets
messy and we need to press Ctrl+C multiple times.
2020-03-18 17:57:41 +00:00
Julian Oes
2c2e314ffe mavsdk_tests: improve output, allow log and stdout
This makes it possible to write to logfiles and at the same time print
everything to console in verbose mode.
2020-03-18 17:57:41 +00:00
Julian Oes
b2e300ad1f mavsdk_tests: improve test output readability 2020-03-18 17:57:41 +00:00
Julian Oes
54645268ed mavsdk_tests: only print error on failure 2020-03-18 17:57:41 +00:00
Julian Oes
911cdc8774 mavsdk_tests: output less verbose unless needed 2020-03-18 17:57:41 +00:00
Julian Oes
bfef243dba mavsdk_tests: write output to logs folder
And create folder if it doesn't exist already.
2020-03-18 17:57:41 +00:00
Julian Oes
70eb444f17 mavsdk_tests: improve naming of argument
We abort when we already failed so this makes more sense.
2020-03-18 17:57:41 +00:00
Julian Oes
41b2362dcf mavsdk_tests: add more config into JSON 2020-03-18 17:57:41 +00:00
Julian Oes
b4916fdecb mavsdk_tests: move config into json file 2020-03-18 17:57:41 +00:00
Julian Oes
44a556717d mavsdk_tests: remove plane tests
We can put them back once it's supported.
2020-03-18 17:57:41 +00:00
Julian Oes
ae2032d29f mavsdk_tests: use the min of the speed factors
Facepalm.
2020-03-18 17:57:41 +00:00
Julian Oes
86f24f5121 mavsdk_tests: remove opt_flow test
It doesn't actually seem to work.
2020-03-18 17:57:41 +00:00
Julian Oes
b7e8837562 mavsdk_tests: add max speed factor
This is required because some tests don't work at more than 1x.
2020-03-18 17:57:41 +00:00
Julian Oes
d8b6c864cd mavsdk_tests: PEP8 fixes 2020-03-18 17:57:41 +00:00
Julian Kent
5bf2648c1f Add debugger options for mavsdk test runner 2020-02-25 17:15:40 +01:00
Lorenz Meier
eb50e89d87
Add flow tests to MAVSDK tests (#14039)
* workflows: remove unneeded pip dep and comments

* mavsdk_tests: simple test using flow/offboard

This adds a first proof of concept of a test using flow and offboard.

* Flow MAVSDK test: check against ground truth

* CleanUp

* workflows: use latest docker image with psutil

* mavsdk_tests: fix PEP8 issues

* Add VTOL CI tests back

* switch to non-rendering flow mockup

* workflows: install mavsdk from GitHub release .deb

* Add vision test and disable VTOL temporarlly

* Fist draft of combining test coverage and test in one workflow

* Add VTOL mavsdk tests back in CI

Co-authored-by: Julian Oes <julian@oes.ch>
Co-authored-by: kritz <kritz@ethz.ch>
2020-01-30 11:57:48 +01:00
Julian Oes
a2e95f171f mavsdk_tests: fix model name of plane
However, the test does not pass yet, so I'm leaving it commented out.
2020-01-28 20:35:25 +01:00
Julian Oes
bb1c08fdd1 mavsdk_tests: enable VTOL, fix TEST_CASE name
We can't use commas in the TEST_CASE name. With the comma, the test did
not run and displayed:

Filters: Takeoff, transition and RTL
No test cases matched 'Takeoff'
No test cases matched 'transition and RTL'
==========================================
No tests ran
2020-01-28 20:35:25 +01:00