summaryrefslogtreecommitdiffstats
path: root/meta/lib
AgeCommit message (Collapse)Author
2021-07-08oeqa/selftest/recipetool: update socat version to fix failing downloadRoss Burton
If the recipetool tests are run with an empty DL_DIR the fetch of socat 1.7.3.0 fails as that URL doesn't exist anymore. Update the version to 1.7.4.1 to fix the test. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-07oeqa/selftest/multiprocesslauch: Fix test raceRichard Purdie
Having two possible failures in multiprocesslauch creates a race where one failure may occur and stop processes being lanuched meaning the second failure may not be seen. Rather than having periodic races appearing on the autobuilder, only have one failure, making the test much more deterministic. [YOCTO #13054] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/dump.py: Add support for QMP command argumentsSaul Wold
Need to ensure that the dump_dir is created correctly and available When command arguemnts are passed construct a filename if needed and convert the arguements to a json object to pass to QMP. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/qemurunner: add support qmp cmd argsSaul Wold
This will enable passing arguments to qmp commands for those that require additional information Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/selftest/archiver: Allow tests to ignore empty directoriesRichard Purdie
If we tweak sstate to not remove empty directories under conditions where a race could occur, we see failures from: "oe-selftest -r archiver.Archiver.test_archiver_filters_by_type archiver.Archiver.test_archiver_filters_by_type_and_name" since an empty directory is left behind. Update the tests to ignore empty directories. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29lib/oe/package_manager: Don't ignore installation failures in ↵Fabio Berton
install_complementary When attempt_only flag is set to True all installation failures are ignored. Disable attempt_only, that is the default in install function, to raise error in install_complementary and to prevent do_populate_sdk task to treat errors as warnings. [RP: Historically some of our addtional packages were in bad shape and wouldn't all install. This is no longer the case so we shouldn't need a hack like this any longer.] Signed-off-by: Fabio Berton <fabio.berton@criticaltechworks.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29oeqa/selftest/runcmd: Tweal test timeoutsRichard Purdie
Load on the autobuilder meant we see occasionaly timeout issues with these tests. Slightly increase the test timeouts to better reflect the real world timings we see. [YOCTO #14262] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27qemurunner: Add info log for qemu startupSaul Wold
Add a couple of logging info to track time between activities, first is from after the Popen(launch_cmd) to after qmp.connect(), second is from qmp.connect() to the release of the qemu via the qmp("cont") command this includes the mmap() activity. Example output: QMP connected to QEMU at 06/24/21 11:11:56 and took 0.9556229114532471 seconds from launch QMP released QEMU at 06/24/21 11:11:56 and took 0.26789021492004395 seconds from connect Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27selftest/fetch: Avoid occasional selftest failure from poor temp file name ↵Richard Purdie
choice The temp file name may contain "_" characters. Switch to a temporary directory and a fixed filename to avoid this to avoid errors like: bb.data_smart.ExpansionError: Failure expanding variable PN, expression was ${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'} which triggered exception ParseError: ParseError in /tmp/tmpd_f2__to.bb: Unable to generate default variables from filename (too many underscores) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-22qemurunner: add second qmp portSaul Wold
This will allow for direct ssh connection without breaking the first one that is used for monitoring. The "nowait" option will cause qmp server connection to NOT block waiting. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-21selftest: do not hardcode /tmp/sdkAlexander Kanavin
This races if there are several copies of the test running at the same time. [YOCTO #14438] (From OE-Core rev: deab11848036941771f2b3dc5cdaee83395280b5) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2021-06-12oeqa/core/target/qemu.py: display contents of dumped filesSakib Sajal
During do_testimage, if the target is not started within a certain timeout, TEST_QEMUBOOT_TIMEOUT, host data is dumped to files for each command in ${TMPDIR}/log/runtime-hostdump/<datetime>_qemu/host_<seq>_<command>. Display the first 20 lines of top output and the last 20 lines of bootlog to standard output for more context for the target not being started up. Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-11qemurunner: Increase startup timeout 120 -> 300Richard Purdie
We now spend time copying the VM image into a tmpfs and with IO load on the system, the time + the boot time of the VM can take longer than 120s. Increase the timeout to match the added overhead of copying the image file. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-11qemurunner: Try to ensure mmap'd libs are paged inRichard Purdie
We've seeing issues where IO load appears to cause strange failures due to timeouts within qemu. One theory for these is that it is is hitting hard page faults at in-opportune moments which cause timing problems within the VM. This patch is a bit of a hack which tries to ensure the data is paged in at a point when we know we can take the time delays (waiting for the QMP start signal). Whilst this isn't ideal, it does seem to improve things on the autobuilder and shouldn't harm anything. The code figures out which files to read my looking at the mmap'd files the process has open from /proc. On Centos7 systems these files are not user readable, if that is the case we just skip them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-03oeqa: remove Clutter usageRoss Burton
Remove the use of core-image-clutter in selftest and manual Toaster QA. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-03classes/buildhistory: Add option to strip path prefixJoshua Watt
Adds an option to strip a prefix from the paths reported in buildhistory. This makes it easier to compare task signatures in the build history when the builds were done from different directories. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-01package_manager/rpm: decode systemctl failuresArmin Kuster
Parse systemctl error to provide more info [Yocto #14395] Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-01oeqa/qemurunner: Improve timeout handlingRichard Purdie
We had debugging for qemu faiing to start which was no longer reachable after the QMP changes. Reorder the code to enable this debugging to work again which may allow insight into autobuilder failures in this area. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-28oeqa/runtime/rpm: Drop log message counting test componentRichard Purdie
This test is flawed since multiple parts of the system can write to the log and we obtain different numbers of log messages depending on factors we can't control. Drop the log testing component of the test. [YOCTO #12465] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-21sstatesig.py: make it fatal error when sstate manifest isn't foundMartin Jansa
* all known issues in this area were fixed, make it fatal that it cannot be overlooked if someone triggers this issue again Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-21lib/oe/gpg_sign.py: Fix gpg verificationDaniel McGregor
A stray space made it into the command for verifying gpg signatures. This caused verification to fail, at least on my host. Removing the space makes it work as expected. Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-13meta/lib/oe/rootfs.py: Fix typo "Restoreing" -> "Restoring"Robert P. J. Day
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-09oeqa/qemurunner: Improve handling of run_serial for shutdown commandsRichard Purdie
When running a shutdown command, the serial port can close without the command returning. This is seen as the socket being readable but having no data. Change the way this case is handled in the code to avoid tracebacks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-09oeqa/qemurunner: Fix binary vs str issueRichard Purdie
The recent logging changes for qemurunner showed up as errors on the autobuilder where decode couldn't be called on the returned string. Since the code returns binary data, return b'' instead of '' to match to avoid tracebacks. One of these cases was newly added, copied from the other which has been there for a long time, always broken. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06qemurunner: change warning to infoSaul Wold
This information is useful, but should not be a warning level. [YOCTO #14382] Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06lib/package_manager: Use shutil.copy instead of bb.utils.copyfile for interceptsRichard Purdie
If the scripts/postinst-intercepts is owned by root/root then the copyfile() calls will fail due to chown issues. We don't care about ownership of these files so use shutil.copy() instead which won't perform any chown. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06oeqa/qemurunner: Handle path length issues for qmp socketRichard Purdie
After the addition of the qmp socket, runqemu started failing: ERROR - Failed to run qemu: qemu-system-aarch64: -qmp unix:/home/yocto/actions-runner-meta-openembedded/_work/meta-openembedded/meta-openembedded/yoe/build/tmp/.3eg5fiid,server,wait: UNIX socket path '/home/yocto/actions-runner-meta-openembedded/_work/meta-openembedded/meta-openembedded/yoe/build/tmp/.3eg5fiid' is too long Path must be less than 108 bytes To avoid this, run qemu within tmpdir and use a relative path to the socket. This avoids having to patch the socket code within qemu. Update the client code to chdir and only use a relative path to the socket to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown testRichard Purdie
Rather than totally disabling the logging, inform it we're about to exit so we can log messages over the exit cleanly too. This aids debugging. It also avoids a race where the logging handler could still error whilst shutting down. Also remove a race window by notificing the handler of the shutdown first, before triggering it. This removes a race window I watched in local testing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06classes/lib/scripts: Use bb.utils.rename() instead of os.rename()Devendra Tewari
Incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than adding fallback code to every call site, use a new wrapper in bitbake which detects this case and falls back to shutil.move which is slower but will handtle the overlay docker filesystems correctly. [YOCTO #14301] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06Revert "oeqa: Set LD_LIBRARY_PATH when executing native commands"Alexander Kanavin
LD_LIBRARY_PATH leaks into host executables too, and breaks them as they are not uninative-enabled. E.g. on ubuntu 18.04 trying to run host bash with a sysroot that was built on Fedora 33: akanavin@ubuntu1804-ty-3:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-24341/tmp/work/x86_64-linux/gnupg-native/2.3.1-r0/recipe-sysroot-native$ LD_LIBRARY_PATH=./usr/lib /bin/bash /bin/bash: ./usr/lib/libtinfo.so.5: no version information available (required by /bin/bash) /bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./usr/lib/libtinfo.so.5) This was seen e.g. here: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2090/steps/14/logs/stdio Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-02rootfs.py: find .ko.gz and .ko.xz kernel modules as wellChristophe Chapuis
* with xz PACKAGECONFIG enabled in kmod and xz module compression enabled in kernel the do_rootfs task doesn't run depmod in the image, because it thinks there are no modules: NOTE: No Kernel Modules found, not running depmod Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Christophe Chapuis <chris.chapuis@gmail.com> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-01prservice: Use new connect APIPaul Barker
The new prserv connect() function decouples the code in oe-core from the exact classes and implementation details used within bitbake. This allows us to more easily switch over to a new asyncrpc based prservice. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27oeqa/selftest: transition to weston imagesAlexander Kanavin
For readonly rootfs tests core-image-weston is appended; everywhere else it replaces core-image-sato. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27oeqa/core/tests/test_data.py: use weston image instead of satoAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27qemurunner: Add support for qmp commandsSaul Wold
This adds support for the Qemu Machine Protocol [0] extending the current dump process for Host and Target. The commands are added in the testimage.bbclass. Currently, we setup qemu to stall until qmp gets connected and sends the initialization and continue commands, this works correctly. If the UNIX Socket does not exist, we wait an timeout to ensure to socket file is created. With this version, the monitor_dumper is created in OEQemuTarget but then set in OESSHTarget as that's where we get the SSH failure happens. Python's @property is used to create a setter/getter type of setup in OESSHTarget to get overridden by OEQemuTarget. By default the data is currently dumped to files for each command in TMPDIR/log/runtime-hostdump/<date>_qmp/unknown_<seq>_qemu_monitor as this is the naming convenstion in the dump.py code. We use the qmp.py from qemu, which needs to get installed in the recipe-sysroot-native of the target image. [0] https://github.com/qemu/qemu/blob/master/docs/interop/qmp-spec.txt Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23meta/lib/oeqa/core/tests/cases/timeout.py: add a testcase for the previous fixAlexander Kanavin
This is the sequence that didn't properly operate: - a test case that skips and isn't executed - a second test case that is skipped via a dependency decorator, and sets a timeout - a third test case that takes longer than the timeout from the second test case Without the fix, the timeout is not cleared, and the third test case is erroneously aborted. With the fix, the timeout is cleared and the third test case is able to complete. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa: tear down oeqa decorators if one of them raises an exception in setupAlexander Kanavin
Some of the decorators need proper cleanup, such as OETimeout which sets a signal handler that needs to be cleared via teardown. If this is not done then the signal gets called later with unpredictable effects. This can be seen if there's a test that is skipped via a decorator and sets a timeout at the same time: the timeout isn't cleared, and is invoked later in a completely unrelated context. The test case for this is added in the next commit. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/selftest/bblayers: Add test case for bitbake-layers layerindex-show-dependsDiego Sueiro
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json : remove ↵Teoh Jay Shen
Test_if_usb_hid_device_works_well_after_resume_from_suspend_state manual test Remove the Test_if_usb_hid_device_works_well_after_resume_from_suspend_state test as it was replaced by the new automated runtime oeqa/runtime/cases/usb_hid.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json :remove ↵Teoh Jay Shen
Check_if_RTC_(Real_Time_Clock)_can_work_correctly manual test Remove the Check_if_RTC_(Real_Time_Clock)_can_work_correctly test as it was replaced by the new automated runtime oeqa/runtime/cases/rtc.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json : remove click_terminal_icon_on_X_desktop manual testTeoh Jay Shen
Remove the click_terminal_icon_on_X_desktop test as it was replaced by the new automated runtime oeqa/runtime/cases/terminal.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json : remove standby and ↵Teoh Jay Shen
Test_if_LAN_device_works_well_after_resume_from_suspend_state manual test Remove standby and Test_if_LAN_device_works_well_after_resume_from_suspend_state test as they was replaced by the new automated runtime oeqa/runtime/cases/suspend.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json : remove ethernet_static_ip_set_in_connman and ↵Teoh Jay Shen
ethernet_get_IP_in_connman_via_DHCP manual test Remove ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP test as they was replaced by the new automated runtime oeqa/runtime/cases/ethernet_ip_connman.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/manual/bsp-hw.json : remove boot_from_runlevel_3 and ↵Teoh Jay Shen
boot_from_runlevel_5 manual test Remove boot_from_runlevel_3 and boot_from_runlevel_5 test as they was replaced by the new automated runtime oeqa/runtime/cases/runlevel.py. Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18oeqa/runtime/cases: Only disable/enable for current bootWes Lindauer
Previously doing a stop/start worked, but using a disable/enable does not work on a read-only rootfs. Add a --runtime flag to systemctl so that systemd only modifies the current boot and does not attempt to write to the filesystem. This also keeps the test from making a permanent (one could argue policy) change to the running system being tested. i.e. What if the image being tested had intentionally disabled the timesyncd service in preference to using chrony or ntpd? The test shouldn't assume that the user wants the timesyncd service enabled. Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18parselogs: ignore floppy error on qemu-system-x86 at boot stageYanfei Xu
We can disable floppy drive by BIOS on a hardware, but an empty floppy drive is connected by default on qemu-system-x86. Linux usually detect the device and modprode the matched floppy.ko at the boot stage. Due to we don't specify a floppy deivce in qemu boot arguments, then the errors about floppy reading comes out. It is harmless and normal, so we could ignore this error message on qemux86. Seen if kernel-modules is included in the image which pulls in the relavent kernel module. https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg01402.html Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18oeqa/runtime: space neededJon Mason
Messages are currently being printed as: Test requires dropbear, oropenssh-sshd to be installed but should be Test requires dropbear, or openssh-sshd to be installed Adding the space after the 'or' corrects this. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18deb: apply postinstall on sdkhongxu
If not postinstall applied, some nativesdk command could not be found in sdk due to update-alternatives in postinst not be executed, such as chroot: $ which chroot /sbin/chroot $ which chroot.coreutils path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils After applying the fix $ which chroot path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot $ which chroot.coreutils path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18oeqa/selftest: Ensure packages classes are set correctly for maintainers testRichard Purdie
The dnf packages aren't parsed if rpm isn't in PACKAGE_CLASSES which means the aintainers test failes for OE-Core (where ipk is the default) but not for poky (where the default is rpm). Ensure PACKAGE_CLASSES is set so it works in all cases. [YOCTO #14277] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18lib/oe/terminal: Fix tmux new-session on older tmux versions (<1.9)Peter Budny
`tmux new -c` fails on tmux older than 1.9, when that flag was added. We can omit the flag for older versions of tmux, and the working directory gets set even without it. Signed-off-by: Peter Budny <pbbudny@amazon.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>