summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2019-11-10OEQA: add crosstab selftestakuster/qa_changesArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-10OEQA: remove manual PACKAGECONFIG_FLAGS testsArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-10OEQA: move list-packageconfig-flags tests from manual to selfArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: remove manual useradd testArmin Kuster
useradd tested is done via: test_non_root_user_can_connect_via_ssh_without_password and test_non_root_user_can_connect_via_ssh_without_password Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: remove manual bash testArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: move manual bash test to runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: eclipse support was dropped in warriorArmin Kuster
remove test Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: remove postinit test done w/selftest runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07manual/bsp-qemu: remove only_one_connmand_in_background test done at runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07manual/bsp-qemu: drop xserver test done at runtimeArmin Kuster
xserver testing is done at runtime and selftest via sato image Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07manual qa/bsp-qemu: remove KVM enabled which is already done in selftest runqemuArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07manual qa/bsp-qemu: remove rpm tests already done in runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07manual qa: drop ltpstress testArmin Kuster
ltpstress was removed in Oct 2018 https://github.com/linux-test-project/ltp/commit/e752f7c19674d9b2f7d37aed123561a3f6410e97#diff-5231627fc8640e0adb955f9e69c3c08d Remove LTP stress tests ltpstress.sh runs stress.part[1-3]. But these runtest files just duplicate definitions: * stress.part1: fs, mm, nfs * stress.part2: ipc, math, nptl * stress.part3: net.multicast, pty, syscalls The definitions are outdated anyway. There is no point trying to keep them sync. Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA/manual: remove crash stress testArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA/runtime: Add ltp stress testArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: update ltp runtimes to use new structureArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: Centrilize the base LTP routinesArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: Add qemu checksArmin Kuster
Some test should not be run in QEMU systems so add some checks to make that easier Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07OEQA: Add a check for MACHINEArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07oeqa: add selftest for running yocto-check-layerArmin Kuster
This is so we can standardize what is being run as a baseline for folks who want to submit for Compatiblitity Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-07oeqa: reproducible: Add option to capture bad packagesJoshua Watt
Adds an option that can be used to copy the offending packages to a temp directory for later evaluation. This is useful on the Autobuilder to investigate failures. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-07selftest/signing: Fix test_locked_signatures to use a temporary layerRichard Purdie
Tests shouldn't be writing to layers during tests as this could corrupt other tests running in parallel. Modify the test to write the bbappend to a separate temporary layer which is added and removed by the test. This avoids race failures on the autobuilder. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-04selftest: add a test for gpl3-free imagesAlexander Kanavin
Existing tests check that adding a specific gpl3 package (bash) to core-image-minimal results in expected behaviour. These tests check the ability to build two common images without gpl3 components in them: 1. core-image-minimal needs no further tweaks and works out of the box. 2. core-image-full-cmdline requires dropping the GNU packages that it pulls in; for good measure this tweaked image is verified with runtime tests. These two tests allow dropping meta-gplv2 from being tested on the autobuilder, however there should be a community consensus first. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-11-04selftest: skip virgl test on centos 7 entirelyAlexander Kanavin
With the sdl frontend, qemu isn't able to even boot fully, so let's skip the test early. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-11-04oeqa/selftest: add test for oe-run-nativeRoss Burton
A small test to verify that oe-run-native is correctly working. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-10-23oeqa/runtime/context.py: ignore more files when loading controllersAndré Draszik
When loading controllers as (external) modules, the code currently tries to load all files ending with .py. This is a problem when during development using an editor that creates a lock-file in the same directory as the .py file, as the lock file is typically called '.#xxxx.py'. Python will try to load the lock file and fail miserably with an exception: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_testimage(d) 0003: File: 'poky/meta/classes/testimage.bbclass', lineno: 114, function: do_testimage 0110: netstat -an 0111:} 0112: 0113:python do_testimage() { *** 0114: testimage_main(d) 0115:} 0116: 0117:addtask testimage 0118:do_testimage[nostamp] = "1" File: 'poky/meta/classes/testimage.bbclass', lineno: 294, function: testimage_main 0290: 0291: # the robot dance 0292: target = OERuntimeTestContextExecutor.getTarget( 0293: d.getVar("TEST_TARGET"), logger, d.getVar("TEST_TARGET_IP"), *** 0294: d.getVar("TEST_SERVER_IP"), **target_kwargs) 0295: 0296: # test context 0297: tc = OERuntimeTestContext(td, logger, target, host_dumper, 0298: image_packages, extract_dir) File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 116, function: getTarget 0112: # XXX: Don't base your targets on this code it will be refactored 0113: # in the near future. 0114: # Custom target module loading 0115: target_modules_path = kwargs.get('target_modules_path', '') *** 0116: controller = OERuntimeTestContextExecutor.getControllerModule(target_type, target_modules_path) 0117: target = controller(logger, target_ip, server_ip, **kwargs) 0118: 0119: return target 0120: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 128, function: getControllerModule 0124: # ImportError raised if a provided module can not be imported. 0125: @staticmethod 0126: def getControllerModule(target, target_modules_path): 0127: controllerslist = OERuntimeTestContextExecutor._getControllerModulenames(target_modules_path) *** 0128: controller = OERuntimeTestContextExecutor._loadControllerFromName(target, controllerslist) 0129: return controller 0130: 0131: # Return a list of all python modules in lib/oeqa/controllers for each 0132: # layer in bbpath File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 163, function: _loadControllerFromName 0159: # Raise ImportError if a provided module can not be imported 0160: @staticmethod 0161: def _loadControllerFromName(target, modulenames): 0162: for name in modulenames: *** 0163: obj = OERuntimeTestContextExecutor._loadControllerFromModule(target, name) 0164: if obj: 0165: return obj 0166: raise AttributeError("Unable to load {0} from available modules: {1}".format(target, str(modulenames))) 0167: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 173, function: _loadControllerFromModule 0169: @staticmethod 0170: def _loadControllerFromModule(target, modulename): 0171: obj = None 0172: # import module, allowing it to raise import exception *** 0173: module = __import__(modulename, globals(), locals(), [target]) 0174: # look for target class in the module, catching any exceptions as it 0175: # is valid that a module may not have the target class. 0176: try: 0177: obj = getattr(module, target) Exception: ImportError: No module named 'oeqa.controllers.' Simply ignore those when collecting the list of files to try to load. Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23oeqa/selftest/recipetool: Use with to control file handle lifetimeOla x Nilsson
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23oeqa/selftest: use specialist assert* methodsRoss Burton
2019-10-19oeqa/runtime/systemd: skip unit enable/disable on read-only-rootfsAndré Draszik
This doesn't work on read-only-rootfs: AssertionError: 1 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl disable avahi-daemon.service Failed to disable unit: File /etc/systemd/system/multi-user.target.wants/avahi-daemon.service: Read-only file system This patch does two things: 1) Decorate the existing test to be skipped if the rootfs is read-only 2) add a new test to be executed only if the rootfs is read-only. This new test remounts the rootfs read-write before continuing to execute the existing test, making sure to clean up correctly after itself (remount r/o again). Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19oeqa/runtime/opkg: skip install on read-only-rootfsAndré Draszik
Images can have package management enabled, but be generally running as read-only. In this case, the test fails at the moment with various errors due to that. Use the new @skipIfFeature decorator to also skip this test in that case. Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19oeqa/core/decorator: add skipIfFeatureAndré Draszik
skipIfFeature will skip a test if a given DIST_FEATURE or IMAGE_FEATURE is enabled. Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19oeqa/runtime/df: don't fail on long device namesAndré Draszik
When device names are long (more than 20 characters), the df test will fail with an exception: self.assertTrue(int(output)>5120, msg=msg) ValueError: invalid literal for int() with base 10: '' at least when busybox is in use. The reason is that busybox breaks the line in that case: Filesystem 1K-blocks Used Available Use% Mounted on /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02 1998672 87024 1790408 5% / and the code tries to extract the fourth field from the second line, which is empty of course. df can be told not to break lines, though, using the -P flag, which turns on the POSIX output format, and is supported by busybox df and coreutils df: Filesystem 1024-blocks Used Available Capacity Mounted on /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02 1998672 87024 1790408 5% / Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-10license_image.bbclass: check and reject packages which have incompatible ↵Alexander Kanavin
licenses The use case is setting INCOMPATIBLE_LICENSE per image, rather than as an awkward, and too strict global setting. This for example would allow building development images with gplv3 tools, but production images without them, and checking that nothing gpl3-licensed gets into the latter. Examples are provided via the selftest: four scenarios are tested: - bash is added to the image, with a default gpl3 license; this is rejected - bash is added to the image, with a "gpl3 & other" license; this is also rejected - bash is added to the image, with a "gpl3 | other" license; this is accepted, but only 'other' is added to the license manifest (this was already handled correctly previously). - bash is added to the image with a default gpl3 license, and is additionally whitelisted for that image; this is accepted. Eventually, this would allow deprecating the meta-gplv2 layer, while still enforcing the no-gpl3 rule where possible and needed. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09oeqa/selftest/imagefeatures: improve test_hypervisor_fmtsRoss Burton
If this test fails then the output doesn't help in any meaningful way, so improve the test to output the unparsable JSON and display unexpected output. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08dnf.py: check busybox for case test_dnf_installrootKai Kang
Check package busybox which is required by cases test_dnf_installroot and test_dnf_installroot_usrmerge. Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08oeqa/sdk: improve Meson testRoss Burton
Verify that the build inside the SDK is detected as a cross compilation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27oeqa/core/case.py: Encode binary data of logNathan Rossi
Do not decode the log content into a string only to re-encode it as binary data again. Some logs might un-intentionally contain bytes that do not decode as utf-8, as such preserve the log file content as it was on disk. Handle the decoding on the resulttool side, but also handle the failure to decode the data. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27oeqa/core/utils/concurrencytest.py: Handle exceptions and detailsNathan Rossi
Handle the streaming of exception content with details data. The testtools package allows both 'err' and 'details' kwargs but can only pass one of them to the parent. To handle the passing of exception traceback and details data at the same time, encode the traceback into the details object and remove the 'err' arg from the add* result call. This encodes the traceback similar to how 'err' is handled without any details object. Decoding is already done by testtools when the traceback is encoded in the details object. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27oeqa/selftest/signing: Fix for hash equivlance serverRichard Purdie
There were two issues with the test one is that an equivalent hash could come from the server meaning the signature didn't change when it should. A uuid string is injected to ensure this does not happen. If there were multiple warnings the test would also fail as only the first is prefixed with WARNING. Tweak the string to avoid that failure mode. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27oeqa: Test multiconfig parsingJoshua Watt
Add a test to verify that when multiconfig conf files changed, recipes are correctly reparsed. [YOCTO #13541] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27cases/bbtests.py: test_bitbake_g(): Check base-files rather than busyboxRobert Yang
It checked whether busybox is in task-depends.dot after run "bitbake -g core-image-minimal", but busybox is not a must for core-image-minimal since it is configurable: VIRTUAL-RUNTIME_base-utils = "" VIRTUAL-RUNTIME_base-utils-syslog = "" VIRTUAL-RUNTIME_base-utils-hwclock = "" VIRTUAL-RUNTIME_login_manager = "shadow" VIRTUAL-RUNTIME_syslog = "" So the case may fail when busybox is not present, check base-files which is more reliable. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27devtool.py: fix buildclean testChangqing Li
when build with EXTERNALSRC, checksum of EXTERNALSRC is one of inputs for generate sigdata of task do_compile. without change any source, the sigdata will not changed, and buildclean task only do make clean, and not clean stampfiles like clean task, so the stampfile keep there, then do_compile after do_buildclean will not rerun, and targetfile tempdir_mdadm/mdadm will not be generated, report below error: AssertionError: '/tmp/devtoolqarxv8ct6u/mdadm' does not exist Fix by add a comments in Makefile to trigger rebuild of compile task Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27devtool.py: change to do clean before remove-layerChangqing Li
if do clean after remove-layer, it will make stamp file not cleaned since variable STAMP has changed, this will cause error like: if testcase buildclean failed, when run next testcase modify, since do_prepare_recipe_sysroot's stamp file exists, sysroot not populated, then do_compile will fail with error: /bin/sh: x86_64-poky-linux-gcc: command not found Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19oeqa/selftest/reproducible: test ipkgs tooRoss Burton
Now that opkg-build can build reproducible ipkgs, we can also add those to the test case. [ YOCTO #13513 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19oeqa/concurrenttest: Use ionice to delete build directoriesRichard Purdie
Autobuilder type infrastructure can benefit from deletion of certain files as background IO due to the way Linux filesystem priority works. We have problems where build directories as part of oe-selftest being delete starves the running tasks of IO to the point builds take much longer to compelte. Having this option of running the deletion at "idle" helps a lot with that. Use the new option added to bb.utils.prunedir(). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16oeqa/selftest/imagefeatures: dump the JSON if it can't be parsedRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-16oeqa/selftest/wic: improve assert messages in test_fixed_sizeRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-16runqemu: decouple gtk and gl optionsAlexander Kanavin
This will allow not having to multiply these options for the sdl frontend, instead combining them as needed. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15oeqa: add case for oe-git-proxyHenning Schild
The escaping, splitting and matching of NO_PROXY in oe-git-proxy deserves its own testcase, add it. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-11oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCaseNathan Rossi
Use OEPTestResultTestCase to collect results and add logfile collection and compression. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-04-26 23:41:46 +0000