summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest/context.py
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/selftest: Improvements to the json loggingRichard Purdie2018-10-291-14/+11
| | | | | | | | | | | Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: write testresult to json filesYeoh Ee Peng2018-10-291-3/+31
| | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the oe-selftest log directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest: Replace bitbake -p with bitbake -eRichard Purdie2018-08-151-2/+2
| | | | | | | | | | | Parsing all the recipes is annoying when trying to re-execute oe-selftest and also unnecessary as its really just a sanity check. When the tests were originally being developed the guard was useful but less so now. Replace it with bitbake -e which is fast and checks the basic configuration is valid. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: check if rm_work is enabledAnuj Mittal2018-07-301-0/+4
| | | | | | | | | | | rm_work if enabled leads to some tests failing that rely on artifacts being present. Check if rm_work.bbclass is included and show an error and exit if it is. Fixes [YOCTO #12694] Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* logging: use warning instead warnChen Qi2018-07-261-1/+1
| | | | | | | | | | | | | The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oeqa: Add selftest parallelisation supportRichard Purdie2018-07-161-2/+6
| | | | | | | | | | | | | | | | | This allows oe-selftest to take a -j option which specifies how much test parallelisation to use. Currently this is "module" based with each module being split and run in a separate build directory. Further splitting could be done but this seems a good compromise between test setup and parallelism. You need python-testtools and python-subunit installed to use this but only when the -j option is specified. See notes posted to the openedmbedded-architecture list for more details about the design choices here. Some of this functionality may make more sense in the oeqa core ultimately. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* context: Include a command line argument to run all except certain testsLeonardo Sandoval2017-08-111-3/+8
| | | | | | | | | | | | | | | | | | A new command line argument (-R, which is the oposite of current -r) that allows to run all test cases except the ones indicated through the command line. Some command line examples: * Run all except the distro test case: $ oe-selftest -R distrodata * Run all except the archiver test case and a single bblayers unit test $ oe-selftest -R archiver bblayers.BitbakeLayers.test_bitbakelayers_add_remove [YOCTO #11847] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oeqa/{core,selftest}: Add support to validate if a specified test case isn't ↵Aníbal Limón2017-07-301-2/+6
| | | | | | | | | | | | | | | found If some test module/case is specified to run and isn't found the OEQA framework didn't notice it, so complete the implementation using modules_required and validate for the test case prescense. Raise an exception when the test module/case required isn't found. [YOCTO #11645] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and SIGTERMAníbal Limón2017-07-211-31/+76
| | | | | | | | | | In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration. [YOCTO #11650] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
* oeqa/selftest/context: Reload testlayer_path when meta-selftest isn't addedAníbal Limón2017-06-231-0/+1
| | | | | | | | When add meta-selftest by the script the testlayer_path needs to be reloaded to avoid None value. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oeqa: Change the order to logDetails and logSummaryAníbal Limón2017-06-091-1/+1
| | | | | | | | | | Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* scripts/oe-selftest: Migrate to new framework into oeqa.selftest.contextLeonardo Sandoval2017-06-051-0/+224
The new OEQA framework aims to re-use code into the different Test components. The previous oe-selftest implements it-self loading, run, and list test cases in a non-standard way (unittest base) and other functionalities like logging that is now on oeqa core. This ends on a compact oe-selftest script. All needed command line options was migrated but there are some of them pending of implementation and others deprecated. Deprecated options: list-tags: The tag functionality into the old oeqa framework isn't work, the selftest doesn't has tag decorators. {run, list}-tests-by: Ambiguos options it accepts all the posibilites module, class, name, id or tag. Remaining to implement: coverage: It enables covrage reports over a test run, currently isn't on on use and some bugs [1], i filed a bug to add support to OEQA core module in this way other Test components could enable it. repository: It push XML results into a git repository and isn't in use, i filed a bug to implement this into OEQA core module. [2] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0 [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0 Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>