Age | Commit message (Collapse) | Author |
|
The previous version only included the numbers 1 and 2 in the allowed characters
for the module name. In the past, this was (\w+) so all numbers were allowed.
Now it explicitly includes all numbers again.
Signed-off-by: Patrick Vacek <patrickvacek@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
test case in format <module name>.<class name>.<test case name>
this is clear when test cases is only 3 item deep.
but confused when it is 4 item deep, eg,
oelib.types.TestList.test_list_nosep
in this case, oelib and oelib.types can both be treated as module
since module name contains only lower cases and class name should
contain atleast one upper case.
so, always treat leading item without upper case as module also allow
module name to contain dot.
[YOCTO #13941]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rework OETestTag so that it does not rely on the existing decorator code
base and instead inserts the tags into an attribute on the decorated
target (e.g. class/type or method). This allows the use of OETestTag on
classes and method.
In order to filter tagged tests rework the loaders filtering code,
removing the generic-ness (with validation and attributes/etc.) and
replace it with a "tags_filter" parameter which is a function that
filters a test based on the tags it has. This allows the loader user to
filter on tags in more specific ways (e.g. include all untagged tests
and any tests tagged with foo). Plumb all this through the context code
and testing code.
Update the associated tests to pass correctly with the changes.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Clean up the warning:
meta/lib/oeqa/core/loader.py:27: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
_failed_test_args = inspect.getargspec(unittest.loader._make_failed_test).args
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
oe-selftest -r <invalid test name>
currently triggers a traceback. Ensure this doesn't happen and the user gets a
sensible error message.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In previous OEQA, having 'auto' in TEST_SUITES results in executing
as many test cases as possible.
This behaviour is broken for now. From the codes in core/loader.py,
I can see that it tries to use another keyword 'all'. But in fact,
it does not work.
I've checked the current manual. The manual says using 'auto'.
Below is the current information in manual.
"""
Alternatively, you can provide the "auto" option to have all applicable
tests run against the image.
TEST_SUITES_append = " auto"
"""
So we should restore this behaviour. This patch does so.
Also, output warning message is some module is named as 'auto', as this
is a reserved keyword.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some test cases (eSDK.oeSDK*, runtime_test/*) does not match
with current regex, fix it accept all.
[YOCTO #12385]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For test modules in a package, e.g., oelib.license, running
`oe-selftest -r oelib.license` or `oe-selftest -r
oelib.license.TestSimpleCombinations` would fail with a message that
the specified test cases could not be found. This was due to the
parsing in _built_modules_dict(), which failed to distinguish between
<package>.<module>.<class> and <module>.<class>.<testcase> and treated
both cases as the latter.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
found
If some test module/case is specified to run and isn't found the OEQA
framework didn't notice it, so complete the implementation using
modules_required and validate for the test case prescense.
Raise an exception when the test module/case required isn't found.
[YOCTO #11645]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Python versions has different features from branches 3.4.x, 3.5.x and
3.6.x, i expected in wrong mode that was incremental for example changes
in 3.4.4 be in 3.5.x but that's not true.
The _make_failed_test internal method differs and is only available in
certain versions >= 3.4.4 and in 3.5.x and 3.6.x branches but not
realeses have been made including it.
So to avoid futher problems inspect the _make_failed_test and generates
function definition according what parameters are needed, the unique
supossition is that exception argument is always passed.
Related to,
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=d8380d098a290510b442a7abd2dd5a50cabf5844
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
Python unittest change the signature of the _make_failed_test
after python 3.4.4 don't pass the method name.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The small module name was added to support run a whole suite that
has more that 3 levels in the test case name, but this broke the
behaviour for use a full test case name.
[YOCTO #11632]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently there was a restriction to only execute tests that's
inherits from OETestCase but in some circunstancies the features
from the OEQA framework isn't needed so we need to support
basic unittests.
[YOCTO #10828]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Our filtering allows to specify which tests to run using,
<module_name>.[test_class].[test_name]
But the module name logic was restricted to only accept one level,
for example: runtime_test vs oelib.types, to support multiple
submodules use only the first part for filtering.
This allows to run the whole tests in a module with more than tree
levels.
Due to the ambiguity on the test filtering options with test cases
with more than tree levels the supported sytnax is,
<module>
or
<module>.[submoduleN].[test_class].[test_name]
[YOCTO #11632]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
After Python 3.5 the init method contains initialization of
_loading_packages set to avoid infinite loops in recursive
loading.
Fix,
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 70, in <module>
ret = main()
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 57, in main
results = args.func(logger, args)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 214, in run
rc = self._internal_run(logger, args)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 175, in _internal_run
self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/context.py", line 51, in loadTests
self.suites = self.loader.discover()
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/loader.py", line 268, in discover
pattern='*.py', top_level_dir=path)
File "/usr/lib/python3.5/unittest/loader.py", line 341, in discover
tests = list(self._find_tests(start_dir, pattern))
File "/usr/lib/python3.5/unittest/loader.py", line 398, in _find_tests
full_path, pattern, namespace)
File "/usr/lib/python3.5/unittest/loader.py", line 473, in _find_test_path
self._loading_packages.add(name)
AttributeError: 'OETestLoader' object has no attribute '_loading_packages'
Signed-off-by: Anibal Limon <anibal.limon@linux.intel.com>
|
|
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This will enable only run certain module tests and filter by
class and test name.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Underscore modules are meant to be run only when manually added to the test
suite, so far another mechanisms are in place to make this happen with
runtime, sdk, and esdk (mostly in test* bbclasses).
This will add such functionality in the core framework so other specific
frameworks can take use this without adding something else.
[YOCTO #10980]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If importing a test with the same name as a built-in module,
it will silently import the built-in and check for tests in
built-in module. This happened with syslog module in debian
based machines, so add a raise to avoid this behavior.
[YOCTO #10978]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
With this we get the class that is actually having the problem,
not just a TypeError with an unknown class causing the error.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Without this an exception would occur
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.
[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]
decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.
context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).
[YOCTO #10230]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|