summaryrefslogtreecommitdiffstats
path: root/scripts/resulttool
diff options
context:
space:
mode:
authorMazliana <mazliana.mohamad@intel.com>2019-02-14 13:50:38 +0800
committerRichard Purdie <richard.purdie@linuxfoundation.org>2019-02-21 12:31:50 +0000
commitf24dc9e87085a8fe5410feee10c7a3591fe9d816 (patch)
tree8ecd79368742af2354db59bf0ade3ba7bcdb3056 /scripts/resulttool
parent78a322d7be402a5b9b5abf26ad35670a8535408a (diff)
downloadopenembedded-core-f24dc9e87085a8fe5410feee10c7a3591fe9d816.tar.gz
openembedded-core-f24dc9e87085a8fe5410feee10c7a3591fe9d816.tar.bz2
openembedded-core-f24dc9e87085a8fe5410feee10c7a3591fe9d816.zip
scripts/resulttool: enable manual execution and result creation
Integrated “manualexecution” operation to resulttool scripts. Manual execution script is a helper script to execute all manual test cases in baseline command, which consists of user guideline steps and the expected results. The last step will ask user to provide their input to execute result. The input options are passed/failed/blocked/skipped status. The result given will be written in testresults.json including log error from the user input and configuration if there is any.The output test result for json file is created by using OEQA library. The configuration part is manually key-in by the user. The system allow user to specify how many configuration they want to add and they need to define the required configuration name and value pair. In QA perspective, "configuration" means the test environments and parameters used during QA setup before testing can be carry out. Example of configurations: image used for boot up, host machine distro used, poky configurations, etc. The purpose of adding the configuration is to standardize the output test result format between automation and manual execution. To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To execute manual test cases, execute the below $ resulttool manualexecution <manualjsonfile> By default testresults.json store in <build_dir>/tmp/log/manual/ [YOCTO #12651] Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'scripts/resulttool')
-rwxr-xr-xscripts/resulttool8
1 files changed, 8 insertions, 0 deletions
diff --git a/scripts/resulttool b/scripts/resulttool
index ebb5fc81c9..13430e192a 100755
--- a/scripts/resulttool
+++ b/scripts/resulttool
@@ -17,6 +17,11 @@
# To perform regression file analysis, execute the below
# $ resulttool regression-file <base_result_file> <target_result_file>
#
+# To execute manual test cases, execute the below
+# $ resulttool manualexecution <manualjsonfile>
+#
+# By default testresults.json for manualexecution store in <build>/tmp/log/manual/
+#
# Copyright (c) 2019, Intel Corporation.
#
# This program is free software; you can redistribute it and/or modify it
@@ -42,6 +47,7 @@ import resulttool.merge
import resulttool.store
import resulttool.regression
import resulttool.report
+import resulttool.manualexecution
logger = scriptutils.logger_create('resulttool')
def _validate_user_input_arguments(args):
@@ -58,6 +64,8 @@ def main():
parser.add_argument('-q', '--quiet', help='print only errors', action='store_true')
subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='<subcommand>')
subparsers.required = True
+ subparsers.add_subparser_group('manualexecution', 'manual testcases', 300)
+ resulttool.manualexecution.register_commands(subparsers)
subparsers.add_subparser_group('setup', 'setup', 200)
resulttool.merge.register_commands(subparsers)
resulttool.store.register_commands(subparsers)