summaryrefslogtreecommitdiffstats
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* wic/bootimg-efi: replace hardcoded volume name with labelChee Yang Lee2019-05-151-2/+4
| | | | | | | | | volume name should refer to --label in .wks. Replace the hardcoded volume name with label. set "ESP" as default name when no lable specified. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-09151-1240/+249
| | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Add option to dump all ptest logsJoshua Watt2019-05-082-16/+54
| | | | | | | | | | | | | Adds an option to dump all the ptest logs to individual files in a specified directory. If multiple test runs are present, the '--prepend-run' argument will create separate directories for each test run under the target directory and put the logs there to prevent each test run from clobbering the others. [YOCTO #13331] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/pybootchart/draw: Fix some bounding problemsRichard Purdie2019-05-081-1/+5
| | | | | | | | | The chart size extents were being incorrectly reported, not accounting for the width of the legend. Set a minimum width to account for that (its fixed size). Also stop printing the chart background off the bottom of the chart extents. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/pybootchart/draw: Clarify some variable namesRichard Purdie2019-05-081-9/+10
| | | | | | | This tweaks some intermediate variable names to make it clearer what is being done. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/pybootchart: Port to python3Richard Purdie2019-05-084-118/+118
| | | | | | | | | | | | This updates the pybootchart code (used for viewing build timing profiles) to use python3. The bulk of the changes are to use gi instead of pygtk, i.e. port from gtk+2 to gtk+3. The main change is to make the bootchart widget inherit gtk.Scrollable and change the way the scrollbars are implemented to match the new method upstream. The drawing code used cairo already so can remain unchanged, Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/pybootchart: Fix mixed indentationRichard Purdie2019-05-081-689/+689
| | | | | | The script had a toxic mix of tabs and spaces, fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan2019-05-081-1/+1
| | | | | | | | | | | | | | | | | | If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add global debug optionRoss Burton2019-05-041-0/+5
| | | | | | | Add a global --debug option to assist debugging. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: change expand behaviour to match docsRoss Burton2019-05-041-1/+1
| | | | | | | | | | | The documentation says that --expand takes a comma-separated list of partition:size pairs, but the code was splitting on hyphens. Hyphens are not a transitional separator for a list of items, so change the code to reflect the documentation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: add LTP compliance sectionArmin Kuster2019-04-292-1/+58
| | | | | Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: add ltp test supportArmin Kuster2019-04-292-1/+59
| | | | | Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Enable test case configuration optionsangeeta jain2019-04-261-2/+12
| | | | | | | | | | | | | | | | | | | | | | Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual There are cases when all test cases all not required to run for a module on specific DUT. Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help in reducing testing time and reporting unneccesary skip or failures. Example pre-defined json format test case configuration file (for build-applince): { "testcases" : [ "build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance", "build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)" ] } Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "wic: Set a miniumum FAT16 volume size."Angus Lees2019-04-261-1/+1
| | | | | | | | | | | | | | This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83. FAT16 volumes do not have a minimum size of 8250 blocks (== slightly over 4MB). Exhibit A: floppy disks. The original commit message suggests this was a workaround for a bug in parted - in which case we should fix it there, or use the wic `--fixed-size` option to pass down an enforced minimum from a more context-aware point in the callstack. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resultool/resultutils: Fix module import errorRichard Purdie2019-04-261-1/+1
| | | | | | | Fix AttributeError: module 'urllib' has no attribute 'request' when using remote http urls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Add log subcommandJoshua Watt2019-04-253-1/+59
| | | | | | | Adds a subcommand for dumping various logs from test results Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Load results from URLJoshua Watt2019-04-255-13/+28
| | | | | | | | Adds support for resulttool to load JSON files directly from a http:// or https:// URL Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: do not check return code of tputChen Qi2019-04-121-2/+4
| | | | | | | | | | | | The subprocess.run was replaced by subprocess.check_call because of compatibility support down to python 3.4. But we really don't care about whether that command succeeds. Some user reports that in some tmux environment, this command fails and gives some unpleasant traceback output. So we use 'call' instead of 'check_call' to avoid such problem. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Refactor and remove duplicate codeYeoh Ee Peng2019-04-111-47/+40
| | | | | | | | | Remove duplicate codes. Replace unnecessary class variables with local variables. Rename variables and arguments with simple and standard name. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: add missed moduleLei Yang2019-04-111-0/+1
| | | | | | | | | | | | When I use recipetool to add a new recipe, it says: [snip] sys.exit(14) NameError: name 'sys' is not defined [snip] Signed-off-by: Lei Yang <Lei.Yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Enable creation of configuration option fileYeoh Ee Peng2019-04-101-1/+53
| | | | | | | | | | Allow the creation of configuration option file based on user inputs. Where this configuration option file will be used by the the manual execution to display options for configuration rather than user need to inputs configuration manually. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Enable configuration options selectionYeoh Ee Peng2019-04-101-9/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current manualexecution required user to input configuration manually where there were inconsistent inputs and human typo issues. Enable manualexecution to have the optional feature where it will use pre-compiled configuration options file where user will be able to select configuration from the pre-compiled list instead of manual key-in the configuration. This will eliminate human error. Expect the pre-compiled configuration options file in json format below { "bsps-hw": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "beaglebone-yocto", "2": "edgerouter", "3": "mpc8315e-rdb", "4": "genericx86", "5": "genericx86-64" } }, "bsps-qemu": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "qemuarm", "2": "qemuarm64", "3": "qemumips", "4": "qemumips64", "5": "qemuppc", "6": "qemux86", "7": "qemux86-64" } } } Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: standard: Handle exporting generated config fragmentsNathan Rossi2019-04-051-0/+14
| | | | | | | | | | | | | The cml1 and ccmake bbclasses generate configuration fragment source files that must be exported from the WORKDIR as a source file to be preserved across builds. This change adds detection of the current recipes inherited classes and for cml1 and ccmake classes checks for the specific generated configuration fragment files. These files are then exported by devtool and included as SRC_URI files from within the target layer. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Refactor and simplify codebaseYeoh Ee Peng2019-04-041-36/+20
| | | | | | | | Simplify and removed unnecessary codes. Refactor to allow pythonic loop. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Fixed step sorted by integerYeoh Ee Peng2019-04-041-1/+1
| | | | | | | | | | | Currently the manual execution display step by sorting the step as string, where steps were not being sorted correctly when there are more than 9 steps. Fixed the step sorting by sorting step as integer. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Enable display full steps without press enterYeoh Ee Peng2019-04-041-2/+3
| | | | | | | | | | Current manualexecution required pressing enter button to show each step information, where this was wasting execution time. Enable display full steps without needing to any press enter button. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Standardize input checkYeoh Ee Peng2019-04-041-2/+2
| | | | | | | | | Current input checking does not match the standard input practiced by QA team. Change the input checking to match the standard input practiced by the QA team. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Allow extraction of ptest dataRichard Purdie2019-04-042-9/+32
| | | | | | | | | | | Rather than simply discarding the ptest data, change the code to discard it when writing out the new testresult files, or optionally either preserve it, or write it as seperate discrete logs. This means the autobuilder should start writing out individual ptest log files as well as allowing ueers to extract these manually. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Allow store to work on single filesRichard Purdie2019-04-041-9/+12
| | | | | | | | Store operations using a single file as a source weren't working as the os.walk command didn't like being given a single file. Fix the store operation to work for single files. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/engine: Add missing newlineRichard Purdie2019-04-031-1/+1
| | | | | | | | | | | | | | On some hosts the wic.Wic2.test_wic_cp_ext selftest was failing as files weren't being copied into the rootfs with "wic cp". This was due to a bug added by: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=89d4a8df074598cfb3a76e41db7c45d845afd961 where there should be a second newline added at the end of the expression due to the difference in the way echo -e and printf behave. [YOCTO #13237] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Enable report for single result fileYeoh Ee Peng2019-04-031-10/+0
| | | | | | | | | | | Current validation check function inside resulttool disallow the report for single result file although the underlying library was able to handle both directory and file as source input to report. Removed the validation check as it was no longer needed and to enable report for single result file. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: prevent starting git gc in backgroundTaras Kondratiuk2019-03-241-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "<snip>/openembedded-core/scripts/devtool", line 342, in <module> ret = main() File "<snip>/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] Signed-off-by: Taras Kondratiuk <takondra@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: To output right test case idMazliana2019-03-181-9/+6
| | | | | | | | | | | | We found that manualexecution does not capture test suite values correctly if there are more than one test suite in test cases. After verification has made we found out we should retrieved full test cases value <test_module.test_suite.test_case> from oeqa/manual/ json file rather than split it them into new variables test_suite and test_cases. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/report: Enable roll-up report for a commitYeoh Ee Peng2019-03-181-3/+13
| | | | | | | | Enable roll-up all test results belong to a commit and to provide a roll-up report. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* correct do_patch for kernel bbappend in sdkYann CARDAILLAC2019-03-121-3/+1
| | | | | | | | | | | do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python that was the case with Phytec's BSP, the fix was to replace the do_patch rule with : do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/resulttool: Enable manual result store and regressionYeoh Ee Peng2019-03-072-19/+26
| | | | | | | | | | | | | To enable store for testresults.json file from manualexecution, add layers metadata to configuration and add "manual" map to resultutils.store_map. To enable regression for manual, add "manual" map to resultutils.regression_map. Also added compulsory configurations ('MACHINE', 'IMAGE_BASENAME') to manualexecution. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Add oe-pylintRichard Purdie2019-03-061-0/+11
| | | | | | | | | | | Add an oe-pylint wrapper around pylint3 to run static analysis on our python codebase. For now display errors and exclude common warnings which are not applicable to our codebase. The aim is to start generating this report on the autobuilder and to improve our score over time. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* send-error-report: Add --no-ssl to use http protocolRobert Yang2019-03-061-3/+8
| | | | | | | | | The script use https protocol by default, but the error-report-web server's https connection may not work (e.g., doesn't work with python 2.7.6), so add an option --no-ssl to make it use http. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe.scriptutils: enable color in a more flexible wayChristopher Larson2019-03-051-6/+7
| | | | | | | | | Rather than recreating handlers and forcing them, iterate over the handlers and enable color on ones we can handle. This makes it easier to handle color properly when we introduce the bb.msg default log filters. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: make gl options gtk+-specificAlexander Kanavin2019-03-041-4/+4
| | | | | | | | If in the future GL is enabled for SDL frontend, or some other frontend, it would be useful to be able to differentiate. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: add an option to choose the SDL frontend instead of Gtk+ defaultAlexander Kanavin2019-03-041-0/+3
| | | | | | | | When Gtk+ and SDL are both enabled, qemu defaults to Gtk+. This option allows to revert to the 'classic' frontend. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/wic: Be consistent about how we call bitbakeRichard Purdie2019-02-281-11/+5
| | | | | | | | Whilst its tempting to call bitbake_main(), the script also directly executes bitbake. By doing both this creates all kinds of races. Therefore change to use subprocess execution in all cases. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Ensure that sourceparams values can contain '='Jan Kiszka2019-02-281-1/+1
| | | | | | | This allows '--sourceparams key1=val1=val11,key2=val2'. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/regression: Ensure regressoin results are sortedYeoh Ee Peng2019-02-281-4/+4
| | | | | | | Sorted regression results to provide friendly viewing of report. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/store: Fix missing variable causing testresult corruptionRichard Purdie2019-02-281-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/report: Ensure ptest results are sortedRichard Purdie2019-02-271-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/report: Ensure test suites with no results show up on the reportRichard Purdie2019-02-271-5/+8
| | | | | | | ptest suites with no results don't show up on the reports even though we have a duration for them. Fix this so the fact they report no tests is visible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/report: Handle missing metadata sections more cleanlyRichard Purdie2019-02-271-3/+6
| | | | | | | Currently some older results files cause the code to give tracebacks. Handle these missing sections more cleanly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/store: Handle results files for multiple revisionsRichard Purdie2019-02-271-16/+23
| | | | | | | Currently we cant store results if the results files span multiple different build revisons. Remove this limitation by iterating. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: do not check for GL librariesAlexander Kanavin2019-02-251-35/+0
| | | | | | | | qemu has been using libepoxy for a long time, and libepoxy loads GL via dlopen() only when instructed to. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>