summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2015-09-29runqemu-internal: Make sure two serial ports always existRandy Witt
Since inittab for qemu images now always tries to start getty on a second serial device, make sure that device exists. Otherwise the following message will be spammed: INIT: Id "S1" respawning too fast: disabled for 5 minutes [YOCTO #8374] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: modify: make bitbake use local files from srctreeMarkus Lehtonen
This change makes it possible to have local files (non-remote SRC_URI files, i.e. files that are located in the "recipe space") under the srctree even if S!=WORKDIR. The files must be placed under the 'local-files' subdirectory. Complements the previous patch that imports local files into srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: better support for local source filesMarkus Lehtonen
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: file mover function that creates target dirMarkus Lehtonen
Helper function for replacing a pattern like: target_dir = os.path.dirname(target) bb.utils.mkdirhier(target_dir) shutil.move(source, target) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update_recipe: refactor patch generationMarkus Lehtonen
Implement new function that handles patch file generation. The new function also does the discovery of new, updated and deleted patches. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update-recipe: add new patches in correct orderMarkus Lehtonen
When adding multiple new patches append them to SRC_URI in correct order so that they apply correctly. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-28runqemu-export-rootfs: update location of unfsd binaryMaxin B. John
Update the location of unfsd binary from "/usr/sbin" to "/usr/bin" in runqemu-export-rootfs [YOCTO #8315] Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28runqemu: don't complain about conflicting machines if they are equalPascal Bach
When the MACHINE variable was set as an environment variable, via "export MACHINE=qemuarm" and runqemu was executed as "runqemu qemuarm" The confusing error message appears: Error: conflicting MACHINE types [qemuarm] and [qemuarm] This checks if the two values are equal, in that case there is no problem and execution can continue. Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-26gdk-pixbuf: move gdk-pixbuf-query-loaders to $libdir for multilib safetyRoss Burton
If for example gdk-pixbuf and lib32-gdk-pixbuf are in an image then only one ${bindir}/gdk-pixbuf-query-loaders will be installed, so only one variant will actually be usable. Solve this by moving gdk-pixbuf-query-loaders into ${libdir} as it's intimately tied to the library and rarely directly invoked by the user, and update the callers to use the right path. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-24oe-pkgdata-util: avoid returning skipped packagesRobert Yang
The skipped packages may be pulled in by another package, for example, when libc6-dbg is already installed and should be skipped, but it would be pulled in by libsegfault, this patch fixes the issue. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: upgrade: use shutil.move instead of os.renameMarkus Lehtonen
Rename fails over filesystem boundaries. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: runqemu: avoid recipe parsePaul Eggleton
We only need the base configuration to get the variable values we want to get here, there's no need to parse recipes / load the cache. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: second fix for running from a different directoryMarkus Lehtonen
Do not change change current working directory permanently, but, only for the duration of tinfoil initialization instead. The previous fix caused very unintuitive behavior where using relative paths were solved with respect to the builddir instead of the current working directory. E.g. calling "devtool extract zlib ./zlib" would always create create srctree in ${TOPDIR}/zlib, independent of the users cwd. (From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add search commandPaul Eggleton
Adds a subcommand to search to find the target recipe name providing some file or capability. This is implemented by searching on recipe name, package name, description, package contents (file names), and runtime file provides. For example: $ devtool search libGL mesa $ devtool search X11 xextproto libxxf86vm xf86driproto xf86vidmodeproto libxfixes xproto libx11 ... $ devtool search /bin/sed busybox sed This is particularly useful within the extensible SDK but is also made available in devtool alongside the build system. Note of course that because this searches pkgdata, useful results depend upon do_packagedata(_setscene) having executed for the recipe being searched for. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add basic means of running runqemu within the extensible SDKPaul Eggleton
We ship the runqemu script and if we build QEMU itself within the extensible SDK, then it would be nice to be able to run it. This is a very thin wrapper around runqemu, supplying the machine and image name so the user doesn't need to. (This subcommand is only available within the extensible SDK since it only really makes sense there where it is otherwise hard to run runqemu directly.) Implements [YOCTO #6657]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool / recipetool: add handling for binary-only packagesPaul Eggleton
Add a means of creating recipes for package files or archives that contain a directory structure to be installed verbatim, for example an rpm file. (We mostly just re-use bin_package here and skip some of the normal build system checks.) This support is available in "recipetool create" and "devtool add" which wraps the former. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: improve image recipe handlingPaul Eggleton
* Make image optional for the extensible SDK (auto-determine it based on the targets the SDK was built for) * Check that specified recipe is in fact an image Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: tell user where to find output filesPaul Eggleton
If the user is running "devtool build-image" within the extensible SDK then they probably won't know where to find the resulting output files, so we should tell them explicitly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: fix recipe/package terminologyPaul Eggleton
We build recipes and include packages into the image, adjust the terminology used in code and messages accordingly. Also fix a few typos. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: move important "recipe created" message to the endPaul Eggleton
If we end up printing a message about the build directory being the same as the source, we should print that first and then print the message about the recipe file possibly needing to be edited to the end so that it has slightly more impact. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: set up fetched source as a git repository by defaultPaul Eggleton
If the fetched source isn't already a git repository, initialise it as one and then branch and tag, just as we do with "devtool modify". This makes it easier to make changes, commit them and then use the "devtool update-recipe" command to turn those commits into patches on the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: better handling for recipes that don't unpack sourcePaul Eggleton
Some recipes don't extract any source (for example, opkg-keyrings). We were producing a traceback in this case because we weren't checking if the directory existed. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: fix extracting source for work-shared recipesPaul Eggleton
Recipes that use work-shared (such as libgcc) are capable of unpacking the source, but it doesn't necessarily unpack to ${WORKDIR}/${BP}. Use the last part of the actual S value instead which is more likely to work. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: show proper error when extracting source for recipes with disabled ↵Paul Eggleton
unpack task If you try to use "devtool modify -x" or "devtool extract" on a recipe where do_unpack has been set as noexec (e.g. glibc-locale), then we get an error because the expected source wasn't ever unpacked. Do a check up front for noexec being set on do_unpack and error out with a reasonable message if that's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix handling of URIs containing #Paul Eggleton
The # character in a URI denotes a fragment; we don't care about this since it is never supposed to be sent to the server, so remove it from the URI before actually trying to fetch it or use it in SRC_URI within the recipe. (This has come up because download links on pypi.python.org seem to have a fragment containing the md5sum of the download; without stripping this off the fetcher will choke on it.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix creating empty shell functionsPaul Eggleton
The shell considers empty functions to be a syntax error, so for template shell functions that contain only comments (or no lines at all) then add a : to act as a no-op which avoids the syntax error. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: properly handle separate build directoryPaul Eggleton
When we were adding a recipe for software that would typically be built in the same directory as the source, we were always using a separate build directory unless the user explicitly specified not to, leading to errors for software that doesn't expect to be built that way (such as Python modules using distutils). Split out the code that makes this determination automatically from the "devtool modify" and "devtool upgrade" code and re-use that here so the behaviour is consistent. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool / lib/oe/recipeutils: ensure we can parse without bbappendsPaul Eggleton
These functions ostensibly allowed parsing a recipe without bbappends but this clearly hadn't been tested because a variable was unassigned in both of them in that case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: ensure --color=never turns off recipetool colour outputPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: check that source tree still existsPaul Eggleton
Sometimes, particularly if you extracted the source to /tmp which is on tmpfs, the external source tree that is being pointed to may no longer exist when you come to run "devtool build" or "devtool update-recipe" etc. Make all of the commands that need to check for a recipe being in the workspace call a single function and have that function additionally check the source tree still exists where appropriate. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22scripts/contrib: add devtool stress testerPaul Eggleton
Add a script to run "devtool modify" followed by a build on every target recipe in the environment (with the option to skip/resume from/only include specific recipes). This takes far too long to run as an oe-selftest test but is still something that is useful to be able to run. There's also a slightly quicker mode that just runs "devtool extract" on each recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22classes/populate_sdk_ext: drop work-config.incPaul Eggleton
This is not actually used for anything - I thought that we would need to use it within devtool to set global configuration, but we're able to do everything we need within the bbappends it creates, which also saves on parse time. If we're not going to use work-config.inc let's just drop it completely. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-21python3-debugger: Adds pkgutils dependency to pdbAlejandro Hernandez
python3-debugger fails to be invoked to debug other scripts complaining about not being able to import pkutil, this patch adds pkgutil as a dependency for python3-debugger fixing the issue. [YOCTO #8334] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21python3-debugger: fix importlib dependencyAlejandro Hernandez
python3-debugger (pdb) needs importlib as a dependency, if not included it produces an error when importing pdb, making pdb unusable, this patch adds importlib dependency fixing the issue. {YOCT0 #8333] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21runqemu-internal: For qemumicroblaze use the QEMU provided device treeNathan Rossi
Setup the qemumicroblaze machine to use the device tree provided by QEMU instead of the device tree located in the images directory. Additionally setup the default memory size to match the QEMU device tree. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21runqemu-internal: Fix qemu networking for qemuzynq an qemumicroblazeNathan Rossi
This patch brings the qemu networking setup for qemuzynq and qemumicroblaze into feature parity with the other qemu machines. Specifically enabling TAP interface attachcment and kernel command line IP configuration. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oe-git-proxy: Allow socks4 as protocol in $ALL_PROXYPeter Kjellerstedt
The current default is to use SOCKS4a when socks is specified as protocol in $ALL_PROXY. However, not all socks servers support SOCKS4a. By allowing socks4 as an additional protocol, this script will happily work with SOCKS4 only servers. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oe-git-proxy: Correct the parsing of a port in $ALL_PROXYPeter Kjellerstedt
Due to an error in the regular expression used to extract a port number specified in $ALL_PROXY, rather than allowing the port number to be followed by an optional "/", the port was required to be followed by "/?". This corrects the regular expression to allow an optional "/". It also allows the odd "/?" suffix for backwards compatibility. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oe-git-proxy: Allow explicit IP addresses in $NO_PROXYPeter Kjellerstedt
Without this fix, if one specified, e.g., 127.0.0.1 in $NO_PROXY, the oe-git-proxy script would fail with a message like this: /home/pkj/yocto/poky/scripts/oe-git-proxy: line 64: 32-127.0.0.1: syntax error: invalid arithmetic operator (error token is ".0.0.1") Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21wic: fix partition size calculationEd Bartosh
It was wrongly assumed that part.size is meagured in Mb. In fact it's in Kb, so there is no need to convert bitbake variable ROOTFS_SIZE as it's also in Kb. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2015-09-16recipetool: add 'newappend' sub-commandChristopher Larson
This sub-command creates a bbappend for the specified target and prints the path to the bbappend. The -w argument, as with some of the other recipetool commands, will make a version-independent bbappend. Example usage: recipetool newappend meta-mylayer virtual/kernel [YOCTO #7964] Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-16mkefidisk: Create interactive menu for the scriptBruno Bottazzini
If it is passed only the image parameter, it will show some interactive options to help to conclude the script. With this patch it will detect devices available in the machine, asking to choose what it is wanted to be flashed. it will suggest the target device. If it is a SD card the suggestion is /dev/mmcblk0. If it is a pendrive it will suggest to use /dev/sda. Signed-off-by: Bruno Bottazzini <bruno.bottazzini@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-12devtool: update-recipe: get srcuri parameters with decodeurl()Markus Lehtonen
Use already existing bb.fetch.decodeurl() for getting the parameters for a URI. This is more fault tolerant and maintainable. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12runqemu: avoid image file name mismatchesPatrick Ohly
Giving anything with -image in it as bootparams or in qemuparams (for example, an additional -drive parameter with an image file or an "-initrd .../core-image-minimal-initramfs-qemux86.cpio.gz") caused runqemu to treat these parameters as names of the rootfs image file. Matching *-image) after checking the current argument for more specific cases like bootparams and qemuparams avoids this misinterpretation of the command line parameters. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-12runqemu: Define OECORE_MACHINE_SYSROOT on setup_sysrootLeonardo Sandoval
At least the OVFM (UEFI Firmware for Qemu and KVM) recipe stores the BIOS under $OE_TMPDIR/sysroots/$MACHINE, now defined as OECORE_MACHINE_SYSROOT. The latter is used when searching BIOS, VGA BIOS and keymaps. As a example, to boot a OVFM BIOS, one can run the following command: $ runqemu qemux86-64 core-image-minimal \ biosdir=usr/share/ovmf \ biosfilename=bios.bin \ nographic Note the bios* parameters: these two are needed to specify the subfolder (parent folder is OECORE_MACHINE_SYSROOT) and BIOS filename (without it, it picks a BIOS named bios-256k.bin). [YOCTO #5654] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-12buildstats: Outputs 'task recipe elapsed-time' from each buildstats' recipeLeonardo Sandoval
Given a 'buildstats' path (created by bitbake when setting USER_CLASSES ?= "buildstats" on local.conf) and task names, outputs '<task> <recipe> <elapsed time>' for all recipes. Elapsed times are in seconds, and task should be given without the 'do_' prefix. Some useful pipelines 1. Tasks with largest elapsed times $ buildstats.sh -b <buildstats> | sort -k3 -n -r | head do_compile perl-5.20.0-r1 221.82 do_configure gettext-native-0.19.4-r0 140.34 do_compile openssl-native-1.0.2a-r0 107.48 do_compile openssl-1.0.2a-r0 102.10 do_configure perl-native-5.20.0-r0 90.70 do_configure gettext-0.19.4-r0 88.17 do_compile gcc-cross-i586-4.9.2-r0 83.98 do_configure m4-native-1.4.17-r0 83.44 do_compile qemu-native-2.2.0-r1 71.69 do_compile glibc-2.21-r0 60.88 2. Min, max, sum per task $ buildstats.sh | datamash -t' ' -g1 min 3 max 3 sum 3 | sort -k4 -n -r do_configure 0.03 140.34 1968.66 do_compile 0.01 221.82 1664.44 do_install 0.03 40.31 330.45 do_populate_sysroot 0.11 34.45 229.23 do_unpack 0.01 36.1 193.54 do_patch 0.01 9.2 62.07 do_fetch 0.01 6.66 32.13 do_populate_lic 0.09 1.65 30.7 Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-12devtool: make plugin_init optionalEd Bartosh
So far all devtool and recipetool plugins were expected to have plugin_init function. This function is empty in most of plugins as they don't require initialisation. Making plugin_init optional would allow not having empty plugin_init in every plugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-12scripts: Move BUILDDIR chmod fix after checks in oe-setup-builddirAlex Franco
Makes more sense to have this fix after checking BUILDDIR exists, is a directory and is writable. [YOCTO #7669] Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-09devtool: fix running from a different directoryPaul Eggleton
Fixes the following error when running devtool from a directory other than the build directory (or the SDK base path when using within the extensible SDK): The BBPATH variable is not set and bitbake did not find a conf/bblayers.conf file in the expected location. Maybe you accidentally invoked bitbake from the wrong directory? Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>