summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2015-10-24runqemu: Enable support for kvm without vhost in x86 and x86_64Aníbal Limón
KVM can be used without vhost so add a new option to runqemu for use kvm with vhost. Example, runqemu qemux86 core-image-minimal kvm # kvm without vhost runqemu qemux86 core-image-minimal kvm-vhost # kvm with vhost [YOCTO #7443] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24scripts/oe-publish-sdk: create directory before making git repoCostin Constantin
This patch fixes a small bug that prevents seting a git repo in exported SDK layers dir. Before setting a git repo, that directory needs to be created. [ YOCTO #6659 ] Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19scripts/gen-lockedsig-cache: fix race with temp file creationPaul Eggleton
As part of populating the sstate-cache with an artifact (.tgz file) we create a temp file and then atomically move it to the final name. Due to the glob used in this script such temp files were being matched, and between the time they were matched and the time the script started copying files, the temp file may have vanished. This fixes random "No such file or directory" failures building the extensible SDK on build setups where the sstate-cache directory is shared amongst multiple build machines. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19devtool: extract: fix error handlingPaul Eggleton
If recipe parsing fails we need to exit out instead of attempting to use the data object that's set to None, which resulted in a traceback. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03recipetool: create: fix change in path structure if --extract-to path existsPaul Eggleton
If the directory specified by --extract-to exists, because we were using shutil.move() to move the temporary extracted directory to the specified path, a subdirectory was being created under that directory instead of moving the contents, which was a different result than if the directory didn't previously exist. We could try to always move the contents but that's complicated when any symlinks are involved; the simplest thing is just to remove the directory (which should be empty anyway) before moving the temporary directory across in its place. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-03devtool: update-recipe: avoid updating patches that have not changedPaul Eggleton
Use "git cherry" against the original tag that we made when we extracted the source in order to find the revisions that are definitely new. This allows you to modify a commit in the middle of the series and then run devtool update-recipe and not have the subsequent patches unnecessarily modified. Fixes [YOCTO #8388]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01fontcache: allow to pass extra parameters and environment to fc-cacheMartin Jansa
* this can be useful for passing extra parameters, pass -v by default to see what's going on in do_rootfs * we need to use this for extra parameter we implemented in fontconfig: --ignore-mtime always use cache file regardless of font directory mtime because the checksum of fontcache generated in do_rootfs doesn't match with /usr/share/fonts directory as seen on target device causing fontconfig to re-create the cache when fontconfig is used for first time or worse create new cache in every user's home directory when /usr/ filesystem is read only and cache cannot be updated. Running FC_DEBUG=16 fc-cache -v on such device shows: FcCacheTimeValid dir "/usr/share/fonts" cache checksum 1441207803 dir checksum 1441206149 * my guess is that the checksum is different, because pseudo (which is unloaded when running qemuwrapper) or because some influence of running the rootfs under qemu. Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: modify: use correct local files directory nameMarkus Lehtonen
The name of the directory for local source files under srctree is 'oe-local-files', not 'local-files'. Fixes a bug that slipped through in b7ab82485e4514e07ab8a76e554da27ddc92e6c0. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: update-recipe: enable var history trackingMarkus Lehtonen
Enable variable history tracking so that the variables are updated in the correct file - i.e. in the file they are already defined. [YOCTO #7715] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01runqemu-internal: qemuarm enable usage of virtio devicesAníbal Limón
We are experiencing occasional segfaults in scsi sym53c8xx driver on qemuarm boot. There are some old discussions into the mailing lists [1] about the scsi problem and seems to be isn't fixed. We use virtio blk/net devices into qemuarm64 also are supported into qemuarm so change to use it because virtio devices are the best choice. [YOCTO #8060] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=8060#c10 Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29runqemu-internal: Make sure two serial ports always existRandy Witt
Since inittab for qemu images now always tries to start getty on a second serial device, make sure that device exists. Otherwise the following message will be spammed: INIT: Id "S1" respawning too fast: disabled for 5 minutes [YOCTO #8374] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: modify: make bitbake use local files from srctreeMarkus Lehtonen
This change makes it possible to have local files (non-remote SRC_URI files, i.e. files that are located in the "recipe space") under the srctree even if S!=WORKDIR. The files must be placed under the 'local-files' subdirectory. Complements the previous patch that imports local files into srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: better support for local source filesMarkus Lehtonen
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: file mover function that creates target dirMarkus Lehtonen
Helper function for replacing a pattern like: target_dir = os.path.dirname(target) bb.utils.mkdirhier(target_dir) shutil.move(source, target) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update_recipe: refactor patch generationMarkus Lehtonen
Implement new function that handles patch file generation. The new function also does the discovery of new, updated and deleted patches. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update-recipe: add new patches in correct orderMarkus Lehtonen
When adding multiple new patches append them to SRC_URI in correct order so that they apply correctly. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-28runqemu-export-rootfs: update location of unfsd binaryMaxin B. John
Update the location of unfsd binary from "/usr/sbin" to "/usr/bin" in runqemu-export-rootfs [YOCTO #8315] Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28runqemu: don't complain about conflicting machines if they are equalPascal Bach
When the MACHINE variable was set as an environment variable, via "export MACHINE=qemuarm" and runqemu was executed as "runqemu qemuarm" The confusing error message appears: Error: conflicting MACHINE types [qemuarm] and [qemuarm] This checks if the two values are equal, in that case there is no problem and execution can continue. Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-26gdk-pixbuf: move gdk-pixbuf-query-loaders to $libdir for multilib safetyRoss Burton
If for example gdk-pixbuf and lib32-gdk-pixbuf are in an image then only one ${bindir}/gdk-pixbuf-query-loaders will be installed, so only one variant will actually be usable. Solve this by moving gdk-pixbuf-query-loaders into ${libdir} as it's intimately tied to the library and rarely directly invoked by the user, and update the callers to use the right path. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-24oe-pkgdata-util: avoid returning skipped packagesRobert Yang
The skipped packages may be pulled in by another package, for example, when libc6-dbg is already installed and should be skipped, but it would be pulled in by libsegfault, this patch fixes the issue. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: upgrade: use shutil.move instead of os.renameMarkus Lehtonen
Rename fails over filesystem boundaries. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: runqemu: avoid recipe parsePaul Eggleton
We only need the base configuration to get the variable values we want to get here, there's no need to parse recipes / load the cache. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: second fix for running from a different directoryMarkus Lehtonen
Do not change change current working directory permanently, but, only for the duration of tinfoil initialization instead. The previous fix caused very unintuitive behavior where using relative paths were solved with respect to the builddir instead of the current working directory. E.g. calling "devtool extract zlib ./zlib" would always create create srctree in ${TOPDIR}/zlib, independent of the users cwd. (From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add search commandPaul Eggleton
Adds a subcommand to search to find the target recipe name providing some file or capability. This is implemented by searching on recipe name, package name, description, package contents (file names), and runtime file provides. For example: $ devtool search libGL mesa $ devtool search X11 xextproto libxxf86vm xf86driproto xf86vidmodeproto libxfixes xproto libx11 ... $ devtool search /bin/sed busybox sed This is particularly useful within the extensible SDK but is also made available in devtool alongside the build system. Note of course that because this searches pkgdata, useful results depend upon do_packagedata(_setscene) having executed for the recipe being searched for. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add basic means of running runqemu within the extensible SDKPaul Eggleton
We ship the runqemu script and if we build QEMU itself within the extensible SDK, then it would be nice to be able to run it. This is a very thin wrapper around runqemu, supplying the machine and image name so the user doesn't need to. (This subcommand is only available within the extensible SDK since it only really makes sense there where it is otherwise hard to run runqemu directly.) Implements [YOCTO #6657]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool / recipetool: add handling for binary-only packagesPaul Eggleton
Add a means of creating recipes for package files or archives that contain a directory structure to be installed verbatim, for example an rpm file. (We mostly just re-use bin_package here and skip some of the normal build system checks.) This support is available in "recipetool create" and "devtool add" which wraps the former. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: improve image recipe handlingPaul Eggleton
* Make image optional for the extensible SDK (auto-determine it based on the targets the SDK was built for) * Check that specified recipe is in fact an image Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: tell user where to find output filesPaul Eggleton
If the user is running "devtool build-image" within the extensible SDK then they probably won't know where to find the resulting output files, so we should tell them explicitly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: fix recipe/package terminologyPaul Eggleton
We build recipes and include packages into the image, adjust the terminology used in code and messages accordingly. Also fix a few typos. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: move important "recipe created" message to the endPaul Eggleton
If we end up printing a message about the build directory being the same as the source, we should print that first and then print the message about the recipe file possibly needing to be edited to the end so that it has slightly more impact. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: set up fetched source as a git repository by defaultPaul Eggleton
If the fetched source isn't already a git repository, initialise it as one and then branch and tag, just as we do with "devtool modify". This makes it easier to make changes, commit them and then use the "devtool update-recipe" command to turn those commits into patches on the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: better handling for recipes that don't unpack sourcePaul Eggleton
Some recipes don't extract any source (for example, opkg-keyrings). We were producing a traceback in this case because we weren't checking if the directory existed. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: fix extracting source for work-shared recipesPaul Eggleton
Recipes that use work-shared (such as libgcc) are capable of unpacking the source, but it doesn't necessarily unpack to ${WORKDIR}/${BP}. Use the last part of the actual S value instead which is more likely to work. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: show proper error when extracting source for recipes with disabled ↵Paul Eggleton
unpack task If you try to use "devtool modify -x" or "devtool extract" on a recipe where do_unpack has been set as noexec (e.g. glibc-locale), then we get an error because the expected source wasn't ever unpacked. Do a check up front for noexec being set on do_unpack and error out with a reasonable message if that's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix handling of URIs containing #Paul Eggleton
The # character in a URI denotes a fragment; we don't care about this since it is never supposed to be sent to the server, so remove it from the URI before actually trying to fetch it or use it in SRC_URI within the recipe. (This has come up because download links on pypi.python.org seem to have a fragment containing the md5sum of the download; without stripping this off the fetcher will choke on it.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix creating empty shell functionsPaul Eggleton
The shell considers empty functions to be a syntax error, so for template shell functions that contain only comments (or no lines at all) then add a : to act as a no-op which avoids the syntax error. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: properly handle separate build directoryPaul Eggleton
When we were adding a recipe for software that would typically be built in the same directory as the source, we were always using a separate build directory unless the user explicitly specified not to, leading to errors for software that doesn't expect to be built that way (such as Python modules using distutils). Split out the code that makes this determination automatically from the "devtool modify" and "devtool upgrade" code and re-use that here so the behaviour is consistent. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool / lib/oe/recipeutils: ensure we can parse without bbappendsPaul Eggleton
These functions ostensibly allowed parsing a recipe without bbappends but this clearly hadn't been tested because a variable was unassigned in both of them in that case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: ensure --color=never turns off recipetool colour outputPaul Eggleton
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: check that source tree still existsPaul Eggleton
Sometimes, particularly if you extracted the source to /tmp which is on tmpfs, the external source tree that is being pointed to may no longer exist when you come to run "devtool build" or "devtool update-recipe" etc. Make all of the commands that need to check for a recipe being in the workspace call a single function and have that function additionally check the source tree still exists where appropriate. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22scripts/contrib: add devtool stress testerPaul Eggleton
Add a script to run "devtool modify" followed by a build on every target recipe in the environment (with the option to skip/resume from/only include specific recipes). This takes far too long to run as an oe-selftest test but is still something that is useful to be able to run. There's also a slightly quicker mode that just runs "devtool extract" on each recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22classes/populate_sdk_ext: drop work-config.incPaul Eggleton
This is not actually used for anything - I thought that we would need to use it within devtool to set global configuration, but we're able to do everything we need within the bbappends it creates, which also saves on parse time. If we're not going to use work-config.inc let's just drop it completely. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-21python3-debugger: Adds pkgutils dependency to pdbAlejandro Hernandez
python3-debugger fails to be invoked to debug other scripts complaining about not being able to import pkutil, this patch adds pkgutil as a dependency for python3-debugger fixing the issue. [YOCTO #8334] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21python3-debugger: fix importlib dependencyAlejandro Hernandez
python3-debugger (pdb) needs importlib as a dependency, if not included it produces an error when importing pdb, making pdb unusable, this patch adds importlib dependency fixing the issue. {YOCT0 #8333] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21runqemu-internal: For qemumicroblaze use the QEMU provided device treeNathan Rossi
Setup the qemumicroblaze machine to use the device tree provided by QEMU instead of the device tree located in the images directory. Additionally setup the default memory size to match the QEMU device tree. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21runqemu-internal: Fix qemu networking for qemuzynq an qemumicroblazeNathan Rossi
This patch brings the qemu networking setup for qemuzynq and qemumicroblaze into feature parity with the other qemu machines. Specifically enabling TAP interface attachcment and kernel command line IP configuration. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oe-git-proxy: Allow socks4 as protocol in $ALL_PROXYPeter Kjellerstedt
The current default is to use SOCKS4a when socks is specified as protocol in $ALL_PROXY. However, not all socks servers support SOCKS4a. By allowing socks4 as an additional protocol, this script will happily work with SOCKS4 only servers. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oe-git-proxy: Correct the parsing of a port in $ALL_PROXYPeter Kjellerstedt
Due to an error in the regular expression used to extract a port number specified in $ALL_PROXY, rather than allowing the port number to be followed by an optional "/", the port was required to be followed by "/?". This corrects the regular expression to allow an optional "/". It also allows the odd "/?" suffix for backwards compatibility. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>