Commit message (Collapse)AuthorAgeFilesLines
* Update version to Purdie2012-07-192-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Warn upon finding tab indentation in python functionsRichard Purdie2012-07-192-2/+4
| | | | | | | | | | Mixed spacing in python functions can cause subtle issues and generally confuses users. We've standardised on 4 space indentation, adding this warning helps ensure consistency and avoid bugs. It also makes _prepend and _append operations on python functions slightly less risky. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/svn.py: Use protocol parameter to get the protocolMartin Jansa2012-07-181-1/+1
| | | | | | | | | * it was send in v1 of proto -> protocol changes but then wasn't in V2 http://patchwork.openembedded.org/patch/31617/ where warning about proto= was moved to shared __init__ Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* hob2: add elf to hcc supported image typesKang Kai2012-07-171-0/+1
| | | | | | | | | | | [Yocto #2709] OE Core Commit c9b01af84b64edba056f959c349895e0698324e6 add elf image type. This will cause hob build image failed. Add elf to hcc supoorted image types to fix this issue. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Insure .siginfo files writes into shared sstate cache are atomicJeffrey C Honig2012-07-171-4/+15
| | | | | | | | | Use tempfile.mkstemp to create a temporary file in the sstate dir and move it into place after closing. The previous code would fail in the chmod() if two users were running jobs that touched the same signature file. Signed-off-by: Jeffrey C Honig <jeffrey.honig@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Improve error output to be more useful when non-existent tasks ↵Richard Purdie2012-07-171-1/+1
| | | | | | are found Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/__init__.py: Warn user if SRC_URI is using "proto" and not "protocol"Andrei Gherzan2012-07-111-0/+4
| | | | | | | As well, if "proto" is used, get the associated value as "protocol" Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/bzr.py: Use "protocol" parameter to get the protocolAndrei Gherzan2012-07-111-1/+1
| | | | | Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/hg.py: Use "protocol" parameter to get the protocolAndrei Gherzan2012-07-111-1/+1
| | | | | Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/osc.py: Use "protocol" parameter to get the protocolAndrei Gherzan2012-07-111-1/+1
| | | | | Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Abort build if runtime dependency conflictWenzong Fan2012-07-112-1/+4
| | | | | | | | | | | | Currently if there are multiple preferred providers available for a runtime dependency, bitbake will print an Error message and let the build go on. Anyways the build should abort while any Errors occured. [YOCTO #2734] Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/wget: Use FETCHCMD_wget to provide the commandline and optionsRichard Purdie2012-07-101-3/+5
| | | | | | | This also changes to use -t 2 -T 30 as the defaults which are more sane for a modern fetcher and already specified in OpenEmbedded metadata. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/cvs.py: Switch to use FETCHCMD_cvs and handle parameters in the ↵Richard Purdie2012-07-101-9/+3
| | | | | | | | | fetcher itself This brings the cvs fetcher more into line with the others and allows consistent usage of the FETCHCMD variable and option handling. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker: Print which pkgs would be built in -g outputRobert Yang2012-07-092-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is for giving the user a clear list to show which pkg would be built, we have the "bitbake -g", but it is not easy to read for people, it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist: $ bitbake -g core-image-sato ... NOTE: PN build list saved to 'pn-buildlist' [snip] The contents of pn-buildlist: busybox shadow-native pth sysfsutils qemu-helper-native curl-native ncurses-native gdbm xserver-xorg linux-libc-headers [snip] [YOCTO #2404] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* usermanual: Fix missing markupRobert P. J. Day2012-07-061-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart: Fix multiple override interaction with append and prepend operatorsRichard Purdie2012-07-051-1/+6
| | | | | | | | | | | | | | | | | Variables which used multiple overrides and the append/prepend operators were not functioning correctly. This change fixes that. This fixes the testcase: OVERRIDES = "linux:x86" TESTVAR = "original" TESTVAR_append_x86 = " x86" TESTVAR_append_x86_linux = " x86+linux" TESTVAR_append_linux_x86 = " linux+x86" [YOCTO #2672] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Fix recursive task pruning to only prune self referencing tasksRichard Purdie2012-07-041-2/+4
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/svn: Enhance to cope with subversion 1.7 upgradeRichard Purdie2012-07-041-5/+10
| | | | | | | | | | | svn changed working checkout formats between 1.6 and 1.7. Its convoluted to detect what format a given working copy is in so the simplest solution is simply to run "svn upgrade" within the working copy. The base svn command variable is relocated slightly to enable this new code to work effectively. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* usermanual: Improve task dependency documentation to match realityRichard Purdie2012-07-041-12/+10
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Allow recrdeptasks that have self referencesRichard Purdie2012-07-041-2/+7
| | | | | | | | | | | | | | | | | | | | In some cases we want to pull in DEPENDS and RDEPENDS of recrdeptask dependencies but we need a way to trigger or avoid this behaviour depending on context. The logical syntax to trigger such behaviour would be a self referencing recrdeptask: do_a[recrdeptask] = "do_a do_b" The dependency chains already recurse this kind of expression correctly, the missing piece is to avoid any circular reference errors. Since the dependencies have already been recursively resolved, simply removing any recrdeptask references is enough to break the circular references. This patch therefore removes any circular references using the set difference_update() operator. There will be metadata tweaks required to add any references needed to the extra taskname. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Hob: fixed some variables not clean bug in detail pageLiming An2012-07-041-11/+3
| | | | | | | [YOCTO #2679] Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* git: do not execute 'git remote prune'Enrico Scholz2012-07-041-1/+0
| | | | | | | | | | | | | | 'git remote prune' at this location does not make much sense because the following 'git remote rm' will prune stale and non-stale branches. The 'prune' can cause trouble because it will access the network bypassing the no-network code in bitbake. When this operation fails and throws an exception, the next command (--> 'git remote rm') will be skipped. This in turn, will make all the following operations fail, because they assume that the remote does not exist yet. Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Hob: change 'run image' and 'deploy' Gui and work flow for building detail pageLiming An2012-06-291-114/+98
| | | | | | | | | | | | | | | | | | In build detail page, the response action of clicked the 'run image' or 'deploy' button will accroding to below as ui design: 1)if there has one file in building result, it will responsed the 'run image'(now, we only support the qemu) or 'deploy' directly 2)if there has more than one file, it will popup a dialog with listed created files type, they are has same action attributes 'deploy' or 'runnable'. Note: because the qemu image (runnable file) can't be deployed and we can't generated a image that has the two attributes now, can be run or can be deployed, so the code will not deal with this case. [YOCTO #2155] Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Handle multiple rdeptask entriesRichard Purdie2012-06-281-2/+2
| | | | | | | | I'm not sure why we don't currently allow multiple entries in rdeptask when we do in deptask. This makes the handling match between the two since its trivial to fix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Revert the regexp removal for the type field and instead anchor regexpRichard Purdie2012-06-281-9/+8
| | | | | | | | | | People are using regexps in the url type field so we need to preserve this bitbake behaviour. To address the issues with https:// urls mapping badly to file:// urls we anchor the regexp if its not already anchored. There should be no expressions in the wild which would break with this change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* test/fetch: Switch the comparision order to make test failures slightly clearerRichard Purdie2012-06-281-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* test/fetch: Ensure cwd is valud for the git cloneRichard Purdie2012-06-281-0/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Reimplement recrdepends so it works more correctlyRichard Purdie2012-06-281-58/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, recrdepends is extremely greedy. For example: do_foo[rdepends] = "somedep:sometask" addtask foo which adds foo with *no* dependencies, will suddenly start appearing as a dependency in every task which uses recrdepends. So far this has been mildy annoying but we now have use cases where this makes no sense at all. This reworks the recrdepends code to avoid this problem. To do this we can no longer collapse things into lists just based on file ID. The problem is this code is extremely performance sensitive. The "preparing runqueue" phase spends a lot of time in these recursive dependency calculations so any change here could negatively impact the user experience. As such, this code has been carefully tested on convoluted dependency trees with operations like "time bitbake world -g". The net result of this change and the preceeding changes combined is a net speed up of these operations in all cases measured. Tests were made comparing "bitbake world -g" task-depends.dot before and after this patch. There *are* differences for example -nativesdk do_build dependencies on -native recipes are no longer present. All removed dependencies appear to be sensible improvements to the system. The "rdepends" cross contamination issue above is also fixed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Convert depends variable to use setsRichard Purdie2012-06-281-12/+8
| | | | | | | This gives some small performance gains and sets the scene for other improvements by removing the need for duplicate detection code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* taskdata: Add gettask_id_fromfnid helper functionRichard Purdie2012-06-282-8/+14
| | | | | | | | This is like gettask_id but doesn't require translation of fnid -> fn first which the function then translates back. This gives a sizeable performance improvement since a significant number of lookups are avoided. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix for Hob bug #2323Cristian Iorga2012-06-252-12/+4
| | | | | | | | | Removed unnecessary button from Recipes screen and replaced the button that diplayed packages size and total image size from Packages screen with a label. Signed-off-by: Cristian Iorga <cristian.iorga@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* hig.py: use module tempfile to create temp fileKang Kai2012-06-251-12/+8
| | | | | | | | I am sorry that use os.tmpname which casue a security warning. Follow Darren's suggestion to use tempfile.NamedTemporaryFile instead. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Ensure star character doesn't end up in mirror tarball namesRichard Purdie2012-06-252-2/+2
| | | | | | | | | | If '*' does end up in mirror urls accidently, some strange things can break since supports_checksum() looks for this, ud.localpath can then get ignored and this can lead to empty directories being downloaded "successfully". '*' is a special case for file urls only at this point so remove any entries that accidentlly make it in through url mapping. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/utils.py: remove unnecessary preserved variablesPaul Eggleton2012-06-251-2/+0
| | | | | | | | | | | | | | | | | | _ and LANG no longer need to be preserved from the external environment. The value of _ changes between non-pseudo-wrapped and pseudo-wrapped invocations (e.g. between "bitbake -p" and "bitbake target") and this will currently trigger a full reparse in the absence of a whitelist entry in BB_HASHCONFIG_WHITELIST, which is not ideal. LANG used to be preserved in order to ensure the C locale was being used for tools invoked by bitbake, however we now set LC_ALL in bitbake.conf to take care of this. Second part of the fix for [YOCTO #2600]. Acked-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Add new mirror syntax to simplify mirror specificationsRichard Purdie2012-06-252-5/+22
| | | | | | | | | | | | | | | | | | | | | When writing mirror specifications, the current regexp syntax can be awkward and hard to get it to do what you want. For example, extracting the 'basename' of a repository: PREMIRRORS = "git://.*/([^/]+/)*([^/]*) git://somewhere.org/somedir/\\2;protocol=file" can now become: PREMIRRORS = "git://.*/.* git://somewhere.org/somedir/BASENAME;protocol=file" which is much clearer. A MIRRORNAME substitution is also added which contains an encoded form of both host and path. One of the problems with the existing regexp syntax is you couldn't access HOST information from PATH and vice-versa which is an issue this patch also addresses. Tests for the new syntax are also added. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Handle errors orruring when building mirror urlsRichard Purdie2012-06-251-2/+11
| | | | | | | | | | When we build the mirror urls, its possible an error will occur. If it does, it should just mean we don't attempt this mirror url. The current code actually aborts *all* the mirrors, not just the failed url. This patch catches and logs the exception allowing things to continue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* process: Improve _logged_communicate bufferingRichard Purdie2012-06-252-14/+35
| | | | | | | | | | | | | | | | | | | | | | | There are two problems with the _logged_communicate that are both caused as a result of buffering I/O issues: 1) log truncation when python fails 2) While a bitbake task is running it is impossible to see what is going on if it is only writing a small incremental log that is smaller than the buffer, or you get only a partial log, up until the task exists. It is worse in the case that stderr and stdout are separate file handles, because previous code blocks on the read of stdout and then stderr, serially. The right approach is simply to use select() to determine if there is data available and also flush the log before exiting. This is based on a patch from Jason Wessel <jason.wessel@windriver.com> with some changes to flush upon exit, abstract the non blocking file descriptor setup and drop the buffer size parameter. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* taskdata.py: Add support for rdepends task flagRichard Purdie2012-06-253-1/+30
| | | | | | | | Currently its not possible to add arbitrary RDEPENDS to a specific task. This can be useful and this patch adds functionality equivalent to the 'depends' task flag. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: Add function to set nonblocking operation on a file descriptorRichard Purdie2012-06-252-1/+6
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* git.py: Remove -l option, its not necessaryRichard Purdie2012-06-251-1/+2
| | | | | | | | | | | If -l is specified and the source and destination are not on a common filesystem an error occurs. The -l option is however the default for git for local paths which the fetcher already now ensures in the file:// case. We can therefore safely drop the -l option. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen.py: log when tainting the signature of a taskPaul Eggleton2012-06-221-0/+1
| | | | | | | | | Log a note when applying a taint to a task signature (e.g. when using the -f or -C command line options) so that the user knows this has been done. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tweak help text for -C slightlyPaul Eggleton2012-06-221-1/+1
| | | | | | | | | What is specified is a command (or "cmd" as in the help text for -f) rather than a task - i.e. you specify compile, not do_compile, so change the sentence to reflect that and reorder it slightly so it makes sense. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git.py: Optimize clone fall back when it is localJason Wessel2012-06-211-1/+3
| | | | | | | | A file:// url should use "clone -l" to greatly speed up the clone in the case of a kernel when it is local. Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ui/depexp.py: use the new progressbar in HobKang Kai2012-06-211-11/+18
| | | | | | | | | | [Yocto 2149] Use the new progressbar in Hob, so that we can reduce the maintenance burden of the multiple GUIs. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Update replace_uri to handle uri types explicitlyRichard Purdie2012-06-211-0/+6
| | | | | | | | | | | | For mirror mapping, we never use regexps for the type component of the url. Doing so causes various slightly bizarre behaviour such as https:// urls being translated to files:// urls which we have no handler for. This patch forces the type matches to be direct, not regexp based and gives the expected bahvriour. (From Poky rev: 695d8aca0999d2d61970b990e83c3132ba6f12cb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* test/fetch: Add a couple of extra test casesRichard Purdie2012-06-211-1/+29
| | | | | | (From Poky rev: 01d8220ef8580ca980a14c05d500765ca2d37e2c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* test/fetch: Add in unit tests for uri_replace() and git premirrorsRichard Purdie2012-06-211-3/+52
| | | | | | (From Poky rev: 4be81b5cadb7813d37152411c23764501f5869d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: uri_replace() only consider ud.localpath where its a fileRichard Purdie2012-06-211-1/+1
| | | | | | | | | | Using ud.localpath as a basename when it points at a directory causes problems. The supports_checksum() method gives a good indication of whether ud.localpath can be used in the way we need. (From Poky rev: 933ec8a44634e33f92f6f76de3a34094c3d63aa6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: uri_replace() improve mirrortarball handlingRichard Purdie2012-06-211-1/+4
| | | | | | | | | | We only consider mirror tarballs when the source and target urls are of differing types. We also should clear all url paramters when handling mirror tarballs. (From Poky rev: da140b8b0b3dda5429f9eee68829ef5247cdfe12) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: uri_replace() remove what amounts to a null operation and add some ↵Richard Purdie2012-06-211-2/+2
| | | | | | | | comments (From Poky rev: 6d67200d052ba72258f5a0a178542ef99500a9cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>