aboutsummaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2
AgeCommit message (Collapse)Author
2016-01-13bitbake: Revert "fetch2/local.py: avoid using PREMIRROR"Richard Purdie
This reverts commit e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c. This is in fact a valid use case, for example the sstate.bbclass code sets up SSTATE_MIRRORS as PREMIRRORS. Its quite common to map those file:// urls to remote http:// urls and with the above change, this no longer works. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11bitbake: fetch2/git.py: Add missing "errno" module import.Kristian Amlie
This goes undetected most of the time, but when updating a repository, if the ud.fullmirror file is not present, you end up getting an exception instead of carrying on because the errno module is not loaded (specifically "if exc.errno != errno.ENOENT"). (Bitbake rev: e6fca8480731ce817df9bee61438347a5e3d3017) Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08bitbake: fetch/git: Change to use clearer ssh url syntax for broken serversRichard Purdie
Some servers, e.g. bitbucket.org can't cope with ssh:// as part of the git url syntax. git itself is happy enough with this but you get server side errors when using it. This changes the git fetcher to use the more common ssh url format which also means we need a : before the path. Seems a shame to have to do this due to broken servers however it should be safe enough since this other form is the one most people use on the commandline so it should be safe enough. [YOCTO #8864] (Bitbake rev: 4193e99adce8e88f12ac88d7578ad39575f7e346) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06bitbake: fetch2/__init__.py: Add support for 7-ZipJuro Bystricky
7-Zip is a file archiver claiming the highest compression ratio. This patch allows using 7-Zip commpressed files in bitbake recipes. Two common formats are supported: SRC_URI = "file://abc.tar.7z" SRC_URI = "file://abc.7z" (Bitbake rev: 7120f5bfaae54e91bc95da5667831424724ce613) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06bitbake: fetch2/local.py: avoid using PREMIRRORRobert Yang
The PREMIRROR isn't useful for "file://", so avoid using it, this is good for searching speed and can reduce useless lines in log.do_fetch. (Bitbake rev: e130dca85bac82bd4d88f94a6bf9fe36e8ad4d7c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18bitbake: fetch: use orig localpath when calling orig methodChristopher Larson
When a mirror tarball is fetched, the original fetch method is called, which unpacks the mirror tarball. After the original method is called, it checks the localpath of the mirror tarball rather than the clone path, which isn't ideal, particularly if the mirror tarball was removed due to being out of date. We know the original fetch method will do what it needs to do to get its content in the form it needs from the mirror tarball, so we can use its localpath instead. (Bitbake rev: 1732ad65d6c7d67b7d07cb30c074f5016adadbea) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08bitbake: bitbake: rename REGEX, REGEX_URI, and GITTAGREGEX.Alexander Kanavin
Rename REGEX to UPSTREAM_CHECK_REGEX, REGEX_URI to UPSTREAM_CHECK_URI, and GITTAGREGEX to UPSTREAM_CHECK_GITTAGREGEX to better reflect their purpose and to reflect a common namespace. (Bitbake rev: f0a9e783f9969573fd74edfa241ef14f18ac684e) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-07bitbake: wget.py: parse only <a> tagsAlexander Kanavin
For two reasons: 1) The important one: we hit the following bug when doing upstream version checks on some webpages: https://bugs.launchpad.net/beautifulsoup/+bug/1471755 2) Also, documentation for beautifulsoup states that memory usage and speed is improved that way. (Bitbake rev: 7546d4aeb3ba8fda9832081b84d93138dc5e58d6) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16bitbake: fetch2: Remove crazy code in unpackRichard Purdie
This looks reasonable until you realise self.localpath is a function. Data expansion of something which isn't a string is the original value so this code just wastes CPU cycles and makes no sense. Remove it. (Bitbake rev: 37214ea9bf484998b75dbc1200d53f1afc5257ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27bitbake: fetcher: svn: Add support for checkout to a custom pathJens Rehsack
Add support for the Subversion fetcher to checkout modules to a custom path than the module name to avoid checkout is always module - svn is path based and tag/branch-checkout might break builds because of invaid path specs. (Bitbake rev: af88f538e61afa1b115be4d7afe00d8477f61750) Signed-off-by: Jens Rehsack <sno@netbsd.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-16bitbake: fetch2/hg: Include missing errno importLogan Buchy
errno.ENOENT checked if deletion of the fullmirror fails. Exception was thrown since module not imported (Bitbake rev: d92ebfc34b69ad5df2d151e6b8299fbb5afa3e5f) Signed-off-by: Logan Buchy <logan.buchy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-19bitbake: bitbake: bb.fetch2.git: Import errno moduleRomain Perier
Currently this module is dereferencing errno.ENOENT but the python module "errno" is not imported, which causes a crash when fetching from a git repository. (Bitbake rev: 93e4c9bb2393b1074f5a01e7eaaac742a59d8086) Signed-off-by: Romain Perier <romain.perier@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: bb.fetch2.{git, hg}: remove tarball if it needs updatingChristopher Larson
We were maintaining state in the form of ud.repochanged to determine whether we need to write the tarball in the case where the tarball already exists, but this state is maintained only in memory. If we need to update the git repo, set ud.repochanged to True, and then are interrupted or killed, the tarball will then be out of date. Rather than maintaining this state, simply remove the out of date tarball when we update the git repo, and it will be re-created with updated content in build_mirror_data. [YOCTO #6366] (Bitbake rev: eaaa81393f181432c8586b17ade623f42c9fed2e) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: tests/fetch.py: Fix recursion failure in url mappingMark Hatle
Instead of reproducessing the same line over and over and over, we remove the current line from the mirror list. This permits us to re-evaluate the list while excluding all matches that have previousily occured. Without this fix, adding this test results in a failure: RuntimeError: maximum recursion depth exceeded in cmp (Bitbake rev: 24a8e9a5b0ba145ae589178d74365c986ebca325) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: fetch2/__init__.py: uri_replace regex handlingMark Hatle
We should only substitute one time. If we do it without a max count, we can end up matching over and over. Before this change: https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz with a mirror of https://.*/[^/]* http://AAAA/A/A/A/ would end up either recursing indefinitely or result in: http://AAAA/A/A/A/A/A/A/A/A/A/bitbake-1.0.tar.gz (Bitbake rev: 4d254e02e2867dd9a6663508c8ca9f2733af71a8) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch: don't remove the clone when an update failsChristopher Larson
When our clone exists, but is out of date, and the attempt to update it fails, we don't necessarily want to remove the entire clone, particularly if it's a large repository. (Bitbake rev: 19af272ba5256653edeff6acbceeb09e3e478d61) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch: handle checksums consistently for mirrorsChristopher Larson
If the main fetch method doesn't support checksums, the user will not be defining them in the recipe, so we don't want to check them for premirrors/mirrors either. This ensures that we never error due to missing checksums on a git mirror tarball. (Bitbake rev: 24c79bbed361b37f12d3351af13602e3d4386f4c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch.git: remove leading '.' from gitsrcnameChristopher Larson
When using an absolute file URI, there's no host, and the path starts with '/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden. Remove any leading '.' to fix this. (Bitbake rev: 8dce6964d56b36a77fb113f2ad496cc992a5ff36) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: Fix default function parameter assignment to a listPaul Eggleton
With python you should not assign a list as the default value of a function parameter - because a list is mutable, the result will be that the first time a value is passed it will actually modify the default. Reference: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments (Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: wget.py: fix incorrect regexesAlexander Kanavin
[\.-_] means "any character between . and _" What was meant here is certainly "any character from the three characters .-_" (Bitbake rev: af13eaba627f199f91c048c435b9dbe19c79527f) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22bitbake: fetch2/wget.py: latest_versionstring now returns (version, revision)Aníbal Limón
Now latest_versionstring method returns (version, revision) for comply the new return convention needed by SCM's like git get the current revision. bb/tests/fetch.py: Updated wget latest_versionstring test for comply new convention. [YOCTO #7605] (Bitbake rev: 8d454646cbe1b04758ca178d8c6fcfd02b818b7b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22bitbake: fetch2/git.py: latest_versionstring now returns (version, revision)Aníbal Limón
We need to know the revision associated with the upstream version in SCM like git, this change broke return convention, oe-core recipeutils get_recipe_upstream_version need to be updated. tests/fetch.py: Updated git latest_versionstring test for comply new convention. [YOCTO #7605] (Bitbake rev: fd175dc90024c503134c11cbd83e77d88c406ac8) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22bitbake: fetch2/git.py: latest_versionstring search in all tagsAníbal Limón
Don't limit the tag search for only tags end with ^{}. (Bitbake rev: 7006ab313766344cf33481228465082ed5977d28) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: fetch2/wget.py: checkstatus disable SSL cert validation.Aníbal Limón
Since Python 2.7.9 ssl cert validation is enabled by default see PEP-0476, this causes verification errors on some https servers so disable by default. (Bitbake rev: e177170200ece76b36e3f7d5597651fdef67736f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: fetch2/wget.py: checkstatus fix using proxy handlerAníbal Limón
Only use ProxyHandler opener when exists proxies in env. (Bitbake rev: 0f062c5d99e12ce20c0e46a8e602448032144ff6) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: fetch2/wget.py: Add support of connection cache in checkstatus.Aníbal Limón
fetch2/__init__.py: Add connection_cache param in Fetch __init__. In order to pass connection cache object to checkstatus method. [YOCTO #7796] (Bitbake rev: 9fa6407e6cefe66c77467419a8040d6957a6bb01) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: fetch2/sftp: Set BatchMode=yes as argument to sftpOlof Johansson
Set BatchMode=yes instead of PasswordAuthentication=no. This will make sftp fail immediately, not only when SSH requires interactive authentication, but also on errors related to host key verifcation. (Bitbake rev: 31305853a177735cc9c4553ea8905cd0acfcb100) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: git.py: create a branch when checkoutRobert Yang
* Create a branch and named as upstream branch when checkout source * Set the branch to track remote branch. (Bitbake rev: 1ba20e4fe9c884515b200589fe379ad5eeda10bd) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09bitbake: fetch2: Add fetch parameter to checkstatusAníbal Limón
In order to pass connection cache object to checkstatus function add fetch parameter. (Bitbake rev: fbb9c6f5538084e125b58118a86968908e6f895b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09bitbake: fetch2/__init__.py: Add FetchConnectionCache classAníbal Limón
FetchConnectionCache class acts as a container for socket connections useful when implement connection cache into fetcher modules. (Bitbake rev: 454da2cd17539ceb9caad6d76f034757e44ee12f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-07bitbake: fetch2: Checkout to correct ref begore init and update submodulesFelipe F. Tonello
This is nessary when specified branch with submodules is different then default (master) branch. [YOCTO #7771] (Bitbake rev: f7b0b5e33e00f3ce0744322eee93835ee76bf184) Signed-off-by: Felipe F. Tonello <eu@felipetonello.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27bitbake: bitbake: fetch2/hg.py: fix unpack error and mirror tarballRobert Yang
Fixed: * do_unpack error: abort: repository DL_DIR/hg/vim.googlecode.com/hg/vim not found! * The mirror tarball doesn't work - Add the build_mirror_data to create the tarball - Unpack the mirror tarball when needed * The hg files will put in the dir like git: DL_DIR/hg, it was DL_DIR/hg/path/to/src/uri/path in the past. (Bitbake rev: 378647281ae883dd726f1e1b775dd35ef6a0e8d1) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27bitbake: fetch/perforce: Fix single file checkoutsRichard Purdie
Fetching directories with p4 worked but single files did not. This patch from Helmut Auer (helmut.auer@harman.com) fixes that issue. [YOCTO #7891] (Bitbake rev: 39da6579901c62a83ed9319c2016c58fbbc108fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23bitbake: bitbake: Add explict getVar param for (non) expansionRichard Purdie
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-26bitbake: fetch/hg: Disable checksums for archived downloadsAsh Charles
Like the Git fetcher, the Mercurial fetcher shouldn't expect recipes to provide a checksum. As described [1], recipes using a mercurial fetcher that don't provide a checksum will fail in a the repository has previously been downloaded and archived. Credit to Rafaël Carré for figuring out the bug. [1] https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg41328.html (Bitbake rev: 2df35a25b4968f64adfa673d5b73442c1a30829d) Signed-off-by: Ash Charles <ashcharles@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24bitbake: fetch2/git.py: Add gitpkgv_revision alternative version informationMike Looijmans
gitpkgv_revision returns a sortable revision number that can be used in the PKGV variable for example. To mimic meta-openembedded gitpkgv behaviour to provide a sortable revision numner, one could set the following: PKGV = "1.0+${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}" This would yield a package version like "1.0+69+fb5eb80". (Bitbake rev: 989c08f62aff7b707c25c692c23284f16506b7bc) Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24bitbake: fetch2/__init__.py: Make get_srcrev output configurableMike Looijmans
The idea here is to support package version numbering similar to gitpkgv in meta-openembedded. This commit is the first step towards such functionality. The original plan was to add a "get_pretty_srcrev" method to the fetcher, as per Richard's suggestion [1]. While writing this, I noticed that it would become a copy of get_srcrev with only two lines changed. So to create something more Pythonic than a boolean argument and conditionals around the calls to the fetcher's sortable_revision, I just made the method to be called on the fetcher an argument to the method. Defaulting to 'sortable_revision' prevents affecting existing code. Now if the git fetcher were to implement, say 'gitpkgv_revision' one could set the following in a recipe: PKGV="1.2+${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}" and this would yield the same result as gitpkgv's GITPKGV variable. See for the discussion leading to this change: [1] http://lists.openembedded.org/pipermail/openembedded-core/2015-January/100345.html (Bitbake rev: 2f1f4483493cc290f5d2c07f9906e90eaea2f4c1) Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19bitbake: fetch/git: Fix uri in git checkstatusMarcin Smoczyński
Pass proper repository url without arguments after a semicolon. Executing checkuri on a rule with git repository in SRC_URI does not report errors when working offline because wrong repository url is passed to the ls-remote command. For example "bitbake -c checkuri glibc" command executes: "git -c core.fsyncobjectfiles=0 ls-remote git://sourceware.org/git/glibc.git;branch=release/2.21/master" command in a shell subprocess to determine if url is valid. Shell subprocess executes in fact 2 commands: "git -c core.fsyncobjectfiles=0 ls-remote git://sourceware.org/git/glibc.git" and "branch=release/2.21/master" First one returns 127 or 128 depending on error but second one returns 0 because it is just env variable setup. Therefore we're not catching connection error. [YOCTO #7558] (Bitbake rev: efa44d04137977f883db4a643b0f774e91514722) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19bitbake: fetch2/hg.py: add clean functionRobert Yang
Fixed when bitbake vim -ccleanall: File: '/path/to/bitbake/lib/bb/fetch2/__init__.py', lineno: 1462, function: clean 1458: def clean(self, urldata, d): 1459: """ 1460: Clean any existing full or partial download 1461: """ *** 1462: bb.utils.remove(urldata.localpath) 1463: 1464: def try_premirror(self, urldata, d): 1465: """ 1466: Should premirrors be used? File: '/path/to/bitbake/lib/bb/utils.py', lineno: 633, function: remove 0629: subprocess.call(['rm', '-rf'] + glob.glob(path)) 0630: return 0631: for name in glob.glob(path): 0632: try: *** 0633: os.unlink(name) 0634: except OSError as exc: 0635: if exc.errno != errno.ENOENT: 0636: raise Exception: OSError: [Errno 21] Is a directory: '/path/to/downloads/hg/vim.googlecode.com/hg/vim' (Bitbake rev: 02763306662e15a4750395e5eab64ba98d1f9939) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16bitbake: fetch2: Improve mirror building function error handlingRichard Purdie
First, when building mirror urls we don't do any fetching so we should never be calling clean functions. Currently, if a mirror url fails, we don't process it further to see if there are any mirrors of the mirror. We should do this even when the mirror url fails, else we may miss out on valid/useful mappings, particularly in the case of file:// urls. (Bitbake rev: b7fd3ec9994f664b17fc86423e6e7afac07e897b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16bitbake: fetch2: Allow GIT_SMART_HTTP to be passed throughLiam R. Howlett
Allow GIT_SMART_HTTP to be passed through to the fetch command so that servers that cannot use GIT_SMART_HTTP can be used by the fetcher. (Bitbake rev: e5c97a85bed0436d48eeaac2e32962cfb5371d2f) Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15bitbake: bitbake: fetch2/git: Add URL when latest_revision generates an ↵Mariano Lopez
exception. The URL is not sent when _latest_revision generates and exception. When performing the sanity checks it is not possible to know the URI that failed. This add the URL when latest_revision generates an exception. [YOCTO: #7592] (Bitbake rev: 9f2115b07a55cb14e4a74dc6fbd3707c28a234d0) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15bitbake: fetch2/git: ensure the unpacked origin remote points upstreamPaul Eggleton
If you're interested in using the checked out repository for development (e.g. in OE with devtool) then you ideally want the origin remote to point to the repository it was fetched from, so just set that after cloning. (As part of this I did a minor refactor so we have one function to generate the repository URL, which was already in two places.) Fixes [YOCTO #7756]. (Bitbake rev: 80ecd1c54d4c748cee3a7ce0d64013a346e7671e) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12bitbake: fetch2: Add BB_ALLOWED_NETWORKS supportLiam R. Howlett
BB_ALLOWED_NETWORKS is a list of hosts that the fetcher will be allowed to use when BB_NO_NETWORK is not set. If BB_NO_NETWORK is set, then networking is still disabled. If BB_ALLOWED_NETWORKS is not set, the behaviour remains the same as today. If BB_NO_NETWORK is NOT set, and BB_ALLOWED_NETWORKS is configured, then only the hosts in the list are usable by the fetcher. eg: BB_ALLOWED_NETWORKS="yoctoproject.org git.gnu.org" The fetcher will be able to download from yoctoproject.org, git.gnu.org, but not ftp.gnu.org or any other hostname that is not in the list. There is also limited support for wildcards on the beginning of the hosts, so BB_ALLOWED_NETWORKS="*.gnu.org" with match git.gnu.org and ftp.gnu.org as well as foo.git.gnu.org (Bitbake rev: c7263096ba31ba45daeeb9de90c1cb9ebef24a28) Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12bitbake: fetch/hg: support submodulesDaniel McGregor
Use hg clone and hg pull to copy the source into the build directory rather than taring up the cloned repository and untarring in the destination. This allows submodules to be cloned. While here, make the default behaviour keep the hg scm data to match the behaviour of the git fetcher. (Bitbake rev: f002b1ca80cb542a4ed0c06c53c914cd5e076565) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12bitbake: fetch2: Allow git to use a different CA Bundle with private keysJason Wessel
Setting BB_ENV_EXTRAWHITE to include GIT_SSL_CAINFO is enough to get "git ls-remote" to work for recipe parsing but it is not enough to get the fetcher to clone properly. This is because the fetcher has its own idea about what variables should be exported in the environment for some operations. It is desirable to use alternate CA Bundles for internal testing prior to using public keys for https, so we should allow the GIT_SSL_CAINFO to pass through. (Bitbake rev: 40ff92282bbf32cf644b021bf7cbb1f393dbb856) Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-17bitbake: fetch/git: Remove a possible trailing '/' in subpathAnders Darander
If the subpath parameter to the git fetcher ends with a trailing '/', bb.utils.prunedir() will be called on '/'... Fixes [YOCTO #7620]. (Bitbake rev: 380a3fb372c8b0a53dd7528562e6e7a222dc76ef) Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-09bitbake: fetch2: Revalidate checksums, YOCTO #5571Clemens Lang
[YOCTO #5571] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=5571 The following workflow (whether accidentally or deliberately) would previously not result in a checksum error, but would be helpful to do so: - Write a recipe with correct checksums - Fetch the sources for this recipe using bitbake - Change the checksums Since the bitbake fetcher writes a done stamp after the initial download and does not verify the checksums again (even if they are changed in the recipe afterwards), the change of checksums is silently ignored. Fix this without the overhead of computing the checksums from scratch on every do_fetch by storing them in pickled format in the done stamp and verifying that they still match those in the recipe. (Bitbake rev: fbd4a0d422cf7f43db2f9eab2e47c41246a9031e) Signed-off-by: Clemens Lang <clemens.lang@bmw-carit.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-16bitbake: fetch2: wget remove scape of - in regexes don't neededAníbal Limón
(Bitbake rev: 8fa43245351f3ec0a5007b2742c08b7ef98e7879) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-16bitbake: fetch2: wget _modelate_version improvmentsAníbal Limón
Fix sustition for rc, beta and alpha releses from -N to N weight. (Bitbake rev: 63a9e60a6d80cfd2693ec1a6359785dc19f98e1f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>