| Age | Commit message (Collapse) | Author |
|---|
|
Move classes to classes-global or classes-recipe as appropriate to take
advantage of new bitbake functionality to check class scope/usage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As stated in our top level license files, the license is MIT unless
otherwise stated. Add SPDX identifers accordingly. Replace older
license statementa with the standardised syntax. Also drop "All
Rights Reserved" expression as it isn't used now, doesn't mean anything
and is confusing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In bc, we found that there are files newer than Changelog (e.g. scan.l)
which means after install runs, the timestamp is clamped to SDE which means
"bitbake bc -C compile" would then try and regenerate generated files and
complain flex was missing.
Rather than assuming Changelog/NEWS are magic for dates, drop that scan
method and scan all unpacked files. This shouldn't be that slow as they'd
be in the disk cache already after an unpack.
For bc, this changes SDE by about two minutes and avoids the problems
rebuilding.
Add a version comment to the task definition as changes in python library
code won't trigger a rebuild and we need one here.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The test for obsolete licenses used in INCOMPATIBLE_LICENSE_EXCEPTIONS
tried to match the "<package>:<license>" tuples with the obsolete
licenses and thus never matched anything.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reduce code duplication to a common function in oe.buildcfg.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a load of duplicated git/layer/revision code which makes
most sesne as a python library, not bbclass code. Start to refactor as such.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We'd like to intercept git calls but we don't want circular references
and HOSTTOOLS currently sets them up. Tweak to avoid them.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're going to use the environment approach for solving this issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In a devshell, recent versions of git will complain if the repo is owned
by someone other than the current UID - consider this example:
------
bitbake -c devshell linux-yocto
[...]
kernel-source#git branch
fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source
kernel-source#
------
Of course the devshell has UID zero and the "real" UID is for "paul" in
this case. And so recent git versions complain.
As the whole purpose of the devshell is to invoke a shell where development
can take place, having a non-functional git is clearly unacceptable.
Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I
suggested we probably will see other similar instances like this and should
make use of PATH to intercept via devshell wrappers - conveniently we already
have examples of this.
Here, we copy the existing "ar" example and tune it to the needs of git to
combine Richard's suggestion and mine.
As such we now also can store commit logs and use send-email with our user
specific settings, instead of "root", so in additon to fixing basic
commands like "git branch" it should also increase general usefulness.
RP: Tweaked the patch so the PATH change only applies to the devshell task
and is a generic git intercept rather than devshell specific.
RP: Also apply the PATH change to do_install tasks since that also runs under
fakeroot and several software projects inject "git describe" output into
their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing
reproducibility issues from systems with different git versions.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code was for old versions of bitbake which we're now long past. Drop it
and simplify the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
We can re-trigger this code and there is little point in stacking a ton of duplicate
paths which just waste time during searches for modules. This could in theory alter
layer module search order but that seems unlikely in common use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Update internal variable names to improve the terms used.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On modern Power systems `uname -m` yields 'ppc64le' while the toolchain
knows the architecture as 'powerpc64le'. Provide a mapping from one to
the other to integrate with the existing architecture configuration
flags.
arch_to_rust_arch() only exists to map the OE *_ARCH variables before
any further processing, unlike arch_to_rust_target_arch() which is
specific to the internal triple handling of rust.
On Linux ppc64le systems the changes give the following config:
```
$ cat ./tmp/work/ppc64le-linux/rust-native/1.58.0-r0/targets/ppc64le-linux.json
{
"llvm-target": "powerpc64le-unknown-linux-gnu",
"data-layout": "e-m:e-i64:64-n32:64-v256:256:256-v512:512:512",
"max-atomic-width": 64,
"target-pointer-width": "64",
"target-c-int-width": "64",
"target-endian": "little",
"arch": "powerpc64",
"os": "linux",
"env": "gnu",
"vendor": "unknown",
"target-family": "unix",
"linker": "gcc",
"cpu": "generic",
"dynamic-linking": true,
"executables": true,
"linker-is-gnu": true,
"linker-flavor": "gcc",
"has-rpath": true,
"has-elf-tls": true,
"position-independent-executables": true,
"panic-strategy": "unwind"
}
```
Change-Id: Ief0c01189185d7d4da31d307270bec4e1de674ca
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want to use SPDX identifiers in LICENSE variables. There is now a
conversion script to make most of the translations. Add a list of
strings which have been replaced so we can show warnings to users
if they're still used anywhere.
Add checks to the package as insane check. This is currently a warning
by default but can be turned off or made an error as per the other standard
checks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This re-writes the INCOMPATIBLE_LICENSE checking code to replace
the WHITELIST_<lic> with
INCOMPATIBLE_LICENSE_EXCEPTIONS = '<pkg>:<lic> <pkg>:<lic> ...'
This initial change leaves most of the code structure in place,
but the code in base.bbclass needs to be re-written to make
the check more consistent around packages (PKGS) and not recipe
names (PN). This also is taking into account the changes for SPDX
licenses.
The aim is to provide a mode consistent variable where the variable
name is known and can easily be queried.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This matters when cross-compiling a cross-toolchain:
we need to specify the system where the toolchain
will run, not the system it will produce output for.
For everything else, HOST and TARGET are the same.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By renaming LICENSE_EXCLUSION to _exclude_incompatible, it makes it
clear that this is an internal variable.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add further tweaks to comments and variable names for license variable change.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove the old class and rename VarFlag to SKIP_RECIPE, handling this
in base.bbclass for efficiency. This means a separate inherit is no longer
needed.
This change better describes what the VarFlag is doing since it
is implemeted with the SkipRecipe() function.
By moving this into base.bbclass we simplify the distro inherit.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After other variables were renamed in bitbake, update OE-Core to match.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the newly added network task flag against tasks where network
access is expected. This is do_fetch, do_checkuri, do_testimage, do_testsdk
and do_testsdkext.
We can't disable networking in sstate tasks due to sstate downloads and
also so we can report hash equivalence to the server so network access
is enabled in sstate tasks.
Access within build-appliance do_image is also allowed due to the use
of pip, this is a poor example made rather obvious now and needs to be reworked.
Network access anywhere else in any other task isn't allowed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add repo-native as prerequisite for the repo fetcher.
Signed-off-by: Jasper Orschulko <Jasper.Orschulko@iris-sensing.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To try and avoid parse/memory overhead of functions within bitbake,
move the bulk of the reproducibility functions to the function library.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Exceptions print more clearly using repr() instead of str(), fix
in fetch and unpack tasks.
Drop part of the test which no longer makes sense after this change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code pattern isn't very pythonic, improve it to drop the unneeded
len() calls.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable zstd PACKAGECONFIG knob for squashfs-tools and add support for zstd
compressed initramfs and squashfs.
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using prepend as part of a function name is a poor choice. Whilst we're
about to make the syntax explict, improve the names anyway making the
conversion easier and the intent clear that this isn't an override.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that 022 is the default in BB_DEFAULT_UMASK in bitbake.conf, we
don't need any of these task flags, clean up.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
decodeurl() has limitations, primarily that it doesn't handle query
parameters at all. If a SRC_URI looks like this:
http://example.com/download.tar.gz?something
Then the returned path attribute is download.tar.gz?something. This means
the filename extension detection fails and required tools are not added
to the dependencies.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pseudo-native is a bit special. It conditionally compiles in support for
xattr, statx and statvfs amongst other options. If a pseudo-native binary is
used on a system where these functions are present but it wasn't compiled in
we see hard to debug permissions problems.
An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target
oe-selftest which shows a cryptic error:
File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target
self.assertEqual(filelist1, filelist2)
File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual
self.assertSequenceEqual(list1, list2, msg, seq_type=list)
File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual
self.fail(msg)
File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail
raise self.failureException(msg)
AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8']
First differing element 0:
'-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor'
'-rwxr-xr-x root root /etc/init.d/mdmonitor'
This is due to a version of pseudo without statx being used on a system where
ls uses statx, hence the files are displayed as 6000.6000 instead of root.root.
Avoid this by always building pseudo-native for the specific distro in question rather
than using a universal sstate feed.
This hopefully fixes one of the mysterious AB-INT issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* to make sure we won't hit such corner cases in future add a warning
to prevent trailing slashes and duplicated slashes even when they
in most cases don't cause harm
* only a few cases were found in layers included in my world builds:
oe-core: 1
meta-oe: 7
meta-python2: 1
meta-qt5: 1
meta-aws: 1
will send patches for these once this warning is approved for oe-core
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
as strings
* cannot use os.path.samefile, because S/B might not exist at this time yet
* there is issue with PSEUDO_IGNORE_PATHS when some recipe sets e.g.
S = "${WORKDIR}/"
whole WORKDIR gets added to PSEUDO_IGNORE_PATHS and then the build
can fail with various strange errors, in my case do_package was
failing when do_package calls:
fix_perms(.../1.0-r0/package/etc, 755, 0, 0, /etc)
and fails with "[Errno 1] Operation not permitted:"
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,
This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.
There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.
This patch fixes a few corner cases in OE-Core when used with this new
ignore list:
* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
user so we don't want the file ownership information in the hashequiv outhash
calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
by bitbake. We therefore need to trigger reparsing if this changes, which means
SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
Rework the variables to handle this. This otherwise breaks some of our sstate
tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are issues with multilib due to the ordering of events where some
functions see the remapped multilib dependencies and some do not. A significant
problem is that the multilib class needs to make some changes before key expansion
and some afterwards but by using existing event handlers, some code sees things
in a partially translated state, leading to bugs.
This patch changes things to use a new event handler from bitbake which makes the
ordering of the changes explcit.
The challenge in doing this is that it breaks some existing anonymous python and
dyanmic assignments. In some cases these used to be translated and no longer are,
meaning MLPREFIX has to be added. In some cases these are now translated and the
MLPREFIX can be removed.
This change does now make it very clear when MLPREFIX is required and when it is
not, its just the migration path which is harder. The patch changes the small number
of cases where fixes are needed.
In particular, where a variable like RDEPENDS is conditionally extended (e.g.
with an override), MLPREFIX is now required.
This patch also reverts:
base: Revert 'base.bbclass: considering multilib when setting LICENSE_EXCLUSION'
This reverts 6597130256a1609c3e05ec5891aceaf549c37985 as the changes
to multilib datastore handling mean its no longer necessary.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move logic checking that all packages licenses are only a subset of
recipe licenses from base.bbclass to the insane.bbclass so that it's
evaluated only once, during do_package_qa.
As explained in the linked bugzilla entry, if a package license is not
part of the recipe license, the warning message gets shown an
unreasonable amount of time because it's evaluated every time a recipe
is parsed.
[YOCTO #10130]
This also makes it possible to silence this error with INSANE_SKIP.
Signed-off-by: Quentin Schulz <quentin.schulz@streamunlimited.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are mutually exclusive PACKAGECONFIGs in recipes. Though it
declares that package configs are exclusive, it can't prevent users to
set them at same time. Extend PACKAGECONFIG to support specifying
conflicted package configs.
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the test depends on which DISTRO_FEATURES and targets are selected.
Similar to SOURCE_MIRROR_FETCH, add PARSE_ALL_RECIPES to allow the test to see
a much wider range of them. This avoids issues added by the new ucontext musl
specific recipe and allows the current whitelist to be cleared out a bit (more
cleanup can follow).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When some hosttool is a dead symlink, bitbake any recipe will
fail with error like below.
FileExistsError: [Errno 17] File exists: '/usr/bin/chrpath' -> '/path/to/builddir/tmp-glibc/hosttools/chrpath'
So we remove dead symlink under hostools/ directory to avoid
such error.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of reporting ${LICENSE} when a package is identified as using
an incompatible license, report the license(s) that are actually
incompatible.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After a number of rewrites, the code checking if a package has been
whitelisted for an incompatible license was calculating the
whitelisted packages twice (as 'whitelist' and as 'incompatwl').
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This helps fix an issue where python interpreter in hg script could
overflow the BINPRM_BUF_SIZE which is 128 on most of systems, because
interpreter is hardcoded and build paths can be deep.
This patch helps, because now the absolute python interp path in hg can
be replaced with '/usr/bin/env python3' and it will ensure that python3
from native install is used instead of the one on host.
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_prepare_recipe_sysroot may perform groupadd, which requires pseudo.
However, do_prepare_recipe_sysroot does not depend on pseudo explicitly,
which sometimes causes a build error when building a recipe that adds
groups.
This issue only occurs when executing do_prepare_recipe_sysroot for a
recipe that adds groups before finishing a task that depends on pseudo
for a recipe that doesn't add groups.
Signed-off-by: Mattias Hansson <mattihn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Turn:
ERROR: Unable to parse Var <OE_IMPORTED[:=]>
Traceback (most recent call last):
File "Var <OE_IMPORTED[:=]>", line 1, in <module>
File "/media/build1/poky/meta/classes/base.bbclass", line 35, in oe_import(d=<bb.data_smart.DataSmart object at 0x7f1d941ad208>):
for toimport in oe.data.typed_value("OE_IMPORTS", d):
> imported = __import__(toimport)
inject(toimport.split(".", 1)[0], imported)
File "/media/build1/poky/meta/lib/oe/sstatesig.py", line 267, in <module>:
>class SignatureGeneratorOEEquivHash(SignatureGeneratorOEBasicHashMixIn, bb.siggen.SignatureGeneratorUniHashMixIn, bb.siggen.SignatureGeneratorBasicHash):
name = "OEEquivHash"
bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception AttributeError: module 'bb.siggen' has no attribute 'SignatureGeneratorUniHashMixIn'
into:
ERROR: Error importing OE modules: module 'bb.siggen' has no attribute 'SignatureGeneratorUniHashMixIn'
which can then trigger a version mismatch error message.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Peter Kjellerstedt
Paul Gortmaker
Saul Wold
Andrew Jeffery
Alexander Kanavin
Jasper Orschulko
Zoltán Böszörményi
Ross Burton
Martin Jansa
Quentin Schulz
Kai Kang
Chen Qi
Khem Raj
Mattias Hansson