| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Otherwise the common name udev-hwdb is only provided by systemd, meaning that
other recipes can't depend on a single name.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When we switched to recipe specific sysroots (rss), performance took a nose dive. Its
easy to blame rss but it turns out not to be entirely at fault.
Three configurations are compared here:
a) Pre-RSS (revision 45df694a9f472ac2f684aadac4d864c3dfdc48a7)
b) Post-RSS (revision 226a508da955439b881b2f0a544a3aee76e59919)
c) as b) with this change
Overall build times:
a) 22794.25user 2687.88system 30:32.84elapsed 1390%CPU (0avgtext+0avgdata 919056maxresident)k
b) 22677.25user 3238.79system 36:16.68elapsed 1190%CPU (0avgtext+0avgdata 918896maxresident)k
c) 23571.84user 3383.65system 31:36.83elapsed 1421%CPU (0avgtext+0avgdata 919068maxresident)k
For the overall build and sstate directories, du -s shows:
a)
3992588 build-pre-rss/sstate-cache
30804484 build-pre-rss/tmp
b)
4013272 build-with-rss/sstate-cache
36519084 build-with-rss/tmp
c)
4014744 build-with-rss2/sstate-cache
35336960 build-with-rss2/tmp
However more worryingly:
$ du -s build-pre-rss/tmp/sysroots/
2506092 build-pre-rss/tmp/sysroots/
$ du -s build-with-rss/tmp/sysroots-components/
3790712 build-with-rss/tmp/sysroots-components/
$ du -s build-with-rss2/tmp/sysroots-components/
2467544 build-with-rss2/tmp/sysroots-components/
These numbers *should* be equivalent but as you can see, b) is ~1.2GB larger. The reason turned out
to be patchelf. Taking a specific binary from a specific recipe, bc from bc-native, in a) its 82kb
(stripped) yet in b) its 2.17MB.
$ ./patchelf --set-interpreter /bin/rp bc
warning: working around a Linux kernel bug by creating a hole of 2084864 bytes in ‘bc’
https://github.com/NixOS/patchelf/blob/master/src/patchelf.cc#L710 shows that this "hole" is just
padded zeros using memset, its not a proper sparse hole.
This patch copies files with cp --sparse=always after modifying them with patchelf, then replacing
the original file. The better fix will be to fix this in patchself itself and seek() there
when writing the new file but that means new uninative tarballs and will take a bit of work
so I'm proposing this workaround in the meantime.
Also, this patch drops error handling since subprocess check_output() tracebacks will print this
information if the command fails so we can simplify the code.
|
|
Used options.outdir instead of image_output_dir.
There is no sense to use extra variable for this.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
It was a typo in current code: mktemp was used instead of
mkdtemp to create work directory. This is fixed by using
mkdtemp.
Create work directory as a subdirectory of output directory
to make sure both are on the same partition to make moving
of result image faster.
This also fixes possible disk space issues as mkdtemp uses
TMPDIR, TEMP or TMP environment variables to get default value
of its 'dir' parameter. Those variables are usually pointing
to /tmp, which is not the best location to create huge images.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Make sure output directory exists before creating an image.
Create it if it doesn't exist.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
libudev will be autodetected by the linkage, the intention here was to depend on
udev-hwdb to ensure that the USB ID lists are installed.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Currently test_postinst_roofs_and_boot is building a full-cmdline image
this is taking a lot of time to execute the test, so a minimal image
will be build instead.
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
In order to prevent multiple instances of the script running at the same
time.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This module contains singe function serial_console_form_kargs, which
is used only by rootfs_pcbios_ext plugin. Moved it there and removed
syslinux module to make it easy to find and mainain plugin code.
[YOCTO #10619]
|
|
Fixed indentation, unused imports, trailing lines etc.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Removed unused enable_logstderr and disable_logstderr APIs.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
The code deals with non-existing directory
and can be removed.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Flattened directory structure:
moved wic/utils/oe/misc.py -> wic/utils/misc.py
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Moved find_canned and get_custom_config APIs to engine module.
Removed empty wic.utils.misc module.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This API is not used in wic code.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Replaced parse_sourceparams function with list comprehension.
Used local variables instead of attributes.
Moved global variable to the local scope.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This file is not used anywhere in the wic code.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
As this files is located in plugins/imager subdirectory it's
obvious that it's an imager plugin. Renamed to direct.py to
be consistent with plugin naming scheme.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Removed unused methods.
Got rid of get_default_source_plugin and _full_name methods.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Fixed wrong continued indentation, unused import and
trailing new line pyling warnings.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Synchronized attribure names in DirectImageCreator and
DirectPlugin for better readability. Simplified code,
removed unneeded global variable disk_methods.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Replaced class method __rootfs_dir_to_dict with a list
comprehension.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This move simplifies directory structure and makes
further refactoring easier. The code from direct.py was used
only in direct_plugin, so it's safe to move it there.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Removed as they're not used anymore in wic code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
Got rid of using configmgr global object in direct_plugin and direct
modules. It was used to implicitly parse kickstart file and set
couple of variables.
Replaced usage of configmgr by passing parameters directly to the
DirectImageCreator.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
confmgr object is complicated for no reason and almost
useless as all configuration info comes from command line and
bitbake variables. It's used it creator.py to store information
about output directory, logs and some never used functionality
like tmpfs for future use, which doesn't actually happen.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
Extra configuration data should be write using the write_config
method instead of manually appending to the local.conf file
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Removed hddimg from FSTYPES in wic test suite as
wic doesn't depend on hddimg anymore.
[YOCTO #10835]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The sysroot of wic-tools is needed for wic, but if rm_work is enabled,
it will be removed before wic has a chance to use it, hence this fix.
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
|
|
When building for the target, pkg-config uses the target glib-2.0 instead of
it's own minimal fork. To find this it needs to use pkg-config so ensure this
dependency exists in case it doesn't exist on the host already.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This puts the dependencies on the correct task and removes pointless
noexec tasks allowing for a slightly cleaner task structure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The number of mkdir calls was showing up high on the profile charts since
it was getting called once per file which is excessive. Each call results
in one or more syscalls which is bad for performance. Cache which
directories we've seen to reduce the calls to a more reasonable number
and speed up recipe specific sysroots.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
According to profile data, repeated calls to bb.debug and bb.note in
the extend_recipe_sysroot() codepath were accounting for 75% of the time
(1.5s) in calls from tasks like do_image_complete.
This batches up the log messages into one call into the logging system
which gives similar behaviour to disabling the logging but retains the
debug information.
Since setscene_depvalid is also called from bitbake's setscene code,
we have to be a little creative with the function parameters and leave
the other debug output mechanism in place. This should hopefully
speed up recipe specific sysroots.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't need to hardcode a path to tail, follow the other tools examples and
don't specify a path since PATH is good enough for us.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't put target libs into a native/cross ${exec_prefix} but having
this in the default search path means all linker scripts have to be relocated.
This is a considerable chunk of files to create multiple copies of for no good
reason.
Instead, patch out the paths we don't need.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Relocation of native .la files during recipe specific sysroot relocation
is probably the final straw in just killing these files off.
Change things so this class is inherited by default. If distros don't want to
do this, they can opt out but it seems like the best thing to do now since
.la files aren't needed on Linux.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The native .pc files currently have hardcoded paths in them meaning each has
to be relocated at final install time. pkg-config has built in functionality
to avoid this, namely the pcfiledir variable.
This function translates .pc files to use the variable meaning further
relocation later is unnecessary.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst the path to the executable is dynamically determined, the passed in
environment variables or parameters are not relocatable and rely on the sstate
remapping code. In the recipe specific sysroot case this has become more costly.
This patch translates such paths into relocatable entries which means that
a sed replacement at final installation isn't needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The run-tests option is optional so if isn't specified set
to None instead of crash on split().
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.
[YOCTO #10964]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are some missing suites on sato-sdk and lsb-sdk images so add it.
The createrepo-native needs to be built before run testimage in
order to create a repository from install packages.
The DL_DIR data also needs to be updated from the current bitbake
environment instead of use the value from testdata file.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
gcc: Removed unneded lines.
multilib: fixed case number.
syslog: added a missing test.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This fixes a potential pollution by the build host and build error
when yacc isn't installed on the build host:
| ../../libtasn1-4.9/build-aux/ylwrap: line 175: yacc: command not found
| Makefile:1116: recipe for target 'ASN1.c' failed
| make[3]: *** [ASN1.c] Error 127
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
All warnings start with "possible bashism in", followed by one or more
(in the case of line continuation) lines of source code. To support
more than one line, we now split by matching against the known intro
text.
Example:
$ verify-bashisms guile
...
/.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb
possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"):
echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \
> ${B}/guile-config.cross
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Found via verify-bashisms.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Andre McCurdy
Ross Burton
Richard Purdie
Ed Bartosh
Ming Liu
Jose Perez Carranza
Markus Lehtonen
Leonardo Sandoval
Kristian Amlie
Aníbal Limón
Mariano Lopez
Patrick Ohly