| Age | Commit message (Collapse) | Author |
|---|
|
In collect_bbfiles() we're monkey-patching os.listdir in order to find
which directories to watch, and then undoing that when we're finished -
however if an exception occurred for any reason there was nothing to
ensure the latter occurred. This may not have caused any issues, but as
this kind of thing really ought to be secured using try...finally just
in case, so do that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The code that was supposed to watch directories along BBFILES for
creation of new files wasn't working in the case where the directory did
not initially contain any matching files - since in updateCache() we are
passing the directory path to add_filewatch() and the latter function
calls os.path.dirname() on the path on the assumption that it is a file
path, and thus the parent of the directory got watched but not the
directory itself. (If the directory wasn't empty everything worked fine
since add_filewatch() was called elsewhere with the path to one of the
files in that directory, and thus the directory got watched). Add a
parameter to add_filewatch() to tell it we are passing it directory
path(s) rather than file path(s).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Fixes [YOCTO #12031]
Applied minor wording changes based on review feedback.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
|
|
Fixes [YOCTO #12029]
There were four links to an old blog post that was leveraged
for the hello world example. The post has evidently been
removed and the links no longer resolved. I have taken them out.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
|
|
Fixes [YOCTO #12031]
Moved an inappropriate example from the bottom of the INHERIT
Configuration Directive section up into the inherit directive
section.
Updated the example at the end of the INHERIT Configuration
Directive section to apply to INHERIT.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
|
|
Fixes [YOCTO #12032]
I applied some re-writing to help clarify the relationships between
the inherit, include, and require directives.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
|
|
Printing the last 10 lines of bitbake-cookerdaemon.log when the server
fails to start can sometimes result in printing the output from a
previous run, which could lead the user completely down the wrong path
in terms of the cause of the failure. Use a known start text containing
the time which we can then look for when scanning through the log, and
then grab the last 10 lines of that part instead.
Fixes [YOCTO #11903].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The whitelist shouldn't have to be populated in order for the
enforcement to work properly - check if the list is not None in order to
determine whether the functionality is enabled or not since that is how
the function that sets up the list behaves.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Issues in start are not being correctly detected by the current readypipe
code. Change it to use specific "ready" or "fail" messages to correctly
determine the correct failure mode and avoid bitbake seeming to hang
(it does currently timeout eventually).
[YOCTO #12062]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update the Toaster stable release selection to 'Rocko' and
the bitbake version to '1.36'.
[YOCTO #12037]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Stop the pending build and report to the user if there is
an internal exception due to a git error, a bitbake server
error, or if the server-only mode halts due to for example
a user syntax error in a layer or recipe.
These exceptions were not caught because they occure before
the normal toastergui processing was started.
[YOCTO #12056]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there is a network failure the return value from latest_versionstring() is
('','') which later causes an exception when comparing versions.
Improve this by checking the return value and failing the test early.
[ YOCTO #12053 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since 2017-08-17 (git version 2.14.1.473.g3ec7d702a) using deprecated
git branch parameter "--set-upstream" causes a fetcher error. Replace
it by "--set-upstream-to".
https://git.kernel.org/pub/scm/git/git.git/commit/?id=52668846ea2d41ffbd87cda7cb8e492dea9f2c4d
says, it's deprecated since 2012-08-30 so hopefully all still supported
host distributions have new enough git to support "--set-upstream-to".
ERROR: PACKAGE do_unpack: Fetcher failure: ...;
git -c core.fsyncobjectfiles=0 branch --set-upstream master origin/master failed with exit code 128, output:
fatal: the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead.
ERROR: PACKAGE do_unpack: Function failed: base_do_unpack
Signed-off-by: Andre Rosa <andre.rosa@lge.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the UI disconnects, we can throw away any server-side remote
datastores we created in response to calls from the UI, and we *must*
drop everything in extraconfigdata or it will taint any future
operations.
Dropping extraconfigdata upon disconnect fixes taskhash mismatch errors
when running devtool.DevtoolTests.test_devtool_update_recipe_local_files
within oe-selftest with BB_SERVER_TIMEOUT=100 in OpenEmbedded.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you have a layer with a blank BBFILE_PATTERN the layer was ignored
when processing the list of layers with priorities. This list is not
only used for processing recipes, but also by additional programs such
as bitbake-layers show-layers.
Without this change, a layer that provides configuration or classes
only does now show up in show-layers, which is used by the
yocto-compat-layer.py script. This causes a failures in the compatibility
check.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we're implementing reset() in SignatureGenerator at all (and we need
to for a basic non-OE BitBake setup where that is the default signature
generator), then we need it to be clearing out the internal values
properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The list of columns in the many 'Edit Columns' pop-ups became unsorted
with the 'Toaster Table' implementation. These entries need to be
gathered and sorted in the column processing.
[YOCTO #12004]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The default layers are missing the recipe link definitions in
the fixture files, and because they are predefined they do not
get the updated information from the Layer Index.
[YOCTO #12006]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The file "lsupdates.py" is missing the import of 'ToasterSetting',
and that breaks setting up and updating Toaster and its database.
[YOCTO #12036]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If no external layers are defined, pokydirname is not set. Rectify
this by taking the 'be.sourcedir' as the pokydirname.
[YOCTO #12015]
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
One of the debug messages during build contains a list of all layers
but without spaces or other separators between them. Use pformat
instead.
[YOCTO #12014]
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The build page provides tables related to performance that
cover build time, CPU time and disk IO. The "Edit columns"
drop down does not allow selection of the Order column and
makes it hidden as well which is not accurate from user
interaction point of view.
This patch enables the hideable property for the Order column
so it is hidden by default but the user can enable it through
the drop down if need be.
[YOCTO #11040]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There must be at least one FSTYPE selected in the Toaster bitbake
variable editor page. When the user deselects all the "Save"
button gets disabled, but the error message is missing.
[YOCTO #8126]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BB_SERVER_TIMEOUT=100 oe-selftest -r bblayers
was failing and highlighted that since parseConfiguation clears data
structures, it needs to also clear parsecache_valid as it no longer
contains correct data.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
commandline option
I can't actually see how this was working, nothing connected the commandline option
to the data in TaskData(). Drop the remaining pieces of this option, it was a relic
from a decade ago and we want deterministic builds, not random tries until something
might work.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix failure after commit "cooker/siggen: Reset siggen when reparsing"
(e4c6ca9440f63761560b49bbe12654441f54687e) when executing without
specifying a BB_SIGNATURE_HANDLER.
Signed-off-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BB_SERVER_TIMEOUT=100 oe-selftest -r devtool.DevtoolTests.test_devtool_build_image
fails, the reason is that internally, the limited_deps flag is set in
the bitbake server and then never cleared. This causes the sysroots to
be setup incorrectly (as per the limited dependency case) and builds
break. There is also potential for corruption of recipecaches.
Add shutdown/cleanup code to ensure these effects don't 'stick'.
This bug is particularly nasty as you can destroy TMPDIR with large
sysroots in build work directories which are prone to break.
Also ensure mtime cache is cleared (to match buildTargets) and
that no lasting changes are made to siggen either which ensures:
BB_SERVER_TIMEOUT=100 oe-selftest -r devtool.DevtoolTests.test_devtool_upgrade_git devtool.DevtoolTests.test_devtool_virtual_kernel_modify
works.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When fetching source for the first time within scripts such as
OpenEmbedded's recipetool, we don't want to be showing warnings about
NPM_SHRINKWRAP or NPM_LOCKDOWN not being set since there's no way we
could have set them in advance. Previously we were using
ud.ignore_checksums to suppress these but since we are now using a more
standard task-based path to fetch the source, we need to disable these
through the metadata. Look for a "noverify" parameter set on the npm URL
and skip the checks if it is set to "1".
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you run some other operations that result in the cache being
populated, and then call buildFileInternal(), then you can end up in a
situation where the cache already contains information about the recipe.
For example in OE this can now happen when you use devtool upgrade.
Normally this doesn't cause any problems, unless you have a non-absolute
path in BBLAYERS - in buildFileInternal() we are calling matchfile() which
will convert the filename to absolute, but later when taskdata goes to find
the providers of the recipe it finds the non-absolute path, sets up the
task information using this and then the runqueue can't find any tasks
matching the absolute path. To fix this, back out the optimisation I did
earlier in bitbake rev ba53e067a2d448dd63b4ca252557ce98aa8e6321 to avoid
calling parseConfiguration() again, which is unfortunate but does result
in the cached information being that causes the problem being cleared
out.
This fixes "Task do_unpack does not exist for target ..." running
devtool upgrade within intel-iot-refkit.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you set tracking=True when creating the tinfoil object, that ensures
history is collected for the main datastore, but at the end of parsing
the configuration, history tracking gets turned off to save time with
the result that we don't collect history for any recipes we parse.
Enable tracking when we parse a recipe (and disable it afterwards if we
enabled it) in order to fix this.
This fixes functionality in OE's devtool that relies upon variable
history (such as devtool upgrade updating PV when it's set within a
recipe).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a task fails during build_targets(), we need to print out the log
lines as knotty does or the user will be missing information about the
failure.
(This should get some deeper refactoring, but now isn't the time for
that.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With config_only=False we launch the UI and it sets up a logger, whereas
when config_only=True we don't, with the result that with True we are
seeing log messages from both our logger and the one set up by the UI.
Suppress our loggers with config_only=True to avoid this.
Fixes [YOCTO #11275] (again).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the server exits quickly its PID may no longer exist. Handle
this gracefully.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
self has a quit function and a variable. Separate this to two different
things as the current setup is prone to breakage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Shutdown from SIGTERM currently has to wait for the handler thread to timeout.
Add a sentinel value which triggers it to loop and allows for a quick exit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If for example you run:
bitbake -r somefile.inc
rm somefile.inc
bitbake -e
bitbake will crash with an error about not being able to find somefile.inc. This
is because it tries to reparse the base config for the early getVariable requests
before it sees the updated missing -r option.
Send the updateConfig command earlier to avoid this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
cooker.reset() can be called before we've actually setup the datastore. Gracefully
handle this case instead of the current traceback+exit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When we reload the configuration, PRSERV_HOST can change. Therefore
restart the PR Server depending on the new configuration at reparse.
Note that the server has to be started from the right process, it
can't be in the UI which shuts down as that shutdown triggers a shutdown
of its children and the PR Server shuts down too. This is why we need
pre_serve() which ensures its executed in the right context.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows for cleaner code in cooker as any existing server is dealt
with before a new one is started.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the original prefile and postfile passed when starting bitbake
server are 'sticky'. With the new memory resident model this doesn't make
sense as the server the system is started with isn't special.
This patch changes the code so the prefile/postfile are used if specified
on the commandline and not used otherwise. This makes the behaviour much
more predictable and expected and as an added bonus simplifies the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current passing of "tracking" backwards and forwards, client to server
is ugly and complex and error prone. Instead, set this during showEnvironment
commands triggering a reset there if/as required.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Drop pointless unused function parameter.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The 'needconfig' flag was meant to be the default as most commands do need
it and the "False" cases were the exception. The code was written backwards
with a default False value. Invert this to match the intent, resulting in
the config being reparsed if metadata has changed.
Also ensure the second level configuration is parsed for the getLayerPriorities
command as otherwise it can return stale info.
With these changes:
BB_SERVER_TIMEOUT=100 oe-selftest -r bblayers.BitbakeLayers.test_bitbakelayers_add_remove
passes instead of fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than just waiting for 5s, try and get the lockfile. If we gain
the lock, we know we're ready to retry and can skip any remaining timeout.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its possible for a connection to connect to the server as its shutting down
but before its removed the socket file. This patch:
a) Removes the socket file earlier to avoid connections.
b) Handles EOFError in initial connections gracefully. These occur if the
socket is closed during the server shutdown.
c) Ensure duplicate events aren't shown on the console. This makes debugging
these issues very very confusing.
With these changes the backtrace that was concerning users is hidden and the
server works as expected with a reconnect when it catches it in a bad state.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
URLs do not have to have a path; currently our npm URLs don't, so
encodeurl() needs to handle if the path element isn't specified. This
fixes errors using OpenEmbedded's devtool add / recipetool create on an
npm URL after OE-Core revision ecca596b75cfda2f798a0bdde75f4f774e23a95b
that uses decodeurl() and encodeurl() to change URL parameter values.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The underlying model is already sorted for use in the other view,
add a sorting model for the 'Dependent Tasks' view.
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster needs to allow the custom fixture file to specific an
alternate Layer Index URL for users that host their own Layer
Index Server via the 'CUSTOM_LAYERINDEX_SERVER' in
'custom.xml'.
Toaster also needs to allow the ability to completely override
the default fixture files, otherwise that content can leak into
the custom environment (by default the custom fixture is an
overlay that cannot remove existing values from the default
fixture) via the 'CUSTOM_XML_ONLY' value in 'setting.xml'.
[YOCTO #11938]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster needs the ability to allow custom extensions to execute when
Toaster is started and stopped. Toaster will look for a custom
extension script in the fixtures directory and execute any applicable
hooks.
[YOCTO #11938]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Paul Eggleton
Scott Rifenbark
Richard Purdie
David Reyna
Ross Burton
Andre Rosa
Mark Hatle
Olaf Mandel
Awais Belal
Caio Marcelo de Oliveira Filho
Jussi Kukkonen