summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorRichard Purdie <richard.purdie@linuxfoundation.org>2021-05-15 10:25:50 +0100
committerMartin Jansa <Martin.Jansa@gmail.com>2021-05-16 10:26:13 +0200
commit89afe39b9c01d9de5dc9ded9822da2dc187a4c89 (patch)
treeb56fa0b70e0f0c69161bf5c4061c5635826ba2ad
parent3b558a49facc4a5e19c544c2cf93a638c1e9c197 (diff)
downloadopenembedded-core-contrib-jansa/hardknott.tar.gz
openembedded-core-contrib-jansa/hardknott.tar.bz2
openembedded-core-contrib-jansa/hardknott.zip
sstate: Handle manifest 'corruption' issuejansa/hardknott
Under certain build patterns, warnings about missing manifests can appear. These are real issues where the manifest was removed and shouldn't have been. Martin Jansa was able to find a reproducer of: MACHINE=qemux86 bitbake zlib-native echo 'PR = "r1"' >> meta/recipes-core/zlib/zlib_1.2.11.bb MACHINE=qemux86-64 bitbake zlib-native MACHINE=qemux86 bitbake zlib-native The code maintains a per machine list of stamps but a per PACAGE_ARCH list of stamp/manifest/workdir mappings. The latter is only appended to for speed with the assumption that once stamps are gone, the code wouldn't trigger. The code only ever appends to the mapping list (for speed/efficency under lock) meaning that multiple entries can result where the stamp/workdir differs due to version changes but the manifest remains the same. By switching MACHINE part way through the build, the older stamp is referenced and the manifest is incorrectly removed as it matches an now obsolete entry in the mapping file. There are two possible fixes, one is to rewrite the mapping file every time which means adding regexs, iterating and generally complicating that code. The second option is to only use the last mapping entry in the file for a given manifest and ignore any earlier ones. This patch implments the latter. Also drop the stale entries if we are rewriting it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-rw-r--r--meta/classes/sstate.bbclass16
1 files changed, 15 insertions, 1 deletions
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 1ccbe9ac7c..e8a43515f2 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -319,6 +319,8 @@ def sstate_install(ss, d):
if os.path.exists(i):
with open(i, "r") as f:
manifests = f.readlines()
+ # We append new entries, we don't remove older entries which may have the same
+ # manifest name but different versions from stamp/workdir. See below.
if filedata not in manifests:
with open(i, "a+") as f:
f.write(filedata)
@@ -1184,11 +1186,21 @@ python sstate_eventhandler_reachablestamps() {
i = d.expand("${SSTATE_MANIFESTS}/index-" + a)
if not os.path.exists(i):
continue
+ manseen = set()
+ ignore = []
with open(i, "r") as f:
lines = f.readlines()
- for l in lines:
+ for l in reversed(lines):
try:
(stamp, manifest, workdir) = l.split()
+ # The index may have multiple entries for the same manifest as the code above only appends
+ # new entries and there may be an entry with matching manifest but differing version in stamp/workdir.
+ # The last entry in the list is the valid one, any earlier entries with matching manifests
+ # should be ignored.
+ if manifest in manseen:
+ ignore.append(l)
+ continue
+ manseen.add(manifest)
if stamp not in stamps and stamp not in preservestamps and stamp in machineindex:
toremove.append(l)
bb.note("Stamp %s is not reachable, removing related manifests" % stamp)
@@ -1227,6 +1239,8 @@ python sstate_eventhandler_reachablestamps() {
with open(i, "w") as f:
for l in lines:
+ if l in ignore:
+ continue
f.write(l)
machineindex |= set(stamps)
with open(mi, "w") as f: