[BUG] scripts/build-tools.sh race can corrupt development topologies #5067
Comments
|
@marc-hb can you add a cmake dependency between them or fix the script to work around this until a proper solution is found ? |
|
I can think of a number of "quick fixes" but I could first use some idea of what purpose development topologies serve. Maybe a quick fix is not to copy / "install" them one level up if no one cares about them. |
|
'Development' topologies are typically added for a specific purpose such as The intent is that they are developer-maintained, but not suitable for release or use by non-developers. there are no plans for any backwards compatibility with e.g. UCM files. As such these topologies do not belong in sof-bin. Whoever needs those topologies should use the git tree and know what they are doing. Does this help? |
Move generated *.conf and *.tplg v1 files down from:
build_tools/topology/topology1/*.{conf,tplg}
_to:
build_tools/topology/topology1/production/*.{conf,tplg}
... then copy/"install" the production/* subdirectory two levels up.
This fixes the race condition thesofproject#5067 that also copied a random number of
development/ and dsp_enhancement/ topologies, sometimes even truncating
these.
In other words, this commit REMOVES the following two copies:
build_tools/topology/development/ # randomly corrupted copy, removed
build_tools/topology/dsp_enhancement/ # randomly corrupted copy, removed
build_tools/topology/topology1/development/ # real build dir, unchanged
build_tools/topology/topology1/dsp_enhancement # real build dir, unchanged
Production topologies are copied only to help with the v1->v2
transition. That duplication makes the build directory confusing enough,
no need to extend that copy to development topologies. A single instance
of development topologies in the build directory is enough.
This removal may break some CI script(s): this is perfect because such
CI script(s) were copying randomly corrupted data.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
It does thank you! This took a while (always with CMake...) but I think I have a pretty good fix: #5071 |
Move generated *.conf and *.tplg v1 files down from:
build_tools/topology/topology1/*.{conf,tplg}
_to:
build_tools/topology/topology1/production/*.{conf,tplg}
... then copy/"install" the production/* subdirectory two levels up.
This fixes the race condition #5067 that also copied a random number of
development/ and dsp_enhancement/ topologies, sometimes even truncating
these.
In other words, this commit REMOVES the following two copies:
build_tools/topology/development/ # randomly corrupted copy, removed
build_tools/topology/dsp_enhancement/ # randomly corrupted copy, removed
build_tools/topology/topology1/development/ # real build dir, unchanged
build_tools/topology/topology1/dsp_enhancement # real build dir, unchanged
Production topologies are copied only to help with the v1->v2
transition. That duplication makes the build directory confusing enough,
no need to extend that copy to development topologies. A single instance
of development topologies in the build directory is enough.
This removal may break some CI script(s): this is perfect because such
CI script(s) were copying randomly corrupted data.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
|
@marc-hb can we close ? |
marc-hb
changed the title
SOF main branch, commit 7c6186a
Run these:
Save and compare the output
sof/tools/build_toolsdirectories each time and compare them. They don't have the same content. In one case some development topologies files are likely to be corrupted.I learned this the really hard way while testing a totally unrelated CMake fix.
Finer-grained reproduction:
Compare with:
Build systems are highly parallel, so a sequencing problem like this is a red flag: a race condition red flag.
The simple problem is
topologiesand (optional)dev_topologies1run concurrently. This would be great if the installer of the former didn't accidentally grab whatever the latter is still building. That makes the number of development topologies "installed" VARIABLE.Much WORSE and how I actually found this in the first place, it can happen that some topology files are TRUNCATED because they're installed before they're finished.
cc: @ranj063 from commit c0bee42 / PR #4363
The text was updated successfully, but these errors were encountered: