Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
multiresolver: crash when used with cache #2694
src/plugins/resolver/resolver.c:1175 seems to cause a crash in some situations.
Tried to reproduce. Does not work:
# create a new folder to not mess up with existing data mkdir x cd x # create two mountpoints kdb mount `pwd`/csv system/tests/csv/lists/cur csvstorage header=colname,columns/index=student/id kdb mount -R multifile -c storage="ini",pattern="*/*",resolver="resolver" `pwd`/multi system/tests/multi # create a csv file echo "student/id,ue/5/kreuzerl" >> csv echo "01234567,X" >> csv # create a multiresolver directory mkdir -p multi/pool cd multi/pool echo "" >> 01234567 >> 01234568 >> 01234569 echo "[student]" >> 01234567 >> 01234568 >> 01234569 echo "id = 01234567" >> 01234567 echo "[ue/5]" >> 01234567 # create caches kdb ls system/tests > /dev/null kdb ls system/tests/multi/pool > /dev/null kdb ls system/tests/csv/lists/cur > /dev/null # now do something directly on the files rm 01234569 touch 01234566 echo "kreuzerl = O" >> 01234567 echo "[something]" >> 01234567 >> 01234568 echo "" >> 01234567 >> 01234568 echo "" >> 01234568 # trigger kdb cp -rf system/tests/csv/lists/cur system/tests/multi/pool # debug kdb export system/tests mini tail * kdb umount system/tests/csv/lists/cur kdb umount system/tests/multi cd ../../.. rm -r x
Did not work:
# create a new folder to not mess up with existing data mkdir x cd x # create two mountpoints kdb mount `pwd`/csv system/tests/csv csvstorage header=colname,columns/index=sec/somekey kdb mount -R multifile -c storage="ini",pattern="*",resolver="resolver" `pwd`/multi system/tests/multi # create a csv file echo "sec/somekey,othersec/deep/otherkey" >> csv echo "a,data2a" >> csv # echo "b,data2b" >> csv # but do not write in the other # create a multiresolver directory mkdir multi cd multi echo "[sec]" >> a >> b echo "somekey = a" >> a echo "somekey = b" >> b echo "" >> a >> b echo "" >> a >> b kdb cp -rf system/tests/csv system/tests/multi kdb umount system/tests/csv kdb umount system/tests/multi
The problem seems to be
The problem might be, that the multiresolver has some internal state (filename) which does not get recovered when it is not called in kdbGet because of the cache hit.
I added steps to reproduce above.
The new backtrace from these steps (Hopfully the same. I simplified some steps from the script that originally caused the problem):
Yes, multiresolver is very buggy and it does not support creating new files. Nevertheless, there was a case where it crashed only with the cache and worked after disabling cache.
I now investigated this problem in detail and the problem is very tricky. I could not reproduce it without local cache files. With my local cache files, it is easy to trigger and I found a quite minimal set of files to trigger it. The CSV only contains:
and I needed only two INI files:
Then the cp crashed only if cache is enabled and a
I updated the top-post but as said, without the problematic cache file it does not work. Unfortunately, the cache file is big (2.2MB) and contains private data.
Do you have some idea what it could be?
Unfortunately, I have no idea what it could be. The example you gave is quite elaborate but it works fine in our debian stretch docker image.
We already have a few separate issues here:
I can only suggest that we start with the first two which are easy to reproduce, make regression tests, fix and then move on to the third problem.