[bug found] res.rhash shrinked during 4s-query and 4s-backend segfault #131
Comments
|
Is this with the version from Git HEAD - or a release? Were you importing with 4s-import, and querying with 4s-query, or doing both with 4s-httpd? |
|
the version is 6e2b4a7, 2 commits before HEAD import is done with 4s-import and query is done with 4s-query |
|
Odd - it seems like that's quite a well-used path in the code. Could you try backing off to an older version (maybe 12 months ago) and trying again. There is a reason why some files are opened RDWR when querying, but I really can't remember why it is. |
|
I tried multiple older versions from 2013 to 2014, and got the same problem for all of them. If you don't mind may be I can sent you my data (98M after zipped on google drive). |
|
If you could share the data that would be fantastic. It's really odd - I'm confident a lot of people have done exactly what you have... |
|
OK, well the good news is I can reproduce - Ubuntu 14.04.1 LTS |
|
So, it doesn't happen with the first 500k triples from that file, but it does happen with the file but with just the first two deleted. I guess there's something in the data that's causing the problem. I can't spend anymore time on it now, but I'll setup a script to try importing it in 500k chunks and see if I can narrow it down. |
|
@msalvadores - have you seen anything like this? |
|
glad you can reproduce the bug |
|
hi, @swh, sorry to borther you, I have a project depending on 4store, so I'll be much appreciated if you could help me fix the bug or provide me some hint on how to fix it. I personally have a simple solution, that is to check the size of rhash before ftruncate() is called, but I'm not sure if this can totally fix the bug, or may cause some other problems. Thanks in advance rushan chen |
|
No problem. Sorry, didn't get time to look at it again last week. I'd suggest putting in your hack and I'll try and find the cause - I suspect it must be something unusual in that file. Cheers, Sent on the move.
|
|
This might be related to #135 (comment) - can you try reverting the same patch? |
|
Sorry, I'm busy working on a project, I'll try it when I have time. Currently, my hack seems to work well. rushan chen |
|
Is there a patch for this issue? @juscodit Could you write some lines about how your fix looks like? I've got the same error here on Fedora21. The error is printed for each line (SPO) of the result set. |
diff --git a/src/backend/rhash.c b/src/backend/rhash.c
index 2ccf3d1..8906ff7 100644
--- a/src/backend/rhash.c
+++ b/src/backend/rhash.c
@@ -25,6 +25,7 @@
#include <glib.h>
#include <string.h>
#include <zlib.h>
+#include <sys/stat.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/file.h>
@@ -141,6 +142,14 @@ void fs_rhash_ensure_size(fs_rhash *rh)
if (!(rh->flags & (O_WRONLY | O_RDWR))) return;
const off_t len = sizeof(struct rhash_header) + ((off_t) rh->size) * ((off_t) rh->bucket_size) * sizeof(fs_rhash_entry);
+ /* crs: check file size before truncate */
+ struct stat statbuf;
+ if (fstat(rh->fd, &statbuf) < 0) {
+ return;
+ }
+ if (len <= statbuf.st_size) {
+ return;
+ }
if (ftruncate(rh->fd, len)) {
fs_error(LOG_ERR, "ftruncate failed for '%s': %s", rh->filename, strerror(errno));
@@ -206,7 +215,13 @@ fs_rhash *fs_rhash_open_filename(const char *filename, int flags)
#endif
fs_rhash_ensure_size(rh);
- const size_t len = sizeof(header) + ((size_t) rh->size) * ((size_t) rh->bucket_size) * sizeof(fs_rhash_entry);
+ /* crs: check file size before truncate */
+ struct stat statbuf;
+ if (fstat(rh->fd, &statbuf) < 0) {
+ return NULL;
+ }
+ // const size_t len = sizeof(header) + ((size_t) rh->size) * ((size_t) rh->bucket_size) * sizeof(fs_rhash_entry);
+ const size_t len = statbuf.st_size;
const int mmapflags = PROT_READ | (flags & (O_WRONLY | O_RDWR) ? PROT_WRITE : 0);
header = mmap(NULL, len, mmapflags, MAP_SHARED, rh->fd, 0);
if (header == MAP_FAILED) { |
|
lovely, thx |
Here's the story
First I imported a file containing 10000000 triples, then I issued a simple query, and I got the following error
Initially, it seems like a network error. But after some debugging, I found that backend segfault. Then I keep debugging to find why backend segfault, and I found that the size of
res.rhashshinked to half, and an access to a hash entry which should have existed now became invalid and caused segfault.I further found the reason of file shrink. When 4s-backend dealing with the
FS_CHOOSE_SEGMENTrequest, thehandle_choose_segmentfunction callsfs_backend_open_files_intlwithflagsset toO_RDWR | O_CREATwhich furtuer callsfs_rhash_open_filename, and according to the logic implemented infs_rhash_open_filename, it will first setrh->sizeto 65536, and then callfs_rhash_ensure_size, since theO_RDWRflag is set,res.rhashwhich contained more than 65536 entries is now shrink to 65536 entriesHere's the call stack:
I don't know why a query request would have
O_RDWRset when open files ?rushan chen
The text was updated successfully, but these errors were encountered: