Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial run with --incremental does weird things #379

rbuels opened this issue Nov 18, 2013 · 0 comments


None yet
1 participant
Copy link

commented Nov 18, 2013

On 11/15/2013 08:18 PM, Richard Hayes wrote:

  1. If I run with the --incremental option but don't have an existing
    names dir to add on to,
    builds the index to a certain point but fails to complete, and the only
    file added to names/ is the meta.json file. Verbose output ends at the
    "Gathering locations" stage like:

[Mon Nov 11 12:42:34 2013] Gathering locations, generating completions:
Error in tempfile() using /names/names-hash-tmp-XXXXXXXX:
Parent directory (/names/) does not exist at
line 167.
Sampled input stats:
avg record text bytes 81.93778
name input records 50000
operation stream estimated count 587283
tracksWithNames ARRAY(0x1297b90)
record stream estimated count 587283
total namerec bytes 4096889
total input bytes 48120711
operations made 50000
namerecs buffered 50000
namerecs converted to operations 50000
Using 12-bit hashing
Hash store cache size: 894 buckets
Temporary key-value DBM file: /names-build-tmp-LpF7sRBC
Estimating 587283 index operations on 587283 completion records

Subsequent incremental runs complete with the expected additional
"Rehashing to final buckets" and "Writing/updating index bucket files"
steps, ending without the error message, like:

[Mon Nov 11 16:06:01 2013] Writing/updating index bucket files: 100%
Sampled input stats:
avg record text bytes 27.0834138486312
name input records 1822
operation stream estimated count 4002608
tracksWithNames ARRAY(0x19b5f10)
record stream estimated count 250163
total namerec bytes 84094
total input bytes 6775288
operations made 50015
namerecs buffered 3105
namerecs converted to operations 3103
Using 12-bit hashing
Hash store cache size: 894 buckets
Temporary key-value DBM file: /names-build-tmp-Mwc47IOI
Estimating 4002608 index operations on 250163 completion records
Temporary rehashing DBM file: /names/names-hash-tmp-hR2psM1K
Hash store bulk load finished.

@rbuels rbuels closed this in 6cc48d8 Nov 18, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.