Skip to content

bucket_importer: rewrite #19

Open
wants to merge 1 commit into from

1 participant

@helllamer

Currently, bucket_importer has several really big issues:

  • original keys is lost, if they have $/ (for example, key <<"a/b/c/d">> will be imported as <<"d">>);
  • fat memory leak: tens of thousands strings of filenames = several GB, and lists:map stacktrace over such list = + 500 Mb of stacktraces
  • really useless return value.

This pull request has rewritten bucket_importer:

  • keys with $/ are imported ok
  • memory fingerprint is really minimal
  • returns ok
  • minimized garbage for gc
  • it works really faster, because gc now eats very small CPU% (no long_gc warnings even on default vm configuration).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.