Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bisect_insert_strategy doesn't accumulate results #39

Closed
jameshiebert opened this issue Aug 24, 2018 · 2 comments
Closed

bisect_insert_strategy doesn't accumulate results #39

jameshiebert opened this issue Aug 24, 2018 · 2 comments
Assignees
Labels

Comments

@jameshiebert
Copy link
Contributor

Just did a run of the moti_insert and found an interesting result:

{"asctime": "2018-08-24 14:59:20,814", "levelname": "INFO", "name": "crmprtd.insert", "message": "Using Chunk + Bisection Strategy"}
{"asctime": "2018-08-24 14:59:21,111", "levelname": "INFO", "name": "crmprtd.insert", "message": "Successfully inserted observations", "num_obs": 512}
{"asctime": "2018-08-24 14:59:21,187", "levelname": "INFO", "name": "crmprtd.insert", "message": "Successfully inserted observations", "num_obs": 128}
{"asctime": "2018-08-24 14:59:21,209", "levelname": "INFO", "name": "crmprtd.insert", "message": "Successfully inserted observations", "num_obs": 32}
{"asctime": "2018-08-24 14:59:21,224", "levelname": "INFO", "name": "crmprtd.insert", "message": "Successfully inserted observations", "num_obs": 16}
{"asctime": "2018-08-24 14:59:21,230", "levelname": "INFO", "name": "crmprtd.insert", "message": "Successfully inserted observations", "num_obs": 2}
{"asctime": "2018-08-24 14:59:21,234", "levelname": "INFO", "name": "crmprtd.insert", "message": "Data insertion complete"}
{"asctime": "2018-08-24 14:59:21,235", "levelname": "INFO", "name": "crmprtd", "message": "Data insertion results", "results": {"insertions_per_sec": 4.76, "skips": 0, "successes": 2, "failures": 0}}

As you can see, the end "result" is 2 successful insertions, even though the logs show that it's successfully inserting larger chunks before that. If you look at the code:

        with Timer() as tmr:
            for chunk in chunks(observations):
                dbm = bisect_insert_strategy(sesh, chunk)

You have forgotten to accumulate the results from each chunk insert, throwing away the results from before. This should be a quick fix:

dbm += bisect_insert_strategy(sesh, chunk)

As long as dbm is initialized prior to the beginning of the loop.

@jameshiebert
Copy link
Contributor Author

You probably know this already, but the issue is only in the dev branch :)

@nikola-rados
Copy link
Contributor

During my testing today I discovered the same issue and fixed it in a commit. Have not pushed the change yet though until I finish testing all the networks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants