Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use FileBackedExperimentInfo class in LoadMD #327

Merged
merged 2 commits into from Mar 3, 2015

Conversation

martyngigg
Copy link
Member

Fixes trac issue #11111

Tester

The major work was completed in #307. This simply uses the new class in LoadMD when the FileBackend option is set to true. The best way to test is to see the time difference in loading a large MD file before and after the changes. A 24Gb test file can be found on the Vates file share at ISIS. Let me know if you want me to point you at it.

Only used in conjunction with the file backend option.
Refs #11111
@martyngigg martyngigg added this to the Release 3.4 milestone Mar 2, 2015
@martyngigg
Copy link
Member Author

Jenkins, retest this please.

@FedeMPouzols FedeMPouzols self-assigned this Mar 3, 2015
@FedeMPouzols
Copy link
Contributor

All looks good, including system tests.

I did a quick non-rigorous comparison using the large 22413_22932_SQW.nxs (~24 GB) file. I'd say that that's more than enough to verify that 'FileBackEnd' does what it is supposed to do:

  • With FileBackEnd enabled: 58.72 s to finish a lazy LoadMD. Goes down to 37.84 s once the file is buffered.
  • Without: 1579.46 s for a traditional, full load. I was going to stop it as It gets awfully slow when it runs out of RAM on a 16GB machine and starts using swap space.

FedeMPouzols added a commit that referenced this pull request Mar 3, 2015
…pt_info

Use FileBackedExperimentInfo class in LoadMD
@FedeMPouzols FedeMPouzols merged commit 55aead4 into master Mar 3, 2015
@FedeMPouzols FedeMPouzols deleted the 11111_loadmd_filebacked_expt_info branch March 3, 2015 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants