Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restart Pecan on Emu with Geoduck files #526

Closed
laurahspencer opened this issue Mar 10, 2017 · 5 comments
Closed

Restart Pecan on Emu with Geoduck files #526

laurahspencer opened this issue Mar 10, 2017 · 5 comments
Assignees

Comments

@laurahspencer
Copy link
Contributor

@laurahspencer laurahspencer commented Mar 10, 2017

@seanb80 please run the following 4 sample files in the following order:

Geoduck1 (corresponds to Geoduck 48 @ Site Fidalgo Bay = Eelgrass)
Geoduck6 (corresponds to Geoduck 58 @ Site Fidalgo Bay - Bare)
Geoduck3 (corresponds to Geoduck 10 @ Site Case Inlet - Eelgrass)
Geoduck9 (corresponds to Geoduck 18 @ Site Case Inlet - Bare)

When these are done I'll have another set to run.

@laurahspencer
Copy link
Contributor Author

@laurahspencer laurahspencer commented Mar 10, 2017

Chatted with Sean to sketch out this next Pecan run timeline.

If we use 4 of the 16 logical cores on Emu (we used 3 in the recent test run, and tried 14 in the big run that didn't work / was cancelled), using the 80 isolation windows: it should take ~13.5 days to complete 2 data files (aka 2 samples).

This is assuming that only using 3 isolation windows and 3 logical cores = 10 hours per sample file, and that increasing the logical cores from 3 to 4 results in a linear decrease in time/isolation window.

Bottom line, with the given settings I should have data from 1 sample file done in a week, which I'll plan on using for the poster.

@seanb80
Copy link
Contributor

@seanb80 seanb80 commented Mar 11, 2017

It looks like Pecan/Grid Engine is still using much more memory than I gave it permission to, but hopefully with fewer processes running this won't end poorly. Will monitor it over the weekend to make sure it's behaving properly.

The Res is the closest approximation to memory used by a process, and it looks like our four python instances are using 0.016 terabytes (16gb), 0.015 terabytes (15gb), 7.6gb and 6.7gb respectively.

screen shot 2017-03-10 at 5 48 29 pm

@laurahspencer
Copy link
Contributor Author

@laurahspencer laurahspencer commented Mar 13, 2017

FYI Sean walked me through killing/restarting Pecan since the memory issue persisted over the weekend and resulted in partially-completed runs.

See my latest lab notebook entry

@laurahspencer
Copy link
Contributor Author

@laurahspencer laurahspencer commented Mar 13, 2017

Update: Steven prepared a shortened background proteome with only stress-response related proteins (see issue #528) - selected 600 proteins from 30k. I restarted Pecan - check out my latest blog post for details.

@sr320 sr320 closed this Mar 13, 2017
@laurahspencer
Copy link
Contributor Author

@laurahspencer laurahspencer commented Mar 13, 2017

FYI the input files for my latest Pecan run are all here: http://owl.fish.washington.edu/generosa/Generosa_DNR/Pecan/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.