Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

basic sge implementation #6

Closed
wants to merge 3 commits into from
Closed

basic sge implementation #6

wants to merge 3 commits into from

Conversation

jhamman
Copy link
Member

@jhamman jhamman commented Mar 2, 2018

closes #3

I don't think this is ready to merge. Turns out the SGE cluster I used to have access to is no more. Perhaps we should get a basic CI implementation setup before pushing more content into the repo?

self.n = 0
self._adaptive = None
self._submitcmd = 'qsub'
self._cancelcmd = 'qdel'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should maybe be class variables?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, agree for submit and cancel commands.

project = project or os.environ.get('SGE_ACCOUNT')
if not project:
raise ValueError("Must specify a project like `project='UCLB1234' "
"or set SGE_ACCOUNT environment variable")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on #7 ?


def test_basic(loop):
with SGECluster(walltime='00:02:00', threads_per_worker=2, memory='7GB',
interface='ib0', loop=loop) as cluster:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interface keyword option here will likely fail on a CI system.

@jakirkham
Copy link
Member

Perhaps we should get a basic CI implementation setup before pushing more content into the repo?

So Travis CI is enabled, but does not have a configuration file yet. Would encourage just stealing stuff from dask-drmaa to get this working. There's a few files that you would want to grab outside of .travis.yml. They would be the docker-compose.yml, Dockerfiles, various assorted shell scripts, and some text files use for configuration.

@jhamman
Copy link
Member Author

jhamman commented Mar 27, 2018

@lesteve - afaik, you're the only current dask-jobqueue user/developer with access to an sge cluster. You may be interested in this initial implementation of the SGECluster. It will require some reworking following #10 but may provide a starting point.

@lesteve
Copy link
Member

lesteve commented Mar 28, 2018

Thanks I'll try to take a look.

@lesteve lesteve mentioned this pull request Apr 4, 2018
@jhamman jhamman closed this in #27 Apr 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Grid Engine
5 participants