Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PTSCOTCH: Large refactor #16

Closed
wants to merge 3 commits into from

Conversation

dalcinl
Copy link
Contributor

@dalcinl dalcinl commented Oct 8, 2017

No description provided.

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@dalcinl
Copy link
Contributor Author

dalcinl commented Oct 8, 2017

@basnijholt @minrk Folks, this is my first attempt to improve the recipe. Some things worth to note:

  • Look at the commit message about the rationale of NOT defining SCOTCH_PTHREAD. To further understand the implications and issues, you should read the INSTALL.txt files in the tarball, and maybe even look at the source code.
  • At this point, I think we should truly split the packages in two. Right now, scotch (sequential) contains a subset of the libraries/headers of ptscotch (sequential+parallel). We could make ptscotch contain just the parallel libraries/headers, and make it depend on scotch.
  • Eventually, despite the fact that we have two separate branches, I would like to have the same Makefile.inc and build.sh in both branches, using make/bash conditionals as appropriate, and trying hard to keep them in sync. This should lower the chances of making inconsistent builds.

@minrk Maybe we should decline #15 and rework the master branch, backporting the changes from this PR?

@dalcinl dalcinl force-pushed the ptscotch-update branch 2 times, most recently from 8e36be1 to 4fb4d67 Compare October 8, 2017 18:48
- Single Makefile.inc (requires GNU Make).
- Do not define `SCOTCH_PTHREAD`. Number of threads is a compile-time
  setting and defaults to one. Client code is forced to call
  `MPI_Init_thread()` requesting `MPI_THREAD_MULTIPLE` support.
- Build/check/install/test scotch, then ptscotch. This should simplify
  a future split in two, with ptscotch depending on scotch and providing
  only the parallel headers and libraries.
- Add trivial patch for a missing function prototype.
- Add pthread patch from https://bitbucket.org/petsc/pkg-scotch.
- Bump build number.
Copy link
Member

@minrk minrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use the exact same Makefile and build.sh in both. I don't think that's hard. I don't really understand making ptscotch a separate package that depends on scotch when that's not the case - they are independent builds of the same package, aren't they? They certainly don't depend on each other.

Look at the commit message about the rationale of NOT defining SCOTCH_PTHREAD

Indeed, and that's why only scotch and not ptscotch defined SCOTCH_PTHREAD in this repo already.

CFLAGS += -DCOMMON_PTHREAD
#CFLAGS += -DSCOTCH_PTHREAD
ifeq ($(UNAME_S),Darwin)
CFLAGS += -DCOMMON_PTHREAD_BARRIER
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it would be easier to set CFLAGS, etc. in build.sh instead of Makefile, and just have CFLAGS = ${CFLAGS} here. Makefiles aren't very good at logic.

@minrk
Copy link
Member

minrk commented Oct 9, 2017

#15 is updated to have a single build.sh and Makefile that works for both scotch and ptscotch. By putting the flag logic entirely in build.sh it's much simpler than a complicated Makefile.

@dalcinl
Copy link
Contributor Author

dalcinl commented Oct 9, 2017

@minrk Look at the file INSTALL.txt in the tarball, particularly section 2.5) Multi-threading and 5) Use. From these notes,

  • We can (and should) make the ptscotch package providing only the parallel bits, and make it depend on the scotch package (sec. 5) Use). Note the comments mentioning that ptscotch.h includes scotch.h, and that users of the parallel library should link to both libptscotch.a and libscotch.a (in that order if relevant).
  • We can build scotch with -DSCOTCH_PTHREAD and ptscotch without (sec. 2.5) Multi-threading).

I'll update this PR following these facts.

* `meta.yaml` can build scotch or ptscotch by just setting `name`.
* `build.sh` installs either scotch or ptscotch headers and libraries.
* ptscotch now (runtime) depends on scotch.
@dalcinl
Copy link
Contributor Author

dalcinl commented Oct 10, 2017

@minrk I'm done with this PR, however I'll not merge it yet. May I suggest that you decline your previous PRs and submit a new one to start fresh? I'll eventually do the same with this one.

Now you can grab the recipe directory and copy it verbatim to the master branch, next change the name to "scotch" in meta.yaml and next add your changes moving configure logics out of Makefile.inc.
Once we have the new scotch build, I can backport everything from the master branch to the ptscotch branch.

Do you know any git trick we can use (maybe subtree merges) so that we can keep the recipe alone in a separate branch without all the other files, and easily merge changes to the master and ptscotch branches?

PS: If you are short of time, give me access to your feedstock clone, I can start updating the master branch, or even add there your previous changes.

@minrk
Copy link
Member

minrk commented Oct 10, 2017

Do you know any git trick we can use (maybe subtree merges) so that we can keep the recipe alone in a separate branch without all the other files, and easily merge changes to the master and ptscotch branches?

If ptscotch didn't depend on scotch, we could use the conda-forge build matrix to build both packages. Unfortunately, if we want ptscotch to build against scotch changed in the same PR, then that's not going to work due to parallel builds.

Other than that, I don't think anything is going to be simpler than keeping files in sync. This isn't a particularly active repo, so issuing PRs against two branches shouldn't be a huge burden. As long as we keep them in sync, applying patches twice isn't so different from having to merge one PR and then open another changing the submodule hash in the other branch.

@dalcinl dalcinl closed this Oct 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants