New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a cached halotools catalogs + HOD cleanup #441

Merged
merged 42 commits into from Nov 18, 2017

Conversation

Projects
None yet
2 participants
@nickhand
Member

nickhand commented Nov 8, 2017

A few additions:

  • HalotoolsCachedCatalog can download + load the built-in halo sims available through Halotools. They seem to be a good size for tutorials (~100 MB) and are useful for comparing 2-pt correlation function results to halotools results
  • Support for astropy Table in ArrayCatalog
  • re-organize HODCatalog a bit so it subclasses HalotoolsMockCatalog, with can populate a mock in parallel given a Halotools catalog + model. This adds support for generic Halotools models.
  • catch and re-raise exceptions when populating mocks with only the root rank. This should prevent the code from hanging and Halotools is pretty good about having useful exception messages.

@nickhand nickhand requested a review from rainwoodman Nov 8, 2017

@nickhand

This comment has been minimized.

Member

nickhand commented Nov 17, 2017

@rainwoodman I am ready to merge this now, if you are.

# missing required column
del cat.halo_table['halo_id']
# no galaxies populated into halos

This comment has been minimized.

@rainwoodman

rainwoodman Nov 17, 2017

Member

Because?

This comment has been minimized.

@nickhand

nickhand Nov 18, 2017

Member

logMmin is very large. I'll add a note

size = hod.csize
# repopulate (with same seed --> same catalog)
hod.repopulate(seed=42)
hod = halos.repopulate(seed=42)

This comment has been minimized.

@rainwoodman

rainwoodman Nov 17, 2017

Member

do you like hod.repopulate() better than halos.repopulate()?

This comment has been minimized.

@rainwoodman

rainwoodman Nov 17, 2017

Member

If that's the case then the first ValueError can be eliminated -- without doing populate first there is no hod object for you to repopulate.

The implication is that the compiled model shall be associated with hod, not halos.

assert hod.csize == size
# repopulate with random seed --> different catalog
hod.repopulate()
hod = halos.repopulate()

This comment has been minimized.

@rainwoodman

rainwoodman Nov 17, 2017

Member

Probably give an explicit seed ? It is unlikely but what if random number generator just gave the same number as you used?

This comment has been minimized.

@nickhand

nickhand Nov 18, 2017

Member

I am actually testing that the seed is set by root and then propagated when it is None − I'll update the assertion

@MPITest([1])
def test_download_file(comm):
# download Gadget1P snapshots directory

This comment has been minimized.

@rainwoodman

rainwoodman Nov 17, 2017

Member

typo: TPM, not Gadget1P

@rainwoodman

This comment has been minimized.

Member

rainwoodman commented Nov 17, 2017

I noted a few minor glitches in the comments.

hod.repopulate() and halo.populate() can the assertion no repopluate before populate. It encourages moving the compiled model to hod -- but I suspect this works against the data model of halotools. We can still mark halo._repopulate() private and only expose it via hod.repopulate(), then comment about the reason for using a delegation .

@nickhand

This comment has been minimized.

Member

nickhand commented Nov 18, 2017

Okay, I agree with your repopulate() comment. Now, halos.populate returns a PopulatedHaloCatalog (a light subclass of ArrayCatalog) which has the repopulate() method (which does the repopulation in place).

I'll merge this if you are ready.

@rainwoodman

This comment has been minimized.

Member

rainwoodman commented Nov 18, 2017

@nickhand nickhand merged commit 779724c into master Nov 18, 2017

3 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-0.09%) to 95.198%
Details

@nickhand nickhand deleted the halotools branch Nov 18, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment