Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a cached halotools catalogs + HOD cleanup #441

Merged
merged 42 commits into from Nov 18, 2017
Merged

add a cached halotools catalogs + HOD cleanup #441

merged 42 commits into from Nov 18, 2017

Conversation

nickhand
Copy link
Member

@nickhand nickhand commented Nov 8, 2017

A few additions:

  • HalotoolsCachedCatalog can download + load the built-in halo sims available through Halotools. They seem to be a good size for tutorials (~100 MB) and are useful for comparing 2-pt correlation function results to halotools results
  • Support for astropy Table in ArrayCatalog
  • re-organize HODCatalog a bit so it subclasses HalotoolsMockCatalog, with can populate a mock in parallel given a Halotools catalog + model. This adds support for generic Halotools models.
  • catch and re-raise exceptions when populating mocks with only the root rank. This should prevent the code from hanging and Halotools is pretty good about having useful exception messages.

@nickhand
Copy link
Member Author

@rainwoodman I am ready to merge this now, if you are.


# missing required column
del cat.halo_table['halo_id']
# no galaxies populated into halos
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logMmin is very large. I'll add a note

size = hod.csize

# repopulate (with same seed --> same catalog)
hod.repopulate(seed=42)
hod = halos.repopulate(seed=42)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you like hod.repopulate() better than halos.repopulate()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that's the case then the first ValueError can be eliminated -- without doing populate first there is no hod object for you to repopulate.

The implication is that the compiled model shall be associated with hod, not halos.

assert hod.csize == size

# repopulate with random seed --> different catalog
hod.repopulate()
hod = halos.repopulate()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably give an explicit seed ? It is unlikely but what if random number generator just gave the same number as you used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am actually testing that the seed is set by root and then propagated when it is None − I'll update the assertion

@MPITest([1])
def test_download_file(comm):

# download Gadget1P snapshots directory
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: TPM, not Gadget1P

@rainwoodman
Copy link
Member

I noted a few minor glitches in the comments.

hod.repopulate() and halo.populate() can the assertion no repopluate before populate. It encourages moving the compiled model to hod -- but I suspect this works against the data model of halotools. We can still mark halo._repopulate() private and only expose it via hod.repopulate(), then comment about the reason for using a delegation .

@nickhand
Copy link
Member Author

Okay, I agree with your repopulate() comment. Now, halos.populate returns a PopulatedHaloCatalog (a light subclass of ArrayCatalog) which has the repopulate() method (which does the repopulation in place).

I'll merge this if you are ready.

@rainwoodman
Copy link
Member

rainwoodman commented Nov 18, 2017 via email

@nickhand nickhand merged commit 779724c into master Nov 18, 2017
@nickhand nickhand deleted the halotools branch November 18, 2017 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants