Added Papio Anubis genome, recombination map, and demographic model #1216

saurabhbelsare · 2022-04-13T20:48:27Z

This fork adds the species Papio Anubis (olive baboon). The sources for the data are:

Assembly: https://doi.org/10.1093/gigascience/giaa134
Mutation rate and generation time: https://doi.org/10.1371/journal.pbio.3000838
Recombination rate/map and demographic model: https://doi.org/10.1093/gbe/evac040

Caveat1: The recombination rate/map was only estimated for the autosomes. Hence I've added the recombination rate for the X chromosome as the mean rate from the autosomes.
Caveat2: I have named the demographic model as PAP_1A22 because I'm not sure what the correct naming convention is.
Caveat3: I have listed a placeholder url following the amazonaws.com format for the genetic map. I have all the map files required for this locally, but am not sure how to upload them to that url.

grahamgower

Hi @saurabhbelsare, this looks really good, thanks! I think the pre-commit/black stuff should be done as a separate pull request, but the rest looks good.

I don't know what's going on with the failing tests though. Do the tests fail when you run them on your computer?

grahamgower · 2022-04-21T18:32:54Z

.pre-commit-config.yaml

-    rev: 20.8b1
+    rev: 22.3.0


Is there are reason to change this in this pull request? I recall the black project did make a change to their output recently, so perhaps you could provide a separate pull request for this instead?

grahamgower · 2022-04-21T18:34:49Z

requirements/development.txt

-black
+git+https://github.com/psf/black


I don't think this is right, as a specific version would be preferred. In any event, we can discuss this in another pull request dealing only with updating black/pre-commit.

Hi Graham, I didn't intend to push any of these changes for black and the version. I only made these changes locally for myself because I ran into this error while trying to commit and used the solution they suggested in that thread. I didn't add the precommit yaml file or development.txt to my commit/pull request. Not sure how they got pushed.

I fixed all the issues with tests on my computer before I submitted the pull request, and didn't have any tests failing at that point. How should I fix these issues? Thanks!

I'm off to bed now, but I'll take a closer look tomorrow and see if I can figure out the failing tests.

stdpopsim/catalog/PapAnu/species.py

grahamgower · 2022-04-22T06:11:09Z

Hi @saurabhbelsare,
There is a test which tries to do a minimal simulation for each species, which is using all the memory when simulating PonAnu (and then the test is killed when memory runs out). It looks like the problem is a typo in the mutation rate, which makes it many orders of magnitude too big!

grahamgower · 2022-04-22T07:17:59Z

Hi again @saurabhbelsare. I've put the update to black in a separate pull request (#1224). After that's merged, you should be able to remove those changes from this pull request and rebase your PapAnu bits.

There is a failing test regarding the ensembl release number. I'm not sure what the best thing to do here is... do you need that specific version? @jeromekelleher @andrewkern do you recall what was the process here for developers? Should all the species be updated to the given ensembl release in the event that a newly added species needs a newer ensembl release? Or can we handle mismatched versions between species gracefully (ie., just fix the test here and move on, as someone will do the update of other species later)?

And, @saurabhbelsare, to answer your initial questions (sorry, I missed these somehow)...

Caveat1: The recombination rate/map was only estimated for the autosomes. Hence I've added the recombination rate for the X chromosome as the mean rate from the autosomes.

I think this is fine.

Caveat2: I have named the demographic model as PAP_1A22 because I'm not sure what the correct naming convention is.

The stdpopsim developer docs have this to say on the matter:

Demographic models are named using a combination of a descriptive name, information about the simulation, and information about the publication it was presented in. Specifically we use ${SomethingDescriptive}_${number_of_populations}${first_author_initial}${two_digit_date} where the descriptive text is meant to capture something about the model (i.e. an admixture model, a population crash, etc.) and the number of populations is the number of populations implemented in the model (not necessarily the number from which samples are drawn). For author initial we will use a single letter, the 1st, until an ID collision, in which case we will include the 2nd letter, and so forth.

So it would be nice to change the "PAP" in the model id to something more informative. Maybe a descriptor of the population that was used? Or the method? Or something descriptive from the publication?

Caveat3: I have listed a placeholder url following the amazonaws.com format for the genetic map. I have all the map files required for this locally, but am not sure how to upload them to that url.

This will need to be uploaded by the AWS keymaster, @andrewkern.

saurabhbelsare · 2022-04-27T00:47:32Z

I have updated the name of the demographic model and shared the recombination map with Andy. l'll update the url as soon as he uploads the map files to aws and then push the changes here.

petrelharp · 2022-05-10T17:43:25Z

Ping @andrewkern - have you put the maps on AWS?

saurabhbelsare · 2022-05-10T19:11:15Z

Hi @jeromekelleher, @andrewkern,
I'm having a test fail, because in my stdpopsim/catalog/ensembl_info.py, the release of ensembl is 106, while the release in the main stdpopsim repo is 103, so the merge tests are conflicting and failing. I'm not sure why my file has the version 106, where it came from, since I haven't edited that file. @petrelharp suggested that you could weigh in on why this is happening and what is the best fix for it?

andrewkern · 2022-05-11T17:42:05Z

I've uploaded the PapAnu map to AWS. It can be found here:

https://stdpopsim.s3.us-west-2.amazonaws.com/genetic_maps/PapAnu/papio_anubis_genetic_map.tar.gz

@saurabhbelsare, you should be able to go ahead and edit the URL in your PR to reflect this

as for the issue with the ensembl release, i need to look in to this a bit. stay tuned.

grahamgower · 2022-05-16T09:53:27Z

Hey @saurabhbelsare. I opened #1248 about the ensembl release. For your PR, just edit the release number to make the test pass (either in the ensembl_release.py or in the test, it doesn't matter).

petrelharp · 2022-05-16T16:38:41Z

stdpopsim/catalog/PapAnu/demographic_models.py

+        id="SinglePopSMCpp_1W22",
+        description="SMC++ estimates of N(t) for Papio Anubis individuals",
+        long_description="""
+        These estimates were obtained from a sample Papio Anubis


Suggested change

These estimates were obtained from a sample Papio Anubis

These estimates were obtained from a sample pf Papio Anubis

petrelharp · 2022-05-16T16:39:26Z

stdpopsim/catalog/PapAnu/genetic_maps.py

+    id="Pyrho_PAnubis1.0",
+    description="Pyrho inferred genetic map for Papio Anubis",
+    long_description="""
+        These estimates were obtained from a sample Papio Anubis


Suggested change

These estimates were obtained from a sample Papio Anubis

These estimates were obtained from a sample of Papio Anubis

petrelharp · 2022-05-16T16:40:33Z

stdpopsim/catalog/PapAnu/species.py

+    common_name="Olive baboon",
+    genome=_genome,
+    generation_time=11,
+    population_size=335505,  # Most recent from Wall et al demographic model


Make sure to put a note about where you got this number in the QC issue, so the person doing QC knows where you got it from.

stdpopsim/catalog/PapAnu/species.py

saurabhbelsare mentioned this pull request Apr 13, 2022

QC for Papio Anubis (olive baboon) #1217

Closed

14 tasks

grahamgower reviewed Apr 21, 2022

View reviewed changes

grahamgower reviewed Apr 22, 2022

View reviewed changes

stdpopsim/catalog/PapAnu/species.py Outdated Show resolved Hide resolved

grahamgower mentioned this pull request Apr 22, 2022

CI tests should run in verbose mode #1222

Closed

saurabhbelsare force-pushed the olive_baboon branch 2 times, most recently from 42fe845 to 75347ae Compare May 10, 2022 18:40

saurabhbelsare force-pushed the olive_baboon branch from af160fe to 016dbe9 Compare May 15, 2022 21:18

petrelharp reviewed May 16, 2022

View reviewed changes

stdpopsim/catalog/PapAnu/species.py Outdated Show resolved Hide resolved

saurabhbelsare mentioned this pull request May 16, 2022

QC for SinglePopSMCpp_1W22 (Papio_anubis) #1254

Closed

saurabhbelsare force-pushed the olive_baboon branch 2 times, most recently from bd81434 to 2564915 Compare May 16, 2022 18:56

Added Papio Anubis genome, recombination map, and demographic model

155873a

saurabhbelsare force-pushed the olive_baboon branch from 2564915 to 155873a Compare May 16, 2022 18:58

petrelharp merged commit 5423db6 into popsim-consortium:main May 16, 2022

attrna mentioned this pull request May 16, 2022

QC for the demographic model for Papio Anubis #1261

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Papio Anubis genome, recombination map, and demographic model #1216

Added Papio Anubis genome, recombination map, and demographic model #1216

saurabhbelsare commented Apr 13, 2022 •

edited

Loading

grahamgower left a comment

grahamgower Apr 21, 2022

grahamgower Apr 21, 2022

saurabhbelsare Apr 21, 2022

grahamgower Apr 21, 2022

grahamgower commented Apr 22, 2022 •

edited

Loading

grahamgower commented Apr 22, 2022

saurabhbelsare commented Apr 27, 2022

petrelharp commented May 10, 2022

saurabhbelsare commented May 10, 2022

andrewkern commented May 11, 2022

grahamgower commented May 16, 2022 •

edited

Loading

petrelharp May 16, 2022

petrelharp May 16, 2022

petrelharp May 16, 2022

	These estimates were obtained from a sample Papio Anubis
	These estimates were obtained from a sample pf Papio Anubis

		black
		git+https://github.com/psf/black

Added Papio Anubis genome, recombination map, and demographic model #1216

Added Papio Anubis genome, recombination map, and demographic model #1216

Conversation

saurabhbelsare commented Apr 13, 2022 • edited Loading

grahamgower left a comment

Choose a reason for hiding this comment

grahamgower Apr 21, 2022

Choose a reason for hiding this comment

grahamgower Apr 21, 2022

Choose a reason for hiding this comment

saurabhbelsare Apr 21, 2022

Choose a reason for hiding this comment

grahamgower Apr 21, 2022

Choose a reason for hiding this comment

grahamgower commented Apr 22, 2022 • edited Loading

grahamgower commented Apr 22, 2022

saurabhbelsare commented Apr 27, 2022

petrelharp commented May 10, 2022

saurabhbelsare commented May 10, 2022

andrewkern commented May 11, 2022

grahamgower commented May 16, 2022 • edited Loading

petrelharp May 16, 2022

Choose a reason for hiding this comment

petrelharp May 16, 2022

Choose a reason for hiding this comment

petrelharp May 16, 2022

Choose a reason for hiding this comment

saurabhbelsare commented Apr 13, 2022 •

edited

Loading

grahamgower commented Apr 22, 2022 •

edited

Loading

grahamgower commented May 16, 2022 •

edited

Loading