[ENH] BEP018: Genetic information

[CPernet]

I created the md file, and this looks fine
one issue, the json examples don't go the next line (despite spaces ??)

[franklin-feingold]

Hi @CPernet,

Thank you for converting and opening this PR! We can work on getting the formatting passed. Travis flagged a number of these formatting issues too. These issues could have resulted in the behavior you are seeing with the json examples. I can work on opening a PR on your fork

Edit: opened to render the examples and pass Travis

Edit2: flagging @bids-standard/everyone may be interested in reviewing!

[CPernet]

thx @franklin-feingold i think i fixed it (just pushed it in my fork)
--> atom beta whitespace package was being annoying ...
--> travis returned an issue with table ending which i did forget

[sappelhoff]

Thanks for opening the PR @CPernet - I made some comments regarding the formatting.

[sappelhoff]

I suggest to revise this paragraph to use our words MUST(=mandatory), SHOULD(=recommended), and MAY(=optional), see rfc2119.

This would also solve the issue that the first sentence is a bit misleading, because it says "two keys can be added", which sounds like both are optional

[rwblair]

Maybe GeneticDataBase could be mandatory if 'GeneticID' is present in any tsv files or if a genetic_info.json file is present.

[CPernet]

The Key GeneticDataBase MUST be added to link to the name of the database and web address. The key GeneticDescriptor MAY also be present refering to the descriptor (e.g. journal article) of the genetic data.

[CPernet]

'GeneticID' is not mandatory if the imaging repo and genetic repo use the same ID (which the validator cannot check)

[sappelhoff]

Suggested change

	This file is the descriptor of the genetic information available either in the participant tsv file and/or the genetic database described in the dataset_description.json. The 'GeneticLevel' and 'SampleOrigin' are the only two mandatory fields.
	This file is the descriptor of the genetic information available either in the participant tsv file and/or the genetic database described in the dataset_description.json. The `GeneticLevel` and `SampleOrigin` are the only two mandatory fields.

Use backticks to format field names

[sappelhoff]

When you give an example of a filepath in text, please format that path using backticks like: /this/is/a/path

[CPernet]

don't got that? i used eg /Study/Molecular what's wrong

[franklin-feingold]

Travis is flagging some of the formating. This can be resolved after the formatting is reviewed. I can assist with that at that time

[CPernet]

making changes in my fork

[CPernet]

@franklin-feingold @sappelhoff the build has work (i think) -- the only stuff i did not do was the / in the table, since that already how this was ??

[sappelhoff]

the only stuff i did not do was the / in the table, since that already how this was ??

The table fences will have to be fixed :-) could you do that please?

[franklin-feingold]

@CPernet should have fixed the formatting issue in my branch (https://github.com/franklin-feingold/bids-specification/blob/enh/genetics/src/04-modality-specific-files/08-genetic-descriptor.md) . My PR in your repo has some conflicts so perhaps can be easiest to grab my file and bring it over to your repo

[CPernet]

@franklin-feingold do you know why Travis failed? couldn't figure from 'details'

[sappelhoff]

@CPernet See my suggestions: Travis (=remark-lint) does not like too many empty lines :-)

[emdupre]

I do acknowledge however, that it seems to be hard to get more feedback on this BEP ... we announced it in several places but so far, no-one has chimed in.

August is a particularly hard month for feedback -- I'm writing this as I'm packing for a flight ! I think you'll be likely to see more community input in September.

All this to say, I'd be disappointed to see this merged before at least a few external members have chimed in. I've sent it to one of my colleagues working with genetic data, but they're on vacation ! So September seems safer to me.

[effigies]

I've added some specific comments, but on the whole I think this could do with an edit for terminological clarity. For instance, database and dataset are used somewhat interchangeably, and we use descriptor to refer to both the entire section, a .json file and an associated publication.

Possibly it would be useful to settle on specific vocabulary, introduce it either at the start of the section or at the end, and go through to conform.

[AlexandreHutton]

The dataset_description.json needs to be clarified. It appears inconsistent with the given example (and other references to it later in the document).
Does this spec allow for the data to be split across multiple files (e.g. by chromosome)? Is there supposed to be a convention for naming the genetics data?
I tried to see if it would be possible to apply this spec to imaging + genetic data I can access (UK Biobank), and it's unclear whether I could make the genetics data compliant with this.

[rwblair]

@CPernet Here's the start of the bids-validator PR:
bids-standard/bids-validator#828

This is checks for 'GeneticDatabase' in dataset_descrtiption.json if a genetic_info.json is present at the top level. It also add a json schema to validate genetic_info.json files.

Issues so far:

• The dataset_descriprion validation uses the flat structure first specified in this bep, not the nested one suggested by @AlexandreHutton. Should I implement the nested structure?
• It is checking for 'GeneticDatabase' instead of the 'GeneticDataBase' listed in the spec, I don't think we want that 3rd capitalized letter.
• It is only looking for genetic_info.json in the root level of the project. I wanted to confirm do we want multiple genetic_info.json files to be able to appear at any level, and assume that they are associated with the closest participants.tsv in the hierarchy? If this is the case I need to spend some time looking at how to test and enforce this.

Since this conversation is getting complex I'd be happy to continue the validator talk in the validator PR.

[CPernet]

@effigies i pushed the last changes but travis fails, can't figure out why? help please

[CPernet]

still fails :-(
thx anyway

[rwblair]

@CPernet I made a PR from my personal fork to your fork that might fix the problem. The github editor was too finicky for getting the spacing right.

[CPernet]

Success! thx - these trailings are so annoying

[CPernet]

@sappelhoff @effigies @franklin-feingold seems ready now? needs approval
we are making examples for the repo + validator testing

[franklin-feingold]

sounds good! perhaps for the specification and validator/examples to stay in lock step, the associated materials (e.g., validator) can be prepared so everything continues to stay in lock?

what do you all think?

[CPernet]

yes, since I PR to merge to master - better have the validator tested, good point

[effigies]

I've merged master, pushed this to the bep018 and added a ReadTheDocs build so its rendering can be seen: https://bids-specification.readthedocs.io/en/bep018/

@CPernet If it's okay with you, could we switch to that branch for the BEP018 PR? It will make making suggestions easier.

[effigies]

Closing in favor of #395.