-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Detect and identify missing metagenomes #245
Comments
How are samples tagged as having metagenome data? Do you mean those samples that have an OTU with the special metaxa_from_metagnome amplicon, or something else? The only tag I'm aware of is the 'type:amdb-metagenomics-analysed tag on CKAN packages. |
there should be a query that the metagenomics tag on the CKAN web portal uses . If you look on the CKAN portal on the lefthand side under tags you will see the metagenomics tag and its currently showing 1893 samples- you might need to ask Mark.T for the exact query he uses for that.
… On 18 Oct 2022, at 9:21 am, David Houlder ***@***.***> wrote:
@Smithmania <https://github.com/Smithmania>
This would probably be acheived by making a list of all IDs tagged as metagenomes
How are samples tagged as having metagenome data? Do you mean those samples that have an OTU with the special metaxa_from_metagnome amplicon, or something else? The only tag I'm aware of is the 'type:amdb-metagenomics-analysed tag on CKAN packages.
—
Reply to this email directly, view it on GitHub <#245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSSEDMU5W3QVPNZVPPSCJLWDXGQJANCNFSM6AAAAAARE5LNRI>.
You are receiving this because you were mentioned.
|
@Smithmania: This might almost be a one-liner using the CKAN python API. Something along the lines of… package_search(q='type:(not amdb-metagenomics-analysed)', fq='tags:metagenomics') … might work. I will investigate. |
This seems to work ckan_remote_object.action.package_search(
q='(tags:metagenomics) AND NOT (type:amdb-metagenomics-analysed)',
rows=3000)['results']) 1893 results |
Clarifying the data model for the production of these CKAN datasets. Datasets with tags:metagenomics and not of type:amdb-metagenomics-analysed are processed to generate new datasets that have tags:metagenomics and are of type:amdb-metagenomics-analysed. Only "input" datasets for this process that have res_format:FASTQ are eligible for processing. |
Sounds about right as far as the current tags/types are named
I found the 2 sampleIDs that don’t have fastq data so we should be able to check your logic
… On 19 Oct 2022, at 11:37 am, David Houlder ***@***.***> wrote:
Clarifying the data model for the production of these CKAN datasets.
Datasets with tags:metagenomics and not of type:amdb-metagenomics-analysed are processed to generate new datasets that have tags:metagenomics and are of type:amdb-metagenomics-analysed. Only "input" datasets for this process that have res_format:FASTQ are eligible for processing.
—
Reply to this email directly, view it on GitHub <#245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSSEDPSAQX5KGNXJVERQBLWD47DLANCNFSM6AAAAAARE5LNRI>.
You are receiving this because you were mentioned.
|
OK, how about a "special" URL, say, |
can we set up a meeting tomorrow morning to discuss - does 10 am work for you?
… On 24 Oct 2022, at 1:59 pm, David Houlder ***@***.***> wrote:
OK, how about a "special" URL, say, /metagenome/status to report this? There's already a precedent for this kind of thing as we have the undocumented /ingest/ URL for the ingest report.
—
Reply to this email directly, view it on GitHub <#245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSSEDKBZWTW4FARYMSQNJ3WEX3SDANCNFSM6AAAAAARE5LNRI>.
You are receiving this because you were mentioned.
|
OK. 10:00AM Tuesday.
On Sun, 2022-10-23 at 20:51 -0700, Smithmania wrote:
can we set up a meeting tomorrow morning to discuss - does 10 am work for you?
On 24 Oct 2022, at 1:59 pm, David Houlder ***@***.***> wrote:
OK, how about a "special" URL, say, /metagenome/status to report this? There's already a precedent for this kind of thing as we have the undocumented /ingest/ URL for the ingest report.
—
Reply to this email directly, view it on GitHub <#245 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSSEDKBZWTW4FARYMSQNJ3WEX3SDANCNFSM6AAAAAARE5LNRI>.
You are receiving this because you were mentioned.
—
Reply to this email directly, view it on GitHub<#245 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWMZYYURL2WGSC3DXZCOKVDWEYBSNANCNFSM6AAAAAARE5LNRI>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Modify metagenome download functionality to mimic the non-denoised amplicon behaviour. In this case MGSD files requested by a user for selected amplicon(s) will be emailed via the bpa help desk to the analytics team. The analytics team will manually distribute the requested files in consultation with the user. As no MGSD data will be housed on CKAN, available metagenome samples should be identified by searching tags
We will need to re-think how the MG search is done.
|
Just to be clear: you still want to be able to restrict by sample context search (e.g. lat, long, vegetation type, environment etc.), yes? i.e. you don't want something as plain as the non-denoised search where all you get to filter by is sample id.
One possibility is to completely wildcard the amplicon part of the search. This should be possible without any dramatic ill effects. The only downside I can think of right now is that there will be more taxonomy selection options at every rank, as the choices will be built from all available values for that rank. (e.g. the Kingdom dropdown would include k_fungi as well as d_Archaea, and every other option ever available in the kingdom dropdown). A further possibility is to remove the taxonomy dropdowns altogether in the metagenome search page. Also, can you confirm that there's no metagenome data for the following samples? These are samples that have no associated otu or taxonomy info at all.
|
I dont see any datasets returned on ckan (using search |
@Smithmania
|
Implemented in https://github.com/BioplatformsAustralia/bpaotu/tree/1.36.0 (see 1f1647a ) In metagenome mode, the amplicon selector can be set to '--', which selects every sample tagged as having metagenome data, regardless of taxonomy. |
Can we add a feature that prints text like you use for files that are not available (e.g., " not available") for metagenomes that are missing from the MGSD tags? This would probably be acheived by making a list of all IDs tagged as metagenomes and comparing to the list polled from MGSD tags and finding the difference.
This feature will be beneficial for us and end users to track the progress of MG analysis.
The text was updated successfully, but these errors were encountered: