Update EcoCyc with new resource information

[kltm]

Update metadata and upstream resource sources for EcoCyc.

Tagging @pgaudet

[kltm]

#1959

[kltm]

@pgaudet I just wanted to make all of your ecocyc changes are on #1959, or if there were any elsewhere?

[kltm]

@pgaudet While I don't think it's a deal breaker, it's always desirable if all files are made available as gz. If the upstream could supply that for their GPI, that would be just that much better.

[kltm]

@pgaudet As of today, we can no longer run pipelines with the current metadata as

go-site/metadata/datasets/ecocyc.yaml

Line 19 in b71d70d

source: https://ecoliwiki.org/gaf/gene_association.ecocyc.gz

is now returning a 404 (it wasn't a few days ago).
I'm assuming that we need to update to #1959, or at least just the source, but I wanted to know if we could just take the whole thing.

[suzialeksander]

@lmoore207 do you have any information we may need at this time?

[lmoore207]

Hi Suzi, I didn’t get the notification about this issue, so thanks for letting me know. I’ve included Markus on this reply as he has been working with Pascale about the GO terms from EcoCyc. I don’t know if he is on GitHub, but I’m hoping he can provide more information regarding this. Regards, Lisa From: suzialeksander ***@***.***> Date: Thursday, February 16, 2023 at 3:21 PM To: geneontology/go-site ***@***.***> Cc: Lisa Moore ***@***.***>, Mention ***@***.***> Subject: [EXTERNAL] Re: [geneontology/go-site] Update EcoCyc with new resource information (Issue #1961) @lmoore207<https://urldefense.us/v3/__https:/github.com/lmoore207__;!!Nv3xtKNH_4uope0!0sjew1_Uzyynr0oyxpk87jt6k_OK5g93ZjcF55HiKHB2O9RutAtSJJKJtEbGVQ$> do you have any information? Specifically- is there an active replacement for this GAF at EcoCyc, and if so can you provide a link? — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/geneontology/go-site/issues/1961*issuecomment-1433873006__;Iw!!Nv3xtKNH_4uope0!0sjew1_Uzyynr0oyxpk87jt6k_OK5g93ZjcF55HiKHB2O9RutAtSJJK1M9a0uw$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/AEVRCZE2GDFUAKWDAKDMT5LWX2Y5VANCNFSM6AAAAAAU5JP37Y__;!!Nv3xtKNH_4uope0!0sjew1_Uzyynr0oyxpk87jt6k_OK5g93ZjcF55HiKHB2O9RutAtSJJJlmqHx8g$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[k--r]

@pgaudet While I don't think it's a deal breaker, it's always desirable if all files are made available as gz. If the upstream could supply that for their GPI, that would be just that much better.

OK, thanks for the feedback. I gzipped the ecocyc.gpi file. Available in same directory. I'll try to remember for the future.

[k--r]

Hi Suzi, I didn’t get the notification about this issue, so thanks for letting me know. I’ve included Markus on this reply as he has been working with Pascale about the GO terms from EcoCyc. I don’t know if he is on GitHub, but I’m hoping he can provide more information regarding this. Regards, Lisa

Hi Suzi and Lisa,
I'm just getting started on GH now, and this is only my 2nd reply I'm posting.
Pascale and I have been communicating about the ecocyc.gaf file and how to resolve some issues.

From: suzialeksander @.> Date: Thursday, February 16, 2023 at 3:21 PM To: geneontology/go-site @.> Cc: Lisa Moore @.>, Mention @.> Subject: [EXTERNAL] Re: [geneontology/go-site] Update EcoCyc with new resource information (Issue #1961) @lmoore207<https://urldefense.us/v3/https:/github.com/lmoore207;!!Nv3xtKNH_4uope0!0sjew1_Uzyynr0oyxpk87jt6k_OK5g93ZjcF55HiKHB2O9RutAtSJJKJtEbGVQ$> do you have any information? Specifically- is there an active replacement for this GAF at EcoCyc, and if so can you provide a link?

Suzi, yes, the plan is that the EcoCyc group will directly submit the ecocyc.gaf . One change is that the older versions from EcoliWiki seem to have converted to UniProt IDs, whereas now, EcoCyc IDs (for proteins, etc.) will be in the file. The file also includes things like RNAs, which seem to have been missing in previous files.

The current experimental version of the ecocyc.gaf, for release 26.5, can be found at:
https://www.ai.sri.com/~kr/go/latest/ecocyc.gaf.gz

[kltm]

In discussion with @pgaudet , to temporarily have ecoli data so that pipelines can tick over again, while we sort out identifiers, we've decided to use https://ftp.ebi.ac.uk/pub/databases/GO/goa/proteomes/18.E_coli_MG1655.goa as a stopgap.

[kltm]

@pgaudet Ugh, naturally the switch to the new file rather upset some of the watchdogs we have out looking for weirdness (i.e. ecocyc as a resource suddenly disappeared), so we lost a couple of snapshot loads. As another TODO note here: we may need to revert the change I'm making to the sanity checks.

[kltm]

After talking to @pgaudet, as a temporary fix to upstream format issues, I've fixed locally

awk -F"\t" '{OFS=FS}{ $13 = "taxon:"$13 ; print   }' 18.E_coli_MG1655.goa > ecoli.gaf

and will aim at that for the short term.

[kltm]

@pgaudet Are the recent changes the final ones for this issue?

[suzialeksander]

not quite. the downloads page is not displaying a file type in the File field:

[ecocyc.gaf] ()

@kltm has fixed this.

[kltm]

Talking to @pgaudet , the current state is "good". Closing.

[cmungall]

I'm just documenting the history here for posterity/transparency

• previously we pulled from https://ecoliwiki.org/gaf/gene_association.ecocyc.gz, but ecocyc is no longer updated, and this doesn't have anything after 2022
• Update ecocyc.yaml #1959 proposed to instead get this from ecocyc (using EcoCyc gene IDs as primary) - this was closed
• Updated source file to GOA ftp #1981 replaced this PR, this gets E coli annotations from UniProt (using uniprot IDs as primary)

I assume there are processes in place at uniprot to pull manual annotations from EcoCyc (but it looks like there are no new manual annotations from EcoCyc from 2023 or 2024)

[cmungall]

It looks like there is nothing in place to pull the latest annotations from EcoCyc into GOA

Annotations to cydC:
https://ecocyc.org/ECOLI/NEW-IMAGE?type=ENZYME&object=ABC-6-CPLX

This was recently shown to be a heme transporter

cydD	involved_in	GO:0035351 - heme transmembrane transport [Wu23]

This is not in
https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=P23886

And hence not in GO Central annotations

[pgaudet]

Do you know when this annotation was made?

We can also @alexsign when was the last EcoCyc load; this is in the GOA loads, see https://ftp.ebi.ac.uk/pub/contrib/goa/GO.annotation_sources

Also, we would need to know how often EcoCyc produces their GAF, but usually discrepancies are due to lags in the different release cycles.

[cmungall]

Do you know when this annotation was made?

It was in the December 08, 2023 release
https://ecocyc.org/ecocyc/release-notes.shtml

Also, we would need to know how often EcoCyc produces their GAF, but usually discrepancies are due to lags in the different release cycles.

We can ask them to create a GAF with every release.

Note we have no manual annotations from 2023 or 2024

I think this is just a communication issue. I don't know if @alexsign uses a github-tracked metadata file like we do to indicate the URL source for loading into Protein2GO/QuickGO but I expect this needs to be changed or we need to reinform the EcoCyc team that we are in fact using this.

[alexsign]

@cmungall I do use https://github.com/geneontology/go-site/blob/master/metadata/datasets/ for many import pipelines, but EcoCyc is not one of them. I use GAF and GPI data at the URLs bellow.
https://www.ai.sri.com/~kr/go/latest/ecocyc.gaf.gz
https://www.ai.sri.com/~kr/go/latest/ecocyc.gpi.gz

[cmungall]

@alexsign it looks like we should follow up with the EcoCyc team. Those URLs don't look like permanent URLs as they point to a user's home directory. I'm guessing that this was set up as a one-off and not updated.

@k--r and @lmoore207 -- can you shine any light on this? Are these the URLs we should be using?

[pgaudet]

Pinged Markus and Lisa by mail.

[cmungall]

It looks like:

1. https://www.ai.sri.com/~kr/go/latest/ecocyc.gaf.gz is now kept up to date with the latest annotations, I see annotations as recent as 20240522. I assume this will remain the canonical location and will be updated.
2. Alex is picking these up and incorporating these into the uniprot annotations. I checked cydC (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=P23886) which has the heme transporter annotations from EcoCyc from 20231005, which was previously missing.

Thanks everyone!