Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken links to ontologies #257

Closed
digitalneoplasm opened this issue May 13, 2016 · 25 comments
Closed

Broken links to ontologies #257

digitalneoplasm opened this issue May 13, 2016 · 25 comments
Labels
specific ontology Issues related to a specific ontology; Consider using that ontology's tracker instead website Issues related to the OBO Foundry website

Comments

@digitalneoplasm
Copy link

As of a few minutes ago there are broken links to the following ontologies:
aero, cheminf, chmo, cmf, cvdo, ido, mirano, ncithesaurus, ogg, ogi, ogms, ogsf, rxno, and vto.

It seems the authors have not updated their purls since the google code shutdown. There are versions of aero, ido, ogg, ogi, ogms, ogsf, and vto available on BioPortal. cheminf, chmo, and rxno are available on their respective github pages. I was unable to find cvdo, mirano, ncithesaurus

For ncithesaurus the issue is more critical - http://www.obofoundry.org/ontology/ncithesaurus.html does not seem to exist. Its website (https://cabig.nci.nih.gov/concepts/EVS/) is also defunct.

@balhoff
Copy link
Contributor

balhoff commented May 13, 2016

I have an SVN checkout of the VTO releases. The files are too big for GitHub. I'll try to get these online elsewhere as soon as possible.

@cmungall
Copy link
Contributor

Thanks @digitalneoplasm we're on it

@balhoff if it's individual file size limits you're hitting you could try modularizing, e.g. ontodev/robot#39

@balhoff
Copy link
Contributor

balhoff commented May 17, 2016

For now I put VTO releases in S3. Pull request is #259.

@simonjupp
Copy link
Contributor

Here are the purls still failing. Is anyone chasing this up?

chmo - should be https://raw.githubusercontent.com/rsc-ontologies/rsc-cmo/master/chmo.owl
ogg - google code cc @yongqunh
mop - should redirect to https://raw.githubusercontent.com/rsc-ontologies/rxno/master/mop.owl
cvdi - google code, contact adrien.barton@gmail.com
rxno - should be https://raw.githubusercontent.com/rsc-ontologies/rxno/master/rxno.owl
ido - google code
cheminf - should be https://raw.githubusercontent.com/egonw/semanticchemistry/master/ontology/cheminf.owl
mirnao - google code contact vdritsou@imbb.forth.gr
ogsf - google code

@yongqunh
Copy link
Contributor

I am working on OGG…
I guess Asiyah Yu Lin is responsible for OGSF.
Oliver

From: simonjupp [mailto:notifications@github.com]
Sent: Monday, June 13, 2016 8:15 AM
To: OBOFoundry/OBOFoundry.github.io
Cc: He, Oliver; Mention
Subject: Re: [OBOFoundry/OBOFoundry.github.io] Broken links to ontologies (#257)

Here are the purls still failing. Is anyone chasing this up?

chmo - should be https://raw.githubusercontent.com/rsc-ontologies/rsc-cmo/master/chmo.owl
ogg - google code cc @yongqunhhttps://github.com/yongqunh
mop - should redirect to https://raw.githubusercontent.com/rsc-ontologies/rxno/master/mop.owl
cvdi - google code, contact adrien.barton@gmail.commailto:adrien.barton@gmail.com
rxno - should be https://raw.githubusercontent.com/rsc-ontologies/rxno/master/rxno.owl
ido - google code
cheminf - should be https://raw.githubusercontent.com/egonw/semanticchemistry/master/ontology/cheminf.owl
mirnao - google code contact vdritsou@imbb.forth.grmailto:vdritsou@imbb.forth.gr
ogsf - google code


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//issues/257#issuecomment-225564197, or mute the threadhttps://github.com/notifications/unsubscribe/AEetQ38tKta51lk1646H_ZfeOr1EqpBiks5qLUm6gaJpZM4IeMEu.


Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

@yongqunh
Copy link
Contributor

Hi Oliver and all,

Thanks for letting me know. Yes. I am responsible for the OGSF.
I will fix it within this week.

Thanks,
Asiyah

On Mon, Jun 13, 2016 at 9:20 AM, He, Oliver yongqunh@med.umich.edu wrote:

I am working on OGG…

I guess Asiyah Yu Lin is responsible for OGSF.

Oliver

From: simonjupp [mailto:notifications@github.com]
Sent: Monday, June 13, 2016 8:15 AM
To: OBOFoundry/OBOFoundry.github.io
Cc: He, Oliver; Mention
Subject: Re: [OBOFoundry/OBOFoundry.github.io] Broken links to
ontologies (#257)

Here are the purls still failing. Is anyone chasing this up?

chmo - should be
https://raw.githubusercontent.com/rsc-ontologies/rsc-cmo/master/chmo.owl
ogg - google code cc @yongqunh https://github.com/yongqunh
mop - should redirect to
https://raw.githubusercontent.com/rsc-ontologies/rxno/master/mop.owl
cvdi - google code, contact adrien.barton@gmail.com
rxno - should be
https://raw.githubusercontent.com/rsc-ontologies/rxno/master/rxno.owl
ido - google code
cheminf - should be
https://raw.githubusercontent.com/egonw/semanticchemistry/master/ontology/cheminf.owl
mirnao - google code contact vdritsou@imbb.forth.gr
ogsf - google code


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#257 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEetQ38tKta51lk1646H_ZfeOr1EqpBiks5qLUm6gaJpZM4IeMEu
.


Electronic Mail is not secure, may not be read every day, and should not
be used for urgent or sensitive issues

@selewis selewis added bug website Issues related to the OBO Foundry website specific ontology Issues related to a specific ontology; Consider using that ontology's tracker instead labels Oct 5, 2016
@matentzn
Copy link
Contributor

matentzn commented Dec 1, 2016

Hey!
For a range of ontology related experiments, I need a current snapshot of the entire OBO foundry ontology repository. I just tried downloading them all manually (from http://www.obofoundry.org/, except the obsolete ones), which resulted in 119 successful downloads. However, 28 failed, which are these ones:

xl.owl
vt.owl
tto.owl
trans.owl
tgma.owl
tads.owl
symp.owl
sep.owl
sbo.owl
pdro.owl
ontoneo.owl
egsf.owl
OGI.owl
ogg.owl
ncit.owl
miro.owl
mirnao.owl
idomal.owl
fix.owl
exo.owl
epidemiology_ontology.owl
DINTO.owl
ddanat.owl
emf.owl
eheminf.owl
el.owl
earo.owl
bto.owl

Is there a way I can get an up to date dump as a zip file including all of them? Is tried chasing down the ontologies using their project links, but failed with the first three attempts, so I though I might as well ask here.

@cmungall
Copy link
Contributor

cmungall commented Dec 1, 2016

Hi, we recently made a change in how we distribute some of the ontologies:

OBOFoundry/purl.obolibrary.org#261

The change went successfully, but it uncovered a hidden issue. For many ontologies the central build was failing, and what was being retrieved may have been a stale copy. This mostly applies to ontologies that are semi-abandoned. However, there are active ontologies like ddanat that are failing, and we will attend to these first (OBOFoundry/purl.obolibrary.org#266).

I note your list contains some ontologies that are not officially registered with OBO - for example OGI. In some cases you have the case wrong (e.g DINTO should be dinto)

OBO-tech people:

This is all documented in http://obofoundry.org/faq/what-is-the-build-field.html

The way to vet this is to look at the log output from the simple obo build job

For convenience, this can be done as

make jenkins-output.txt

in this repo. The most common cause of error here is a source obo file that does not declare an ontology tag. We must inject this, see:

build:
  source_url: ftp://ftp.informatics.jax.org/pub/reports/adult_mouse_anatomy.obo
  method: obo2owl
  infallible: 1
  insert_ontology_id: true

@cmungall
Copy link
Contributor

cmungall commented Dec 1, 2016

But really we should be pushing more groups to take control of their builds. Unfortuately a lot of important ontologies are released by their maintainers as simple obo files made with OE

@matentzn
Copy link
Contributor

matentzn commented Dec 1, 2016

Hey Chris, thanks for that. The DINTO download button for example does not even point to a file: It points to a GitHub repo home (https://github.com/labda/DINTO). And with respect to OGI, perhaps you are thinking about an ontology with the same id, but there is definitely something here: http://www.obofoundry.org/ontology/ogi.html

Just to be plain, is there a chance I can get these 28 ontologies somehow with a bit or URL hacking (trying the old berkeleybob etc), or is it best I wait until the issues are resolved? I need as much of the OBO foundry as I can get until Wednesday next week. What I dont have by then, I will simply ignore for the experiment. Cheers.

@cmungall
Copy link
Contributor

cmungall commented Dec 1, 2016

OGI - ah yes, I was getting confused with OGG.

OK, if you want a quick hack by wednesday, then for any failed PURL, try getting it from the old http://www.berkeleybop.org/ontologies/

@matentzn
Copy link
Contributor

matentzn commented Dec 1, 2016

Perfect, thanks!

@cmungall
Copy link
Contributor

cmungall commented Dec 1, 2016

I started working through the list; consolidated with @weasteam's list

See this ticket #76 on migrating ontologies from googlecode and sourceforge. It looks like despite repeated prodding a number of ontologies have failed to move

@TomConlin
Copy link

@cmungall
starting a new monarch release, our jenkins job load-scigraph-ontology-on-dev
fails with

....
WARNING: Failed to load ontology: http://purl.obolibrary.org/obo/upheno/monarch.owl  
org.semanticweb.owlapi.model.UnloadableImportException:   
Could not load imported     ontology: <http://purl.obolibrary.org/obo/upheno/metazoa.owl>   
Cause: Could not load imported ontology: <http://purl.obolibrary.org/obo/upheno/vertebrate.owl>   
Cause: Could not load imported ontology: <http://purl.obolibrary.org/obo/vt.owl>   
Cause: http://ontologies.berkeleybop.org/vt.owl
at  ...

http://ci.monarchinitiative.org/view/dev/job/load-scigraph-ontology-on-dev/lastBuild/console

cmungall added a commit to obophenotype/upheno that referenced this issue Dec 1, 2016
cmungall added a commit to OBOFoundry/purl.obolibrary.org that referenced this issue Dec 1, 2016
@matentzn
Copy link
Contributor

matentzn commented Dec 2, 2016

Hey, here a revised list of the few in question. (Sorry for the typos before, I copied them from an unreadable download log of chrome)

xl.owl - id: xl, name: Cross-linker reagents ontology, home: http://www.psidev.info/groups/controlled-vocabularies
egsf.owl -> http://www.obofoundry.org/ontology/ogsf.html
epidemiology_ontology.owl -- does not exist (id: epo, http://www.obofoundry.org/ontology/epo.html, download https://epidemiology-ontology.googlecode.com/files/epidemiology_ontology.owl does not resolve.
eheminf.owl - (you mean cheminf? YES) googlecode. Download does not resolve.
el.owl - no such ontology : SHOULD BE ceph.owl (id: ceph, name: Cephalopod Ontology, http://www.obofoundry.org/ontology/ceph.html, Download does not resolve).
earo.owl - no such ontology: SHOULD be caro, Common Anatomy Reference Ontology, Download does not resolve (not allowed) , http://ontologies.berkeleybop.org/caro.owl

@cmungall
Copy link
Contributor

cmungall commented Dec 2, 2016

ceph and caro fixed

(it looks like the CARO that was being resolved was "caro1". We made "caro2" some time ago but caro1 was never replaced: obophenotype/caro#13 -- in fixing this issue I just used caro2 as we have a github repo for this. See OBOFoundry/purl.obolibrary.org@2df5887)

@matentzn
Copy link
Contributor

matentzn commented Dec 5, 2016

Update: there are now only 24 ontologies with broken PURLs: out of those, 13 can be found on the old berkleybob pages, but the purl points to a NoSuchKey Error message. The remaining 11 have no listing on the old berkleybob pages, and mostly fail because of googlecode. However, at least four (dinto, cheminf, ogsf, pdro) should be super easliy fixable by redirecting the purl to github (they all have github repos). xl clearly does not exist (not even a OBO home page) and should be removed from the foundry web site listing (or at least made obsolete).

Berkleybob (13): All have NoSuchKey errors, but are available from the old berkleybob pages.
tto - NoSuchKey - #345
trans - NoSuchKey #343
tgma - NoSuchKey #346
tads - NoSuchKey #346
symp - NoSuchKey
sep - NoSuchKey
sbo - NoSuchKey #351
miro - NoSuchKey
idomal - NoSuchKey #346
fix - NoSuchKey
exo - NoSuchKey
ddanat - NoSuchKey
bto - NoSuchKey (not open. Not clear we have permission to reproduce)

Not on berkleybob / PURL does not resolve 404 (11):
xl - 404, http://obofoundry.org/ontology/xl.html - DELETE FROM LISTING
pdro - ontology exists, but PURL unresolved (http://www.obofoundry.org/ontology/pdro.html) #342 (Should be easy though as home link points to github which has the ontology clearly there)
ontoneo - ontology exists, http://www.obofoundry.org/ontoneo.com, has never worked, there is a github, suggested redirect: https://raw.githubusercontent.com/ontoneo-project/Ontoneo/master/ontoneo.owl #341
ogsf - ontology exists, refs to old googlecode, purl does not resolve. Interestingly, there is a PURL purl.bioontology.org/ontology/OGSF which points to BioPortal. From their newer github repository, I beleive the purl should point to https://raw.githubusercontent.com/linikujp/OGSF/master/src/ogsf-merged.owl
ogi - ontology exists, googlecode repo, not GitHub, AberOWL and Ontobee copies are there.
ncit - ontology exists, special case - #348
mirnao - ontology exists, googlecode
epo -- ontology exists, googlecode
DINTO.owl - ontology exists, purl does not resolve. Github repo exists: https://github.com/labda/DINTO/blob/master/DINTO1.2/DINTO_1.2.owl.zip, but ontolgy disseminated as many files in zip. Known issue: labda/DINTO#1
cheminf.owl - ontology exists,googlecode purl, but GitHub repo exists. Redirect PURL! https://github.com/egonw/semanticchemistry
cmf.owl - ontology exists, googlecode repo.

@cmungall
Copy link
Contributor

cmungall commented Dec 6, 2016 via email

@lschriml
Copy link
Contributor

lschriml commented Dec 8, 2016

I have updated the Transmission Process and Symptom Ontology URLs to their GitHub sites.
Cheers,
Lynn

@matentzn
Copy link
Contributor

Still broken links as of today:

rnao
sep

@cmungall
Copy link
Contributor

cmungall commented Mar 28, 2017 via email

@cmungall
Copy link
Contributor

cc @graybeal

cmungall added a commit that referenced this issue Sep 14, 2017
@jvendetti
Copy link
Contributor

jvendetti commented Sep 23, 2017

I encountered broken PURLs for:

cro: http://purl.obolibrary.org/obo/cro.owl
dpo: http://purl.obolibrary.org/obo/dpo.owl
ecocore: http://purl.obolibrary.org/obo/ecocore.owl
sepio: http://purl.obolibrary.org/obo/sepio.owl

... which didn't seem to be mentioned yet in comments above.

Also, xl is marked as fixed in a comment above, but the xl page results in a 404 error, and the PURL is broken: http://purl.obolibrary.org/obo/xl.owl

@matentzn
Copy link
Contributor

matentzn commented Nov 22, 2017

I produced another OBO Foundry dump today!
156 ontologies listed
143 downloadable (after 3 attempts)
13 (12) broken:

404 - Not found (2):
http://purl.obolibrary.org/obo/eo.owl (404-Not found)
http://purl.obolibrary.org/obo/ecocore.owl (404-Not found)

Berkleybob, no such key (6), (all same as 7 months ago)
http://purl.obolibrary.org/obo/idomal.owl
http://purl.obolibrary.org/obo/tads.owl
http://purl.obolibrary.org/obo/bto.owl
http://purl.obolibrary.org/obo/sep.owl
http://purl.obolibrary.org/obo/miro.owl
http://purl.obolibrary.org/obo/tgma.owl

Googlecode (4), (all same as 7 months ago)
http://purl.obolibrary.org/obo/ogi.owl (googlecode)
http://purl.obolibrary.org/obo/epo.owl (googlecode)
http://purl.obolibrary.org/obo/ogsf.owl (googlecode)
http://purl.obolibrary.org/obo/mirnao.owl (googlecode)

Works manually
Works if you try to download it manually, but it is not easy downloadable programmatically (SSL issue?)
http://purl.obolibrary.org/obo/mamo.owl

(sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
specific ontology Issues related to a specific ontology; Consider using that ontology's tracker instead website Issues related to the OBO Foundry website
Projects
None yet
Development

No branches or pull requests

10 participants