Api v2

[chinchliff]

New names and organization for version 2 of the opentree api.

These changes depend on some recent db structural changes introduced by taxomachine (already merged to master) which will necessitate a db update for oti. I have pushed an updated database to devapi and re-indexed the studies there as well.

@jar398, @jimallman, would you review this?

[jimallman]

@jar398: FYI, I'm reviewing this now...

[mtholder]

Is "is_deprecated" really a property? does it need an ot: namespace?

[chinchliff]

I don't know. It can certainly be changed in oti if we feel that it would
be worth designating an ot: property. Currently, that property is set to
true if the tree is tagged with anything matching the search string "del*".
Obviously this is hacky and not ideal. It would be better to have an
explicit property, but maybe we don't need this property now that curators
can delete trees?

On Wed, Sep 10, 2014 at 4:33 PM, Mark T. Holder notifications@github.com
wrote:

In TESTS.md:

•  "oti_tree_id" : "1495_tree3010"
  

• } ],
• "ot:studyId" : "1495"
• } ]
  +}
  +`

+### Get a list of properties available for searching studies and trees:
+
+bash +curl -X POST http://devapi.opentreeolife.org/ext/studies/graphdb/properties +
+
+`json
+{

• "tree_properties" : [ "ot:studyPublicationReference", "is_deprecated", "ot:focalCladeOTTTaxonName", "ot:studyLastEditor", "ot:studyModified", "ot:studyLabel", "ot:comment", "ot:studyId", "ot:dataDeposit", "ot:authorContributed", "ot:studyUploaded", "ot:studyYear", "ot:focalCladeTaxonName", "ot:tag", "ot:curatorName", "ot:studyPublication", "ot:focalCladeOTTId", "ot:focalClade" ],

Is "is_deprecated" really a property? does it need an ot: namespace?

—
Reply to this email directly or view it on GitHub
https://github.com/OpenTreeOfLife/oti/pull/26/files#r17386871.

[jimallman]

maybe we don't need this property now that curators can delete trees?

FWIW, the current curation app completely ignores the is_deprecated property. Also, I don't see any reference to this in the GitHub wiki or WikiSpaces wiki about NexSON.

It's briefly mentioned in this oti issue and it definitely sounds like legacy cruft.

[pmidford]

On 9/10/14 5:01 PM, Jim Allman wrote:

In TESTS.md:

•  "oti_tree_id" : "1495_tree3010"
  

• } ],
• "ot:studyId" : "1495"
• } ]
  +}
  +`

+### Get a list of properties available for searching studies and trees:
+
+bash +curl -X POST http://devapi.opentreeolife.org/ext/studies/graphdb/properties +
+
+`json
+{

• "tree_properties" : [ "ot:studyPublicationReference", "is_deprecated", "ot:focalCladeOTTTaxonName", "ot:studyLastEditor", "ot:studyModified", "ot:studyLabel", "ot:comment", "ot:studyId", "ot:dataDeposit", "ot:authorContributed", "ot:studyUploaded", "ot:studyYear", "ot:focalCladeTaxonName", "ot:tag", "ot:curatorName", "ot:studyPublication", "ot:focalCladeOTTId", "ot:focalClade" ],

maybe we don't need this property now that curators can delete trees?

FWIW, the current curation app completely ignores the |is_deprecated|
property. Also, I don't see any reference to this in the GitHub wiki
https://github.com/OpenTreeOfLife/phylesystem-api/wiki/NexSON or
WikiSpaces wiki http://opentree.wikispaces.com/NexSON about NexSON.

It's briefly mentioned in this oti issue
#9 and it definitely
sounds like legacy cruft.

—

Yes, it was a work around for studies that could not be deleted before
phylografter supported deletion. Agree it's legacy cruft.

Reply to this email directly or view it on GitHub
https://github.com/OpenTreeOfLife/oti/pull/26/files#r17388582.

[chinchliff]

Ok. I have made issue #27 to remove this property. Currently, I don't think it matters much if it's there, and the goal of this pull request is to get the new api services up and working this week. So it seems like now that we know what to do, we can deal with this later?

[jimallman]

@chinchliff: Oddly, I'm getting an error on exactly one study, pg_719 (all the others are indexed properly):

Indexing http://devapi.opentreeoflife.org/api/ study pg_719 from http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json
Calling "http://devapi.opentreeoflife.org/oti/ext/studies/graphdb/index_studies" with data="{'urls': ['http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json']}"

Indexing failed for URL "http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json", study http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json. Message = "More than one hit found for ot:studyId == 719. The database is probably corrupt."

Any chance this is confused by the presence of another "719" study? But I don't see a ot_719... at least not in the GitHub repo (will check the API server's "local" repo next).

[chinchliff]

Ugh. I expect it's because of the way the indexing was performed initially.
pg_719 is the first study to be indexed and it probably got screwed up
somehow. It would be nice not to have to re-build the entire db. We just
need to remove the study and then I expect indexing will proceed as
planned. I will think about this...

On Wed, Sep 10, 2014 at 10:33 PM, Jim Allman notifications@github.com
wrote:

In TESTS.md:

@@ -0,0 +1,65 @@
+Currently, there are no automated test procedures for OTI. You can however, fairly thoroughly test the performance manually. Run the following command to index the current phylesystem repo on devapi.opentreeoflife.org:
+
+```bash
+./index_current_repo.py http://devapi.opentreeoflife.org/oti http://devapi.opentreeoflife.org/api

@chinchliff https://github.com/chinchliff: Oddly, I'm getting an error
on exactly one study, pg_719 (all the others are indexed properly):

Indexing http://devapi.opentreeoflife.org/api/ study pg_719 from http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json
Calling "http://devapi.opentreeoflife.org/oti/ext/studies/graphdb/index_studies" with data="{'urls': ['http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json']}"

Indexing failed for URL "http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json", study http://devapi.opentreeoflife.org/api/default/v1/study/pg_719.json. Message = "More than one hit found for ot:studyId == 719. The database is probably corrupt."

Any chance this is confused by the presence of another "719" study? But I
don't see a ot_719... at least not in the GitHub repo (will check the
API server's "local" repo next).

—
Reply to this email directly or view it on GitHub
https://github.com/OpenTreeOfLife/oti/pull/26/files#r17401670.

[jimallman]

In case it helps, the error message seems to be coming from ot-base.

[chinchliff]

I think the easiest way is just to write a custom method to remove this
study and then run it. If I don't remember to do this, please remind me...

On Wed, Sep 10, 2014 at 10:49 PM, Jim Allman notifications@github.com
wrote:

In TESTS.md:

@@ -0,0 +1,65 @@
+Currently, there are no automated test procedures for OTI. You can however, fairly thoroughly test the performance manually. Run the following command to index the current phylesystem repo on devapi.opentreeoflife.org:
+
+```bash
+./index_current_repo.py http://devapi.opentreeoflife.org/oti http://devapi.opentreeoflife.org/api

In case it helps, the error message seems to be coming from ot-base
https://github.com/OpenTreeOfLife/ot-base/blob/9c0352af452518de3a3e1e17a96ad0bbd959934e/src/main/java/org/opentree/graphdb/DatabaseUtils.java#L115
.

—
Reply to this email directly or view it on GitHub
https://github.com/OpenTreeOfLife/oti/pull/26/files#r17402013.

[jimallman]

Sounds good. Meanwhile, there are two copies of the "Tippery N., 2011" study in oti. You can see them in the main study-list page by searching for its DOI: "10.1600/036364411X605092"

[jimallman]

@chinchliff: It looks like we both fixed the CURL tests. They're all returning sensible results at this point, so I'm going to merge this pull request. We can chase down the indexing problem above, and I'll look more into the apache config we'll need to support nicer URLs like /v2/studies/find_trees.