logic to pick API server urls

[colleenXu]

Right now, when an API (TRAPI or not) is ingested by BTE, BTE picks the first entry in the server list of the SmartAPI yaml.

However, smarter logic would be nice (what if the first entry isn't the production or encrypted url). Also the Translator consortium may want ARAs to be able to query different sets of KP API servers ("production" vs "development" vs "staging").

Here's some ideas for how this logic would work (will have to check the priority of staging vs development):

• production: BTE will only use an API if it has a server url with x-maturity:"production". If it doesn't have any entries with the x-maturity:"production", BTE won't use it (and will log that this is happening).
  • if there's more than 1 server url with x-maturity:"production", we could pick the first entry....or try smarter logic like picking the encrypted url (https)
• staging: BTE will use the server urls with x-maturity:"staging" when possible. If an API only has "production" urls, BTE can use those too (with the logic above for production stuff).
• development: BTE will use the server urls with x-maturity:"development" when possible. If an API only has "staging" urls, BTE can use those. If an API only has "staging" and "production" urls, BTE will use the "production" urls (with the logic above for production stuff).

Example SmartAPI registrations with multiple server urls to pick from:

• COHD: http://smart-api.info/registry?q=51c178099fa2dc99b5d8fff8bf9f1a0d
• MyGene: http://smart-api.info/registry?q=59dce17363dce279d389100834e43648

Note that right now some APIs have separate registrations for their production levels, like ICEES right now...

[colleenXu]

May be related to https://github.com/biothings/smartapi-kg.js module

[rjawesome]

I can start working on this, just wondering when I'm done would I make a pull request on the relevant module?

[andrewsu]

correct! If you haven't figured it out already, check with @tokebe on which module this pertains to...

[tokebe]

@colleenXu Could you clarify a few things here, because I'm a little confused with your original post:

• Is the priority order here (earlier being "more preferred"): development -> production -> staging?
• Where does x-maturity: testing fit into this priority order?
• And just to confirm, an API should not be used at all if it doesn't have a production x-maturity level?

[colleenXu]

@tokebe @rjawesome

When I reread this issue, I see it as "can a deployed BTE instance use TRAPI api server urls that match that deployment's x-maturity level"? This is for the TRAPI APIs that are listed in the config file of BTE (aka we ingest them).

It seems to me to be a bigger issue than the smartapi-kg package....like maybe there needs to be an environmental variable or something that tells BTE what x-maturity level it's being deployed at?

I think we can ignore the non-TRAPI / x-bte APIs, since their x-maturity situations don't map onto whatever Translator/ITRB is doing...

---

For example, imagine our non-ITRB instance (x-maturity:development):

• ideally, it would use the TRAPI api server urls that are also "x-maturity:development". So it would use "https://dev.cohd.io/api" for COHD (see the servers section of the registered yaml))
  • what if there's still more than 1 url? Maybe pick one that starts with https:// (aka encrypted?)?
  • And if that still leaves more than 1 url, just pick the first one?
• But what if the TRAPI api we ingest doesn't have a server url tagged "x-maturity:development"? We could do the following logic:
  • does the api have a server url tagged "x-maturity:staging" (ITRB CI)? Then use that (see logic above for > 1 url if there's more than 1 staging url).
    • So currently, BTE would do this for Connections Hypothesis Provider, which has a "x-maturity:dev" endpoint that we'll skip (since it's not the exact keyword "development") and then an "x-maturity:staging" url: http://chp-dev.thayer.dartmouth.edu. So we'd use this one.
  • does the api have a server url tagged "x-maturity:testing" (ITRB test)? Then use that (see logic above for > 1 url if there's more than 1 testing url)
  • does the api have a server url tagged "x-maturity:production" (ITRB prod)? Then use that (see logic above for > 1 url if there's more than 1 production url).
    • So currently, BTE would do this for the Automat APIs since they only seem to 1 "production"-tagged url.

[colleenXu]

More examples:

For BTE's ITRB-CI instance (x-maturity:staging):

• It would not use any TRAPI api server urls that are "lower" aka "development".
• It would use TRAPI api server urls that are also "staging", or "testing", or "production", in that order of preference

For BTE's ITRB-test instance (x-maturity:testing):

• It would not use any TRAPI api server urls that are "lower" aka "development" and "staging".
• It would use TRAPI api server urls that are also "testing", or "production", in that order of preference

For BTE's ITRB-prod instance (x-maturity:production):

• It would not use any TRAPI api server urls that are "lower" aka "development", "staging", and "testing".
• It would use TRAPI api server urls that are also "production"

[colleenXu]

I think it's worth asking if there's still confusion regarding this issue, stuff to consider regarding ITRB and how to make BTE different on different instances, and whether we want to modify the logic I just described in the past 2 posts....

[rjawesome]

@colleenXu If an environment variable is added for the BTE maturity level, I could modify my code to change the priorities of maturity levels based on the BTE maturity level (my code already gives priority to HTTPS if same maturity level). Just had a question, How do I tell if an API is a TRAPI / x-bte API (ie. if the API should be put through the priority system or not)?

[tokebe]

@rjawesome You can use the environment variable INSTANCE_ENV, which will be set to ci test or prod depending on the ITRB instance. If it's relevant we can also set it to dev on our dev instance.

[rjawesome]

Alright, I don't think a dev environment variable is necessary as if it isn't ci, test, or prod we can assume it is dev. I was still wondering about how I tell if an API is a TRAPI / x-bte API (ie. if the API should be put through the priority system or not)?

[tokebe]

Closed by automation, reopening until deployed to Prod

[tokebe]

Deployed to prod, marking as completed.