How should we encode our different instances in SmartAPI?

[edeutsch]

One of our homework items for Architecture was to get our SmartAPI registry entries for dev, prod, and CI all tidied up, but what should it look like? I see at least 3 ways of doing it in use. Shall we settle on one way to do it?

ARAX has two entries, one for prod and one for dev and one is titled "dev":

ARAGORN also has two entries, but they are on the face of it (by looking at titles) exactly the same:

(although deep inside the "Servers" tag is different.

BTE has a single entry:

but has all three prod, dev, and CI listed under servers:

which seems moderately pleasing and is 3 valued, whereas the first two actors have only 2 out of the 3 (CI is missing)

Shall we recommend one way to do it?
Or should clients have to deal with the variability?

[edeutsch]

MolePro has 4 entries, with 2 entries for development:

I think I understood from the last call that we want to be able to distinguish between dev, prod, and ci?
Or maybe it was prod, test, and ci?
So shouldn't x-maturity have 3 values? production, development, and test?
And if we have 3 values, production, development, and test, does that mean that development is synonymous with CI?

Would be great to set this all out clearly somewhere. Or maybe it already is?

[ehinderer]

I recall having a discussion about this before, but cannot remember which meeting it was in. I thought the consensus was in line with how ARAX is doing it, with (dev) in the title.

[cbizon]

I suppose I lean towards having a single approach here. The variation is legal, but it's not terribly helpful.

It seems as though there are 2 (interrelated) questions.

1. How to handle different instances (one vs many registrations)
2. Do we have a convention for titles?

For the first, there will be times when we have to have more than one registration for different maturity servers (e.g. whenever those servers implement different versions of TRAPI). Given that, I think we should go ahead and say 1 instance per registration.

For the titles, I honestly don't see much difference either way. The main point (IMO) of the registry is for computers to read, and those consumers should be paying no attention to this name: it is the infores+x-maturity that defines the registration, and should be used in discovery. If we're always having fully independent registrations for different x-maturity then having the name defined like {infores} ({x-maturity}) is fine, but I don't think it adds much.

[codewarrior2000]

MolePro has 4 entries, with 2 entries for development:

I think I understood from the last call that we want to be able to distinguish between dev, prod, and ci? Or maybe it was prod, test, and ci? So shouldn't x-maturity have 3 values? production, development, and test? And if we have 3 values, production, development, and test, does that mean that development is synonymous with CI?

Would be great to set this all out clearly somewhere. Or maybe it already is?

One of the two "MolePro development instances/environments" (i.e., https://molepro-trapi.ci.transltr.io/molepro/trapi/v1.2) is the first point in the ITRB Jenkins/Docker pipeline. As the "Continuous Integration (CI)" point, ideally ITRB's Jenkins continuously pulls code changes from GitHub while they happen into this environment and cascade the changes to Test and Production.

The other "MolePro development instance/environment "(i.e., https://translator.broadinstitute.org/molepro/trapi/v1.2) is hosted by Broad Institute prior to and independently of the ITRB deployment process. Consequently, the codebase in the two instances are intended to be exactly the same.

[edeutsch]

Consequently, the codebase in the two instances are intended to be exactly the same.

It would be good to document whether it is okay for there to be more than one instance of each of the different levels (i.e. can there be 2 prod servers, 3 test servers?) and what other actors should do when encountering several (e.g. can they just pick one and use it, or SHOULD they or MUST they fail over to other instances if one fails, or SHOULD they send queries to all and use whichever comes back first?). Or if the consensus is that actors can do whatever they want, that's fine, we should just be clear about it and document it.

[edeutsch]

So shouldn't x-maturity have 3 values? production, development, and test?

I learned on the Architecture call today that there are 3 levels: production, development, and staging. There was some disagreement on how development, staging, test, and ci are all related to each other. It should be clearly documented.

[codewarrior2000]

Consequently, the codebase in the two instances are intended to be exactly the same.

It would be good to document whether it is okay for there to be more than one instance of each of the different levels (i.e. can there be 2 prod servers, 3 test servers?) and what other actors should do when encountering several (e.g. can they just pick one and use it, or SHOULD they or MUST they fail over to other instances if one fails, or SHOULD they send queries to all and use whichever comes back first?). Or if the consensus is that actors can do whatever they want, that's fine, we should just be clear about it and document it.

All in all, "TRAPI development service for MolePro" maybe a misnomer for this site: https://molepro-trapi.ci.transltr.io/molepro/trapi/v1.2 because it is just ITRB's deployment mirror of this Broad Institute site: https://translator.broadinstitute.org/molepro/trapi/v1.2

[cbizon]

Sounds like we have 4 environments:
ITRB Prod
ITRB Test
ITRB CI
Not-ITRB (eg. https://translator.broadinstitute.org/molepro/trapi/v1.2)

@newgene could we just have 4 x-maturity values, production, test, ci, development? Then they map very easily to the environments.

For multiple servers per environment - is this a function that people want/need? Is anybody doing this or planning on doing this?

[codewarrior2000]

Thank you, Chris, that perfectly encapsulates MolePro's order of progression: Not-ITRB, ITRB CI, ITRB Test, ITRB Prod.
It is far from MolePro's intention to have more than one API endpoint per environment.

[vdancik]

I would propose to use the following maturity terminology:

x-maturity	ITRB environment
production	ITRB Prod
testing	ITRB Test
staging	ITRB CI
development	Not-ITRB

[newgene]

Yes, we can use four stages in servers.x-maturity. Let me try to document down what we have discussed at the Arch meeting:

• servers.x-maturity values mapped to ITRB deployment types:

  • production (➡️ ITRB prod)
  • staging (➡️ ITRB Test)
  • development (➡️ ITRB CI)
  • testing (➡️ Non-ITRB, or anything else used during development process, can be multiple if you want)

I actually don't know the difference between ITRB Test v.s. ITRB CI. If we need to swap the mappings, I am fine with it too.

• You have the option to put your API metadata into one or multiple SmartAPI entries. The basic rules of thumb are:

  • If the difference between deployments is none or trivial/minimal, put multiple servers entries in the same API metadata entry.
  • Otherwise, you can register multiple SmartAPI entries corresponding to the multiple deployments you have.

• Filter or query for multiple deployments with SmartAPI API:

  https://smart-api.info/api/query?q=servers.x-maturity:production&fields=info,servers&tags=translator

  should give you the list of Translator APIs provides production server info. You can adjust query terms passed to q to fit your needs.

[vdancik]

I think we can register entries in smart-api.info according to TRAPI version (see https://github.com/NCATS-Tangerine/translator-api-registry/tree/master/molecular_data_provider). Right now we have same version thus we can have single specification file with multiple servers/maturity. When we get a new version, we register a new file and, as the API "matures", we will update server/maturity info in the new registration.

[newgene]

At BTE, we are implementing this logic to select servers.url based on the stages of "production|staging|development":

More details here at this issue: biothings/biothings_explorer#442

The basic logic is like this (e.g. for "production"):

• Use SmartAPI API (see my comment above) to filter for API entries with "production" value at servers.x-maturity
• Get servers.url values with the matching servers.x-maturity value of "production"
  • If only one value available, use it
  • If multiple values found, use one starts with https, otherwise, just use the first one.

Not sure if this fits your use cases, but post it here as a reference in case useful.

[cbizon]

So we have a few issues here:

1. Granularity of registrations
2. Are multiple servers allowed per infores/x-maturity ?
3. Mapping of x-maturity levels to environments
4. Conventions for names in registrations.

1 seems like the one we don't seem to be converging on necessarily. I see 3 possibilities:

A. One registration per infores/x-maturity 😄
B. One registration per infores/TRAPI version ❤️
C. It doesn't matter, clients have all the information they need. 🚀

I think any of these are workable. C is where we are now and it seems to work, but I can understand wanting something more rigid. Can I get a quick show of emojis on this comment for which others prefer?

[cbizon]

Excellent, seems like folks are in violent agreement. I've started a document to capture decisions here:
https://github.com/NCATSTranslator/TranslatorArchitecture/blob/registry/SmartAPIRegistration.md.

I haven't really added enough details though. If anybody is excited and wants to expand it, please do.

[newgene]

I communicated with ITRB team last week for the actual difference between ITRB-CI and ITRB-Test in terms of deployment process. This is the reply I got:

ITRB CI === Development (The application in this environment is triggered to be built and deployed whenever there is a change in the application github repository)
ITRB Test === Staging (The application in this environment in most time will be running the same version of code as in production. It is for pre-production testing and troubleshooting)
ITRB Production=== Production

In ITRB managed Test and Production environment, all deployments are based request. For example, you have a new commit in application repo then it is deployed to CI environment; after evaluation, you think this version can be used to release; you can submit a request in Slack or through email to us that you want to deploy it to Test; once it is deployed test, you will do integration test or more complicate testing process against it; once Test env passes all the testing cases, you may submit a request to deploy to production.

Based on this, I think we should adjust the mapping as:

x-maturity	ITRB environment
production	ITRB Prod
staging	ITRB Test
development	ITRB CI

Not sure if we still need "testing" as an official level, but people can use it for whatever extra developmental stage they need to use, as an optional stage.