THREESCALE-3927 Update Swagger to Open API 3.0

[nidhi-soni1104]

What this PR does / why we need it:

Update our own docs(sourcetoswagger) to OAS 3.0(Open API 3.0)

Which issue(s) this PR fixes

Main Task: https://issues.redhat.com/browse/THREESCALE-3927

There is a subtask: https://issues.redhat.com/browse/THREESCALE-8904 which is NOT part of this PR.

Special notes for your reviewer:

Outstanding issues:

1. Serialization does not work for POST /transactions.xml in Service Management API, see details in THREESCALE-3927 Update Swagger to Open API 3.0 #3103 (comment)
   UPDATE: this is fixed in Serialize Service Mgmt API's report endpoint body correctly #3294

2. Auto-complete does not work for all cases:

• doesn't work in the request body, see THREESCALE-3927 Update Swagger to Open API 3.0 #3103 (comment)
• also doesn't work for metrics in Service Management API, because the examples are provided as JSON objects, the metrics are not defined as separate fields
  I (@mayorova) think this is OK - we may outline this as a limitation in the release notes.

3. The /p/admin/api_docs/account_data endpoint is called multiple times - for every API, because it is part of the existing auto-complete logic, which is called for every SwaggerUI invocation. We may want to refactor this part, so that the account data is pulled just one time, and then reused by all of the APIs.

I (@mayorova) think that we need to somehow fix issue 1, and leave 2 and 3 for later.

Also, cleaning up the code and JSON specs that become "dead" after this upgrade will be done in another PR.

TODO:

• Tests for the ThreeScaleApiDocs JS module
• (Optional) rename the files in doc/active_docs to follow snake case (e.g. service_management_api.json ?

[nidhi-soni1104]

Yes I will remove rswag completely now

[nidhi-soni1104]

@jlledom Rswag is removed completely

[nidhi-soni1104]

The list of API is updated to open API 3.0 shown on the UI

Listed API's are
1.Analytics API
2.Billing API
3.Service Management API
4.Account Management API
5. Policy Registry API

Here is the screenshot attached to check how they will look

Billing API

Policy Registry API

[jlledom]

I observed that account_data.json is being called once per call. could we avoid that?

[jlledom]

I think this tags should be fixed:

[mayorova]

Didn't review/try properly, as I see there are quite a few change requests from @jlledom already.
just a few notes:

• I think the title of the PR needs to be updated, as it is confusing
• @nidhi-soni1104 I am wondering if you could provide some details (could be in the PR description) of what the different files are used for. Because, without further investigation I am a bit confused on why we have Service Management API (on-premises).json and Service Management API.json, and also why there is analytics-s20.json and also Analytics API.json.

I think the -s20.json files might have initially been added to upgrade to Swagger 2.0, but if we're upgrading to OpenAPI 3.0 already, then I think these files are not needed (and the code that is used to load these files either). There must be also a rolling upgrade, and if we are just upgrading the docs without rolling upgrade, then we will need to delete that code as well (as it's otherwise convert into dead code).

Also, I don't know what @3scale/system-ruby think, but I personally don't like spaces if filenames (even less as these files will eventually get loaded by the browser via URL), so maybe we could use this opportunity to rename the specification files and use hyphens instead (I'd prefer hyphens to underscores to be more URL-friendly).

[jlledom]

I think the -s20.json files might have initially been added to upgrade to Swagger 2.0, but if we're upgrading to OpenAPI 3.0 already, then I think these files are not needed (and the code that is used to load these files either). There must be also a rolling upgrade, and if we are just upgrading the docs without rolling upgrade, then we will need to delete that code as well (as it's otherwise convert into dead code).

Yes, there's some code to show some specs in OpenAPI v2 format. That code should be cleaned up. There also a javascript library ThreeScale.APIDocs that is currently being used to load the docs in OpenAPI v1 format, which could probably be removed too. I was going to suggest this later, when my current change requests were done.

Also, I don't know what @3scale/system-ruby think, but I personally don't like spaces if filenames (even less as these files will eventually get loaded by the browser via URL), so maybe we could use this opportunity to rename the specification files and use hyphens instead (I'd prefer hyphens to underscores to be more URL-friendly).

Agree

[nidhi-soni1104]

I think this tags should be fixed:

Tag problem is fixed also its UI is improved

[jlledom]

I observed that account_data.json is being called once per call. could we avoid that?

Could you work next on fixing this?

[nidhi-soni1104]

hreeScale.APIDocs

sure

[nidhi-soni1104]

@jlledom
I have checked the reason of keep loading account.json for mutliple time is due to every time we create a new dynamic div, we make a call to Open API to list the API and there this is the url given used as one of the parameter so that is the reason it keeps load and I tried but I don't think I found something which make render dynamic div and listing API in open API format and not load it every time.

[jlledom]

This is better 😎 👍

[jlledom]

It seems this is being exported only for tests. IMO this has became a private function now, so it shouldn't be exported. Tests shouldn't condition the code, they should adapt to the code. In this case, the tests to assert autocompleteOAS3 logic should be included as tests for the 'fetching OpenAPI spec' context of autocompleteRequestInterceptor test, since that's the 'unit' being exported. What do you think?

[mayorova]

It seems this is being exported only for tests. IMO this has became a private function now, so it shouldn't be exported. Tests shouldn't condition the code, they should adapt to the code.

That is right, and I agree with you that ideally the tests should adapt to the code, and not the other way around. On the other hand, when we code, we generally think about making the code testable (splitting codes in smaller units, adding dependency injection mechanism to allow for easier mocking of the dependencies, etc.).

In this case, I would really prefer to test the function autocompleteOAS3 itself, and not the only "public" function autocompleteRequestInterceptor.

I am not sure if there is another easy way to do this, without exporting the function (there are ways, but not too straightforward).

In this case, the tests to assert autocompleteOAS3 logic should be included as tests for the 'fetching OpenAPI spec' context of autocompleteRequestInterceptor test, since that's the 'unit' being exported.

I am not sure I understand what you mean 🙃

@josemigallas what do you think?

[jlledom]

I meant basically move 'should inject servers to the spec' and 'should autocomplete fields of OpenAPI spec with x-data-threescale-name property' to line 100 after 'should update the response interceptor'. But OK.

[mayorova]

@jlledom like this? 8200ed9

[jlledom]

Multiple calls to account_data are not fixed yet, is it going to be merged in another PR?

This would require a significant (IMO) refactoring in the OAS3Autocomplete.ts module, which will also affect service_active_docs.ts and active_docs.ts packs (along with the newly added provider_active_docs.ts, so I would prefer to leave it for later, to avoid this PR becoming even more complex.

After all the changes you made on OAS3Autocomplete.ts, is this still hard to do?

[mayorova]

After all the changes you made on OAS3Autocomplete.ts, is this still hard to do?

I don't see it as a trivial task, but I do agree that this might be actually a good time to fix it, as we are changing the interface for all three related packs anyway. I'll give it a spin tomorrow.

[mayorova]

@jlledom done in e6a7755

I'm wondering if @josemigallas will approve 😬