Jira structure: back to swagger schemas, but fix incorrect #5523

vovavovavovavova · 2021-08-19T10:22:26Z

What

Closes #5256

How

returned to previous schemas (discard auto-generated)
fixes: schemas, that had "items" root caused Source JIRA: certain stream schemas are not properly defined #4950 now have properties root
repaired some files like airbyte-integrations/connectors/source-jira/source_jira/schemas/avatars.json
It's schema, had both system and custom roots, when in the code we expand and get inner value. Now schema suits to output
add additionalproperties, where response contains.more fields, than written in schema. Some original schemas had additional properties as false, that caused error because of some fields.
add null values
remove parse_response_root from some classes. As example, class IssueVotes. Using this parse_response_root returns only the voters (users), but we need also the information about the votes https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-votes/#api-rest-api-3-issue-issueidorkey-votes-get . We skipped info hasVoted, votes amount, self. Those schemas and roots were repaired to return full information, not only voters. Several changes in streams have the same reason.
some other issues were fixed to pass the normalization through Airbyte UI

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed.

vovavovavovavova · 2021-08-19T10:22:48Z

/test connector=connectors/source-jira

🕑 connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1146579627
✅ connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1146579627

vovavovavovavova · 2021-08-19T11:28:04Z

airbyte-integrations/connectors/source-jira/source_jira/schemas/avatars.json

-}
+  "additionalProperties": false,
+  "description": "List of system avatars."
+}


todo add newline

vovavovavovavova · 2021-08-20T06:33:46Z

/test connector=connectors/source-jira

🕑 connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1149720958
✅ connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1149720958

gaart · 2021-08-20T14:50:30Z

airbyte-integrations/connectors/source-jira/source_jira/schemas/labels.json

@@ -1,33 +1,24 @@
 {
-  "$schema": "http://json-schema.org/schema#",
+  "type": ["object", "null"],


Should there be a
"$schema": "http://json-schema.org/draft-07/schema#",
too?

autogenerated schema always have this entry $schema. The swagger big hairy json is https://developer.atlassian.com/cloud/jira/platform/swagger.v3.json . Accordingly to it, the labels are https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-labels/#api-group-labels PageBeanString. In this json document it is described here root['components']['schemas']['PageBeanString'] and it just does not contain $schema entry. Schemas were restored to the state before #4947 and it seems that schemas, as well swagger one's just does not contain this entry. We can modify existing jsons and set it, BUT the previous version and swagger do not have it, and we use swagger.

vitaliizazmic

Could you provide some details about changes?

vovavovavovavova · 2021-08-31T15:45:11Z

Could you provide some details about changes?

upgraded description

yevhenii-ldv · 2021-09-02T13:59:02Z

/test connector=connectors/source-jira

🕑 connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1194467390
✅ connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1194467390

yevhenii-ldv · 2021-09-02T14:34:21Z

/publish connector=connectors/source-jira

🕑 connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1194598138
✅ connectors/source-jira https://github.com/airbytehq/airbyte/actions/runs/1194598138

vovavovavovavova added 3 commits August 17, 2021 17:30

revert schemas from response to previous

ab3e828

modify to be continued

c5edddf

forgot to save

25c5791

github-actions bot added the area/connectors Connector related issues label Aug 19, 2021

jrhizor temporarily deployed to more-secrets August 19, 2021 10:25 Inactive

vovavovavovavova commented Aug 19, 2021

View reviewed changes

vovavovavovavova changed the title ~~Valdemar/#5256 jira structure~~ Jira structure: back to swagger schemas, but fix incorrect Aug 19, 2021

vovavovavovavova added 2 commits August 19, 2021 17:26

fixing: newlines; permission_scheme normalization failure

ad7abc0

add nulls

a0dae30

jrhizor temporarily deployed to more-secrets August 20, 2021 06:36 Inactive

vovavovavovavova requested review from vitaliizazmic, gaart and keu August 20, 2021 06:45

gaart approved these changes Aug 20, 2021

View reviewed changes

vitaliizazmic reviewed Aug 25, 2021

View reviewed changes

vitaliizazmic approved these changes Aug 31, 2021

View reviewed changes

yevhenii-ldv self-assigned this Sep 2, 2021

yevhenii-ldv requested review from sherifnada, davinchia and tuliren September 2, 2021 06:48

sherifnada approved these changes Sep 2, 2021

View reviewed changes

ykurochkin added 2 commits September 2, 2021 16:52

merge with master

f2977d3

format

0ef30d5

jrhizor temporarily deployed to more-secrets September 2, 2021 14:01 Inactive

update changelog, bump version

763c95d

github-actions bot added the area/documentation Improvements or additions to documentation label Sep 2, 2021

jrhizor temporarily deployed to more-secrets September 2, 2021 14:37 Inactive

yevhenii-ldv merged commit 7216de8 into master Sep 2, 2021

yevhenii-ldv deleted the valdemar/#5256_jira_structure branch September 2, 2021 15:22

jrhizor mentioned this pull request Sep 3, 2021

Bump Airbyte version from 0.29.13-alpha to 0.29.14-alpha #5831

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jira structure: back to swagger schemas, but fix incorrect #5523

Jira structure: back to swagger schemas, but fix incorrect #5523

vovavovavovavova commented Aug 19, 2021 •

edited

Loading

vovavovavovavova commented Aug 19, 2021 •

edited by github-actions bot

Loading

vovavovavovavova Aug 19, 2021

vovavovavovavova commented Aug 20, 2021 •

edited by github-actions bot

Loading

gaart Aug 20, 2021

vovavovavovavova Aug 31, 2021

vitaliizazmic left a comment •

edited

Loading

vovavovavovavova commented Aug 31, 2021

yevhenii-ldv commented Sep 2, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Sep 2, 2021 •

edited by github-actions bot

Loading

Jira structure: back to swagger schemas, but fix incorrect #5523

Jira structure: back to swagger schemas, but fix incorrect #5523

Conversation

vovavovavovavova commented Aug 19, 2021 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

vovavovavovavova commented Aug 19, 2021 • edited by github-actions bot Loading

vovavovavovavova Aug 19, 2021

Choose a reason for hiding this comment

vovavovavovavova commented Aug 20, 2021 • edited by github-actions bot Loading

gaart Aug 20, 2021

Choose a reason for hiding this comment

vovavovavovavova Aug 31, 2021

Choose a reason for hiding this comment

vitaliizazmic left a comment • edited Loading

Choose a reason for hiding this comment

vovavovavovavova commented Aug 31, 2021

yevhenii-ldv commented Sep 2, 2021 • edited by github-actions bot Loading

yevhenii-ldv commented Sep 2, 2021 • edited by github-actions bot Loading

vovavovavovavova commented Aug 19, 2021 •

edited

Loading

vovavovavovavova commented Aug 19, 2021 •

edited by github-actions bot

Loading

vovavovavovavova commented Aug 20, 2021 •

edited by github-actions bot

Loading

vitaliizazmic left a comment •

edited

Loading

yevhenii-ldv commented Sep 2, 2021 •

edited by github-actions bot

Loading

yevhenii-ldv commented Sep 2, 2021 •

edited by github-actions bot

Loading