Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP sources fail to sync (HTTP GET Request) #2910 #3008

Merged
merged 5 commits into from
Apr 30, 2021
Merged

HTTP sources fail to sync (HTTP GET Request) #2910 #3008

merged 5 commits into from
Apr 30, 2021

Conversation

vovavovavovavova
Copy link
Contributor

@vovavovavovavova vovavovavovavova commented Apr 21, 2021 •
edited by keu
Loading

closes #2910

What

Supports array of Objects in source_http_request

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

Recommended reading order

  1. test.java
  2. component.ts
  3. the rest

@keu keu requested review from keu, vitaliizazmic and yevhenii-ldv and removed request for jrhizor and ChristopheDuong April 21, 2021 13:03
keu
keu reviewed Apr 21, 2021
View reviewed changes
airbyte-integrations/connectors/source-http-request/source_http_request/source.py
)

return (m for m in [message])
data = r.json()
Copy link
Contributor

@keu keu Apr 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, lets make it DRY

if not isinstance(data, list):
   data = [data]

for record in data:
    yield AirbyteMessage(
                    type=Type.RECORD,
                    record=AirbyteRecordMessage(
                        stream=SourceHttpRequest.STREAM_NAME,
                        data=record,
                        emitted_at=int(datetime.now().timestamp()) * 1000
                    )
                )

keu
keu suggested changes Apr 21, 2021
View reviewed changes
Copy link
Contributor

@keu keu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion

@keu keu linked an issue Apr 21, 2021 that may be closed by this pull request
HTTP sources fail to sync (HTTP GET Request) #2910
Closed
@keu keu changed the title Valdemar/#2910 1 HTTP sources fail to sync (HTTP GET Request) #2910 Apr 21, 2021
@vovavovavovavova
Copy link
Contributor Author

vovavovavovavova commented Apr 28, 2021 •
edited by github-actions bot
Loading

/test connector=source-http-request

🕑 source-http-request https://github.com/airbytehq/airbyte/actions/runs/792684364
✅ source-http-request https://github.com/airbytehq/airbyte/actions/runs/792684364

@keu keu self-requested a review April 28, 2021 15:41
@vovavovavovavova
Copy link
Contributor Author

vovavovavovavova commented Apr 30, 2021 •
edited by github-actions bot
Loading

/test connector=source-http-request

🕑 source-http-request https://github.com/airbytehq/airbyte/actions/runs/800697565
✅ source-http-request https://github.com/airbytehq/airbyte/actions/runs/800697565

@keu
Copy link
Contributor

keu commented Apr 30, 2021 •
edited by github-actions bot
Loading

/publish connector=connectors/source-http-request

🕑 connectors/source-http-request https://github.com/airbytehq/airbyte/actions/runs/800873940
❌ connectors/source-http-request https://github.com/airbytehq/airbyte/actions/runs/800873940

@keu keu merged commit 3569b68 into master Apr 30, 2021
@keu keu deleted the valdemar/#2910-1 branch April 30, 2021 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@keu keu keu approved these changes

@sherifnada sherifnada sherifnada approved these changes

@vitaliizazmic vitaliizazmic Awaiting requested review from vitaliizazmic

@yevhenii-ldv yevhenii-ldv Awaiting requested review from yevhenii-ldv

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

HTTP sources fail to sync (HTTP GET Request)
3 participants
@vovavovavovavova @keu @sherifnada