🎉 New Source: PersistIq #9515

Zaimwa9 · 2022-01-14T13:58:07Z

What

This PR adds a new source: PersistIq api

Comments

Not sure about badge.md

Pre-merge Checklist

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here
After the connector is published, connector added to connector index as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
Documentation which references the generator is updated as needed.

This change is

CLAassistant · 2022-01-14T13:58:13Z

All committers have signed the CLA.

alafanechere · 2022-01-14T16:14:48Z

Hi @Zaimwa9, thank you for your contribution!
Could you please share evidence that the acceptance tests are passing. Could you also please run ./gradlew format if you did not already?
We need to create a sandbox account for PersistIq in order to run the acceptance tests in our CI.

alafanechere

@Zaimwa9 I tried to run the acceptance tests but they are failing, please make sure they run successfully and I'll go for a second review.

Zaimwa9 · 2022-01-17T09:13:05Z

Hey @alafanechere, Indeed, I was looking at the unit tests. I'm working on making them pass but I'm stuck on the read one.
https://airbytehq.slack.com/archives/C01MFR03D5W/p1642324171491000

Would love a bit of help to understand how to pass it :)
Thanks !

… into zaimwa9/persistiq-source

alafanechere · 2022-01-19T10:17:18Z

@Zaimwa9 I made some required changes on my side branch to make the acceptance pass. Could you please grant us permission to push on your fork (guide here) so that I can push my commits on it?

alafanechere

I made a first quick review of all the boilerplate files and detected some required changes.
Please check my comments.
I'll go for a python code review soon.

airbyte-integrations/connectors/source-persistiq/integration_tests/abnormal_state.json

airbyte-integrations/connectors/source-persistiq/source_persistiq/bootstrap.md

airbyte-integrations/connectors/source-persistiq/integration_tests/sample_state.json

...yte-integrations/connectors/source-persistiq/integration_tests/expected_campaigns_stream.txt

airbyte-integrations/connectors/source-persistiq/source_persistiq/schemas/campaigns.json

airbyte-integrations/connectors/source-persistiq/spec.json

docs/integrations/sources/persistiq.md

Zaimwa9 · 2022-01-19T14:25:27Z

Requested changes done, thanks :)

alafanechere

Minor comment and suggestions. Let me know what you think.

airbyte-integrations/connectors/source-persistiq/source_persistiq/source.py

alafanechere · 2022-01-19T18:47:35Z

airbyte-integrations/connectors/source-persistiq/source_persistiq/spec.json

@@ -0,0 +1,19 @@
+{
+  "documentationUrl": "https://docsurl.com",


Suggested change

"documentationUrl": "https://docsurl.com",

"documentationUrl": "https://docs.airbyte.io/integrations/sources/persistiq",

airbyte-integrations/connectors/source-persistiq/unit_tests/test_streams.py

alafanechere · 2022-01-19T18:52:21Z

Could you let me know when you give me maintainer permission to push my commits on your branch?

alafanechere · 2022-01-21T16:31:57Z

Thank you @Zaimwa9 for the changes. I'm running the test on our CI now and will let you know how it goes.

alafanechere · 2022-01-21T18:39:57Z

@Zaimwa9 thanks for your contrib, I made minor changes to make the test pass.

Zaimwa9 · 2022-01-26T12:36:56Z

Thanks a lot, couldn't look at the evolution earlier.

🎉

Wadii added 12 commits January 12, 2022 15:03

Spec json file for PersistIq

3981f0f

Implement check_connection for persistiq

71ce269

Read Users data from persistiq

8ccafcc

Read Leads data from persistiq and pagination

2a3ffd5

Read Campaigns data from persistiq

82cb3a1

Add persistiq logo

944c236

Completed unit tests

13dc51c

Source definitions for PersistIq

ddc306c

Added docs

45e7e4c

Added bootstreap.md

6b0c529

Removed comments and useless block

f164552

Removed comments in source.py

47c5190

github-actions bot added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Jan 14, 2022

octavia-squidington-iii added the community label Jan 14, 2022

Merge branch 'master' into zaimwa9/persistiq-source

2f751a3

alafanechere mentioned this pull request Jan 14, 2022

CI sandbox for PersistIq #9519

Closed

alafanechere self-assigned this Jan 14, 2022

alafanechere mentioned this pull request Jan 14, 2022

Publish 9515 #9520

Closed

alafanechere requested changes Jan 14, 2022

View reviewed changes

alafanechere temporarily deployed to more-secrets January 14, 2022 17:16 Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 14, 2022 17:17 Inactive

Zaimwa9 added 3 commits January 18, 2022 15:39

Passing acceptance tests

e886fe7

Expected campaigns records

e10493e

Merge branch 'zaimwa9/persistiq-source' of github.com:Zaimwa9/airbyte…

14ac923

… into zaimwa9/persistiq-source

Zaimwa9 requested a review from alafanechere January 18, 2022 14:42

alafanechere temporarily deployed to more-secrets January 18, 2022 18:49 Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 18, 2022 18:50 Inactive

alafanechere requested changes Jan 19, 2022

View reviewed changes

Deleted useless files

da571e1

Zaimwa9 requested a review from alafanechere January 19, 2022 14:25

alafanechere requested changes Jan 19, 2022

View reviewed changes

Zaimwa9 and others added 2 commits January 21, 2022 17:11

ran gradlew format

0e016c4

set expected output data according to our sandbox account data

492d773

fix tests

79d4dec

alafanechere temporarily deployed to more-secrets January 21, 2022 17:30 Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 21, 2022 17:31 Inactive

fix tests

5793a7b

alafanechere temporarily deployed to more-secrets January 21, 2022 17:40 Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 21, 2022 17:41 Inactive

fix source def

9434a94

alafanechere temporarily deployed to more-secrets January 21, 2022 17:57 Inactive

add doc link to spec

780baa2

alafanechere temporarily deployed to more-secrets January 21, 2022 17:59 Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 21, 2022 18:00 Inactive

update source specs

b4027bc

alafanechere temporarily deployed to more-secrets January 21, 2022 18:14 Inactive

alafanechere approved these changes Jan 21, 2022

View reviewed changes

alafanechere merged commit 1a74de6 into airbytehq:master Jan 21, 2022

octavia-squidington-iii mentioned this pull request Jan 21, 2022

Bump Airbyte version from 0.35.6-alpha to 0.35.7-alpha #9708

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 New Source: PersistIq #9515

🎉 New Source: PersistIq #9515

Zaimwa9 commented Jan 14, 2022 •

edited

Loading

CLAassistant commented Jan 14, 2022 •

edited

Loading

alafanechere commented Jan 14, 2022 •

edited

Loading

alafanechere left a comment

Zaimwa9 commented Jan 17, 2022

alafanechere commented Jan 19, 2022

alafanechere left a comment

Zaimwa9 commented Jan 19, 2022

alafanechere left a comment

alafanechere Jan 19, 2022

alafanechere commented Jan 19, 2022

alafanechere commented Jan 21, 2022

alafanechere commented Jan 21, 2022

Zaimwa9 commented Jan 26, 2022

	"documentationUrl": "https://docsurl.com",
	"documentationUrl": "https://docs.airbyte.io/integrations/sources/persistiq",

🎉 New Source: PersistIq #9515

🎉 New Source: PersistIq #9515

Conversation

Zaimwa9 commented Jan 14, 2022 • edited Loading

What

Comments

Pre-merge Checklist

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

CLAassistant commented Jan 14, 2022 • edited Loading

alafanechere commented Jan 14, 2022 • edited Loading

alafanechere left a comment

Choose a reason for hiding this comment

Zaimwa9 commented Jan 17, 2022

alafanechere commented Jan 19, 2022

alafanechere left a comment

Choose a reason for hiding this comment

Zaimwa9 commented Jan 19, 2022

alafanechere left a comment

Choose a reason for hiding this comment

alafanechere Jan 19, 2022

Choose a reason for hiding this comment

alafanechere commented Jan 19, 2022

alafanechere commented Jan 21, 2022

alafanechere commented Jan 21, 2022

Zaimwa9 commented Jan 26, 2022

Zaimwa9 commented Jan 14, 2022 •

edited

Loading

CLAassistant commented Jan 14, 2022 •

edited

Loading

alafanechere commented Jan 14, 2022 •

edited

Loading