New Source: Lever Hiring

[sherifnada]

Tell us about the new integration you’d like to have

https://hire.lever.co/developer

Describe the context around this new integration

Describe the alternative you are considering or using

┆Issue is synchronized with this Asana task by Unito

[yevhenii-ldv]

Integration Vetting

Webhook-based?

no

Available authentication modes (API key/Oauth/other)

Access and Refresh Tokens can be used

Creating an account

@sherifnada @arhip11
I could not create an account for Lever in any way, I could only send a request for a demo.
I need help from Airbyte to create account for Lever.

How to populate the account with data?

Using API and UI.

Available streams for sync

• Applications
• Archive Reasons
• Audit Events
• Feedback
• Feedback Templates
• Files
• Interviews
• Notes
• Offers
• Opportunities
• Panels
• Postings
• Profile Forms
• Profile Form Templates
• Referrals
• Requisitions
• Requisition fields
• Resumes
• Sources
• Stages
• Tags
• Users

Integration supports incremental sync?

Some

Other information/blockers

[sherifnada]

Email sent to integration-test@airbyte.io titled "Product Integration with Lever - Next Steps" with next steps on how to create account

[johnlafleur]

@sherifnada we now have our own account on Lever. Don't hesitate to ping Anthony to get an access

[sherifnada]

@johnlafleur thanks for the heads up! We also have a sandbox we can use for testing ;) (separate from the recruiting one)

[sherifnada]

@karinakuz can you move this into the sprint once it's ready for implementation?

[karinakuz]

Lever

Integration Vetting

Webhook-based? (no/partially/yes)
if partially, mention which endpoints are webhook-based
yes https://hire.sandbox.lever.co/developer/documentation#webhooks

Available authentication modes (API key/Oauth/other)
Oauth (API Keys are used by customers to build internal workflows. Integrations are built using OAuth authentication.)

Connecting to the data source
what kind of API is it? (SOAP/GraphQL/REST/other)
REST API

is there an SDK available? Can we connect with HTTP requests directly?
no

If this is a database/datalake/pubsub source, does it have a JDBC driver or Java SDK?
no

Creating an account
yes, Sandbox access will last three months 

How to populate the account with data?

Can it be done entirely through the API?
no, we sent request to get token with write access, but no answer for more than a week

Will it require some work through the UI? If so, what is required?
yes

Will it require performing some paid action like running an Ad campaign? If so, please note the action needed, estimated cost, and estimated effort in hours. If a task will take more than 4 hours, please break it down into smaller tasks each of which take no more than 4 hours
no

Do you need any actions from Airbyte to populate the account with data? If so, please tag @ sherifnada for support
no

Available streams for sync
If this is an API connector:

1. Provide a link  to the reference docs
   _https://hire.sandbox.lever.co/developer/documentation#lever-api-reference
   Main streams:

• Applications https://hire.sandbox.lever.co/developer/documentation#list-all-applications
• Interviews https://hire.sandbox.lever.co/developer/documentation#list-all-interviews
• Offers https://hire.sandbox.lever.co/developer/documentation#list-all-offers
• Opportunities https://hire.sandbox.lever.co/developer/documentation#list-all-opportunities
• Postings https://hire.sandbox.lever.co/developer/documentation#list-all-postings
• Resumes https://hire.sandbox.lever.co/developer/documentation#list-all-resumes
• Sources https://hire.sandbox.lever.co/developer/documentation#sources
• Stages https://hire.sandbox.lever.co/developer/documentation#list-all-stages
• Users https://hire.sandbox.lever.co/developer/documentation#list-all-users_

2. Please mention if any of the endpoints are asynchronous (meaning we have to make a request, wait for the server to compute a response, then make another request to obtain the result)

if this is a source where the schema is dynamic/not fixed e.g: database, mention the data model of the connector i.e: relational, document-based, graph-based, etc.
no

Do any streams support incremental sync?
not necessary to mention all of them, just saying “none”, “some”, or “all” is sufficient. If only some streams, please mention which ones support incremental sync
some

Preparation scope & blockers
What information, materials, or anything else you will need from Airbyte in order to perform preparation for this connector?
no

Other information/blockers
place any other useful information you learned here
no

[karinakuz]

Populating the account with data
[ ] Do activities on the account: open/close 10 times, change profile data, fill account with the data manually (Lever didn't give write access to token). Create API for main streams and verify I get in response correct data from account. Add notes to LastPass Description (8h).

Get correct data in responses for GET requests:
Get Users https://api.sandbox.lever.co/v1/users
Get Opportunities https://api.sandbox.lever.co/v1/opportunities
Get Postings https://api.sandbox.lever.co/v1/postings
Get Sources https://api.sandbox.lever.co/v1/sources
Get Stages https://api.sandbox.lever.co/v1/stages
Get Resumes (for Opportunity) https://api.sandbox.lever.co/v1/opportunities/a739d309-dfae-408d-a316-7f06fcd9150f/resumes
Get Interviews (for Opportunity) https://api.sandbox.lever.co/v1/opportunities/784ff8e2-4536-4d65-8b29-e2dbeb0e8bef/interviews
Get Applications (for Opportunity) https://api.sandbox.lever.co/v1/opportunities/9ea413b6-432a-4806-a6cb-683e55dda4c2/applications
Get Offers (for Opportunity) https://api.sandbox.lever.co/v1/opportunities/2a4e55a9-42f6-4039-acf8-238ff099fa35/offers

We don't have Lever access to streams:

• audit_events
• contact
• feedback
• requisition_fields
• requisitions
• tags
• webhooks

[karinakuz]

Describing the output schema
[ ] Converting JSON to JSON Schema (30m)
https://drive.google.com/drive/folders/1H1XQkC5-avs97zZ9ELvpgTEmtbRaDKRN?usp=sharing

[yevhenii-ldv]

Scoping summary

1. What is the “ideal end state” for the features offered by this connector?

• Lever REST entities (subentities) map to Airbyte streams
• We will use client_id, client_secret and refresh_token to obtain access_token, and use it to send HTTP requests.

2. What is the release roadmap for this connector?

• Which features are absolutely necessary for the MVP release?

  I believe that all streams can be realized in MVP, since there are no particular problems to get each of them.

• Which features can be released as follow ups?

  So far there are none

• Task breakdown

  • Create source scaffolding and populate schemas folder
  • Implement spec, check, and discover
  • Implement Read
    • Create API base abstract class
  • Implement Full Refresh for all streams
  • Implement Incremental for streams, that support this feature
  • Handle rate limiting
  • Implement SAT

3. Check if and how Fivetran or Stitch Data support this connector

• Do they do something better than our proposed implementation?

  I believe that no, we can receive absolutely all streams that REST API allows us. Fivetran does the same.

• Is there something we can/should do better?

  We will remove duplicate elements in streams from other streams (for example, the parent stream's entity, minimizing it to the <parent_stream_name>_id)

Sheet with all streams here