Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2024 07 04 ai and form fields #211

Merged
merged 34 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
edf4aed
turns out getting the assistants API to work is a little bit of a lar…
Ches-ctrl Jul 4, 2024
fefdd88
ca marche!
Ches-ctrl Jul 4, 2024
024d58e
updates to outputs
Ches-ctrl Jul 4, 2024
4c425f3
update sidekiq
Ches-ctrl Jul 6, 2024
f6f26e3
update seeds
Ches-ctrl Jul 6, 2024
ba89b4b
puts & pretty_generate
Ches-ctrl Jul 6, 2024
ba6269c
we're getting somewhere
Ches-ctrl Jul 6, 2024
35f6852
getting there
Ches-ctrl Jul 6, 2024
2c2f05d
old file for now
Ches-ctrl Jul 6, 2024
ed0c5f0
better
Ches-ctrl Jul 7, 2024
dfb4bd1
updates and deletions
Ches-ctrl Jul 7, 2024
f4885be
comparing against old code
Ches-ctrl Jul 7, 2024
4d7c121
handle additional field sets
Ches-ctrl Jul 7, 2024
eac7d94
updating
Ches-ctrl Jul 7, 2024
f228418
delete capybara scrape
Ches-ctrl Jul 7, 2024
8acc5ca
add pretty generate
Ches-ctrl Jul 7, 2024
fb77e74
getting the fields from the API is ridiculously easier
Ches-ctrl Jul 7, 2024
9741f92
add old method in case
Ches-ctrl Jul 7, 2024
a3f5153
add testing rake task
Ches-ctrl Jul 7, 2024
fa05929
call service class for fields
Ches-ctrl Jul 7, 2024
dc6f051
update api comments
Ches-ctrl Jul 7, 2024
0447c21
pass job variable to gff
Ches-ctrl Jul 7, 2024
83b2c75
Setup testing
Ches-ctrl Jul 7, 2024
4c24552
setup for testing and add missing fields
Ches-ctrl Jul 7, 2024
95da870
add iframe for original link to job posting
Ches-ctrl Jul 7, 2024
1fafec8
remove iframe for now
Ches-ctrl Jul 7, 2024
e10cdc2
slight format update
Ches-ctrl Jul 7, 2024
bf9af52
add temporary link to original posting
Ches-ctrl Jul 7, 2024
3e24014
Add GetApiFieldsJob & fix seed
patanj101 Jul 9, 2024
47e0e81
get rid of getformfields
Ches-ctrl Jul 15, 2024
cba13af
call getapifields
Ches-ctrl Jul 15, 2024
8f7389d
create classs to inherit from
Ches-ctrl Jul 15, 2024
13ec639
Merge branch '2024-07-04-ai-and-form-fields' of github.com:Ches-ctrl/…
Ches-ctrl Jul 15, 2024
136c938
update empty file
Ches-ctrl Jul 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/assets/builds/tailwind.css

Large diffs are not rendered by default.

12 changes: 12 additions & 0 deletions app/jobs/importer/get_api_fields_job.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module Importer
class GetApiFieldsJob < ApplicationJob
include Sidekiq::Status::Worker

queue_as :importers
sidekiq_options retry: false

def perform(job)
Importer::GetApiFields.call(job)
end
end
end
4 changes: 2 additions & 2 deletions app/models/concerns/ats/greenhouse/application_fields.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ module Ats
module Greenhouse
module ApplicationFields
def get_application_criteria(job, _data)
job.application_criteria = CORE_FIELDS
# GetForm.perform
# job.application_criteria = Importer::GetFormFields.call(job)
job.application_criteria = Importer::GetApiFields.call
end

CORE_FIELDS = {
Expand Down
4 changes: 4 additions & 0 deletions app/services/faraday_helpers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,8 @@ def stream_xml_data(body)
def log_error(message)
Rails.logger.error(message)
end

def pretty_generate(json)
JSON.pretty_generate(json)
end
end
56 changes: 56 additions & 0 deletions app/services/importer/get_api_fields.rb
Copy link
Collaborator

@daniel-sussman daniel-sussman Jul 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JB suggested I take a look and offer suggestions. Here are a few thoughts I had:

  • could we call this alternatively with job_data rather than with a url? That avoids calling the API a second time to get the application_criteria after first calling it to get the job_details, which is unnecessary if they're both part of the same json response (as with GH). Other ATSs will require a new API call to get the application_questions anyway.
  • I would put the logic that builds @fields in a separate method, so it can be easily swapped out for different ATSs
  • likewise, fetch_json should go in a separate method. It's sometimes going to be more complicated than with GH. With ashby, for example, this request is a GraphQL query via a POST request with specific info in the request body and headers.
  • just fyi, running FormFiller with GH jobs is a bit trickier if you're getting the questions via API, because mostly the API doesn't give you any useful locators other than the text of the question itself. I did some experimenting and verified that it's possible to get FormFiller to work using the text of the questions as locators, although it runs somewhat more slowly than if we get the element IDs via GetFormFields.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is helpful context thanks Dan - noted on the updates to the Faraday helper and also calling with job_data instead - ideally we do this as you note (we'll need to build the URL carefully for Greenhouse in this case as perhaps we'll sometimes want to pass the additional params and sometimes not? Keen to discuss / hear thoughts)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a note on this - we can add further customisation into how Faraday makes the connection than what we're doing at the moment - think we might want to bring this in in future so just noting (e.g. custom user agent, proxy etc.)

Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
module Importer
# Core class for getting form fields directly from the API
# Splits based on category of fields - main, custom, demographic, eeoc
# Known Issues - Building the checkbox logic for the data_compliance section
# Allowable file types (Greenhouse): (File types: pdf, doc, docx, txt, rtf)
# NB. Must include all params to get additional fields from the API
class GetApiFields < ApplicationTask
include FaradayHelpers

def initialize
# @job = job
# @url = @job.api_url
@url = "https://boards-api.greenhouse.io/v1/boards/cleoai/jobs/4628944002"
# @url = "https://boards-api.greenhouse.io/v1/boards/cleoai/jobs/7301308002"
# @url = "https://boards-api.greenhouse.io/v1/boards/monzo/jobs/6076740"
# @url = "https://boards-api.greenhouse.io/v1/boards/axios/jobs/6009256"
# @url = "https://boards-api.greenhouse.io/v1/boards/11fs/jobs/4060453101"
# @url = "https://boards-api.greenhouse.io/v1/boards/forter/jobs/7259821002"
# @url = "https://boards-api.greenhouse.io/v1/boards/cleoai/jobs/4628944002"
@url += "?questions=true&location_questions=true&demographic_questions=true&&compliance=true&pay_transparency=true"
# @ats_sections = %w[main_fields custom_fields demographic_questions eeoc_fields data_compliance security_code_fields]
@fields = {}
@errors = false
end

def call
return unless processable

process
rescue StandardError => e
Rails.logger.error "Error running GetFormFields: #{e.message}"
nil
end

private

def processable
@url # && @job
end

def process
p "Hello from GetApiFields!"

json = fetch_json(@url)
return unless json

@fields['main_fields'] = json['questions']
@fields['demographic_questions'] = json['demographic_questions']
@fields['location_questions'] = json['location_questions']
@fields['data_compliance'] = json['data_compliance']

puts pretty_generate(@fields)
@fields
end
end
end
Loading
Loading