Skip to content

feat: update linkedin script for backfill 🚀#828

Merged
ramiAbdou merged 24 commits intomainfrom
rami/go
Jun 21, 2025
Merged

feat: update linkedin script for backfill 🚀#828
ramiAbdou merged 24 commits intomainfrom
rami/go

Conversation

@ramiAbdou
Copy link
Member

@ramiAbdou ramiAbdou commented Jun 20, 2025

Description ✏️

This PR:

  • Updates the syncLinkedInProfiles function:
    • Supports a limit.
    • Handles a failed profile appropriately.
    • Moves all of the preprocessing logic for syncLinkedInProfiles to the Zod schemas for the LinkedIn profile.
  • Updates the Google Maps location functions to be cached before validating with the Zod schema. It also now supports the establishment and geocode types. It also removes caching from the getMostRelevantLocation function since we already cache the underlying methods that it uses.
  • Fixes a bug in the RateLimiter.doWhenAvailable function that decremented the active count in Redis by 1 too much.
  • Updates the runActor function to be independent of a Zod schema. This ensures that we don't not return the API scraping result when the data doesn't match the input. Also limits the actorId/body combination.
  • Updates the saveSchoolIfNecessary function to parse the result after it succeeds. It also looks for the address from the Google Places API using the establishment property.
  • Updates the saveCompanyIfNecessary function to parse the result after it succeeds.
  • Updates the education experience UI to support nullable dates.
  • Makes the schools.address_zip column nullable.

Type of Change 🐞

  • Feature - A non-breaking change which adds functionality.
  • Fix - A non-breaking change which fixes an issue.
  • Refactor - A change that neither fixes a bug nor adds a feature.
  • Documentation - A change only to in-code or markdown documentation.
  • Tests - A change that adds missing unit/integration tests.
  • Chore - A change that is likely none of the above.

Checklist ✅

  • I have done a self-review of my code.
  • I have manually tested my code (if applicable).
  • I have added/updated any relevant documentation (if applicable).

@ramiAbdou ramiAbdou self-assigned this Jun 20, 2025
@ramiAbdou ramiAbdou marked this pull request as ready for review June 21, 2025 00:54
@ramiAbdou ramiAbdou merged commit 5259d40 into main Jun 21, 2025
1 check passed
@ramiAbdou ramiAbdou deleted the rami/go branch June 21, 2025 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant