Update hubspot users and owners configurations

[earmenda]

Purpose

• Add/Update users and owners endpoints configurations for hubspot

Changes

• Update hubspot dataset config for owners and users
• Fix owners endpoint configuration as endpoint does not support query parameter filtering
• Add users endpoint configuration
• Remove marketing_emails comments from dataset config as there is no identity tied to this endpoint
• Change authentication method to use bearer for hubspot
• Update integration tests for new use cases addad

Checklist

• Update CHANGELOG.md file
  • Merge in main so the most recent CHANGELOG.md file is being appended to
  • Add description within the Unreleased section in an appropriate category. Add a new category from the list at the top of the file if the needed one isn't already there.
  • Add a link to this PR at the end of the description with the PR number as the text. example: #1
• Applicable documentation updated (guides, quickstart, postman collections, tutorial, fidesdemo, database diagram.
• If docs updated (select one):
  • documentation complete, or draft/outline provided (tag docs-team to complete/review on this branch)
  • documentation issue created (tag docs-team to complete issue separately)
• Good unit test/integration test coverage
• This PR contains a DB migration. If checked, the reviewer should confirm with the author that the down_revision correctly references the previous migration before merging
• The Run Unsafe PR Checks label has been applied, and checks have passed, if this PR touches any external services

Ticket

Fixes #901 #361 #878

[ghost]

This addition of user endpoints look good. But we are missing some things here:

1. We need to add assertions to verify results of both user access and erasure endpoints. Please take a look at assertions of other endpoints in file test_hubspot_task.py
2. We need to create some test data for user erasure endpoint, please have a look at function hubspot_erasure_data in hubspot_fixtures.py file and observe contact creation for our hubspot_erasure_identity_email

Let me know if you need any clarification/ help in understanding this.

[earmenda]

This addition of user endpoints look good. But we are missing some things here:

1. We need to add assertions to verify results of both user access and erasure endpoints. Please take a look at assertions of other endpoints in file test_hubspot_task.py
2. We need to create some test data for user erasure endpoint, please have a look at function hubspot_erasure_data in hubspot_fixtures.py file and observe contact creation for our hubspot_erasure_identity_email

Let me know if you need any clarification/ help in understanding this.

Thanks, that all makes sense! I had alot of difficulties with the integration tests because I had created the new private app key in a different account than the current api key. It's unfortunate because I was under the impression it was the same account so it was really confusing. Should be able to get this working now!

[earmenda]

@HamzaWaseemOnBench I made sure integration tests cover the new cases now

[earmenda]

There's still some things we need for this PR to be merged. If we want to use a new Hubspot account then we need to do the following to get integration tests to work:

1. Create private app in hubspot and insert the token into our vault under private_app_token.
2. Create integration test contact with email adaptors.india+saruser@ethyca.com
3. Create integration test user with email adaptors.india+saruser@ethyca.com

other todos:

• Figure out specific scopes needed for the hubspot private app
• Add this to docs^
• Do we need to also add owners to integration tests?

[earmenda]

@galvana @HamzaWaseemOnBench This should be ready for review. I ended up making the erasure request delete users as there was no data to mask. Let me know your thoughts!

[galvana]

Everything is working as expected! I linked two other issues that are resolved by these changes. The requested changes are more around testing preferences that have been established after this connector was developed.

[earmenda]

@galvana I removed #992 from the fixes as I think there is still some timing issues where these tests can fail. It's hard to reproduce but I think the reason is that the search endpoints are eventually consistent. Even though we use poll_for_existence, we might be hitting a different host. Maybe we can change the poll_for_existence function to have X number of successes for success.

The changes Hamza requested have been addressed

[seanpreston]

Since we're adding new assertions here, please consider asserting them in this style rather than looking for an exact match — hopefully we'd see less random failures moving forward.