Migrate to GraphQL Streams

[vishalp-dev]

Description of change

This PR Migrates few streams to graphQl due to the upcoming sunset on 01/02/25

Integration tests will be updated in a different pr

Streams that are migrated:

• Products
• Inventory Items (prev. dependancy on products)
• Metafields
• Product Variants (new stream added)

What is changed in each stream

Products

- Primary Key int -> string
- Replication key updated_at -> updatedAt
- Removed Images and Image fields
- Variants now available as a new stream
- removed dependant streams (metafields and Inventory Items)
- other minor schema changes and new field additions

Inventory Items

- Primary Key int -> string
- Replication key updated_at -> updatedAt
- minor schema name and structure change, no new fields

Metafields

- Primary Key int -> string
- Replication key updated_at -> updatedAt
- minor schema name and structure change, no new fields
- Removed dependancy on parent streams (orders, inventory items, products, customers)
- All parents are fetched through graphql to maintain consistency between schema and primary keys

Product Variants

- New Stream addition (migrated from product stream)

Structural Changes

• Added base class implementation for graphQL Streams
• Added Queries for each graphQl stream

QA steps

• automated tests passing
• manual qa steps passing (list below)
• ran discovery and checked schema
• ran sync and checked records for transformation issues
• ran with bookmarks to confirm functionality
• checked pagination with lower page size limits
• Checked Record Count for Discrepancies

Risks

Rollback steps

• revert this branch

AI generated code

https://internal.qlik.dev/general/ways-of-working/code-reviews/#guidelines-for-ai-generated-code

• this PR has been written with the help of GitHub Copilot or another generative AI tool

[RushiT0122]

Suggested change

	cursor , has_next_page = page_info.get("endCursor"), page_info.get("hasNextPage")
	cursor, has_next_page = page_info.get("endCursor"), page_info.get("hasNextPage")

[RushiT0122]

I see amount, Price fields are marked string in schema, should those be singer.decimals?

[RushiT0122]

Minor fix, won't block the PR but get it fixed.

Suggested change

	resource_alias = {
	"customers":"customer",
	"products":"product",
	"collections": "collection",
	"orders": "order"
	}
	resource_alias = {
	"customers": "customer",
	"products": "product",
	"collections": "collection",
	"orders": "order"
	}

[RushiT0122]

Will wait for integration test PR to get merged for final approval.

[vishalp-dev]

I see amount, Price fields are marked string in schema, should those be singer.decimals?

I have checked the schema and updated the relevant fields to use singer.decimals

[bslayton]

@somethingmorerelevant Are there plans to update the other streams? Specifically events? I believe I found a bug related to events but it's irrelvant with the REST API being depreciated.