Skip to content

feat(gooddata-sdk): report whether create_or_update created or updated - #1705

Merged
hkad98 merged 2 commits into
gooddata:masterfrom
vondravl:feat/upsert-outcome
Aug 5, 2026
Merged

feat(gooddata-sdk): report whether create_or_update created or updated#1705
hkad98 merged 2 commits into
gooddata:masterfrom
vondravl:feat/upsert-outcome

Conversation

@vondravl

@vondravl vondravl commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What

Adds UpsertOutcome (CREATED / UPDATED) and returns it from all nine create_or_update* methods:

service method
catalog_workspace create_or_update, create_or_update_workspace_setting, create_or_update_user_data_filter, create_or_update_filter_view
catalog_user create_or_update_user, create_or_update_user_group
catalog_organization create_or_update_jwk, create_or_update_export_template
catalog_data_source create_or_update_data_source

Why

Each of these already does an existence check and branches on it, then discards the answer:

try:
    found_workspace = self.get_workspace(workspace.id)
    ...update...
except NotFoundException:
    ...create...

A caller that needs to know which happened — to log accurately, count creations, or skip follow-up work — has to repeat the GET the SDK just made, because create_entity_* / update_entity_* are reachable only through the private _entities_api. This returns information the SDK has already computed.

The concrete case that prompted it: a deploy tool logging "Workspace %s created successfully" after every create_or_update, reporting a creation on every re-upload of an existing workspace. The only public-API fix is a duplicate get_workspace() call.

Should this be in the SDK at all?

I genuinely don't know, and I'd rather surface the trade-off than argue one side. Happy to close this if the answer is no — the workaround (an extra GET in the caller) is cheap.

Reasons to take it

  • The information exists and is thrown away. No new I/O, no new failure mode.
  • It removes a duplicate round-trip from every consumer that currently needs the answer.
  • Backward compatible in practice: every in-repo caller (gooddata-pipelines, gooddata-dbt, gooddata-eval) ignores the return value, and adding a return to a -> None method breaks no caller or type checker.
  • There's recent precedent for surfacing already-computed results — feat(gooddata-eval): report created metric/automation id from single-turn evaluators #1694 (report created metric/automation id from single-turn evaluators).

Reasons to reject it

  • It may be a deliberate API-surface choice. Void upserts keep "the entity now looks like this" as the whole contract. Returning the branch invites callers to depend on a distinction the HTTP API itself doesn't promise.
  • The outcome is best-effort, not authoritative. The existence check is not atomic with the write that follows, so a concurrent create/delete can make the reported branch wrong. The enum docstring says so, but a returned value tends to get trusted more than a docstring caveat. If anyone treats it as an audit record, that's a footgun the current None doesn't have.
  • It widens the public API by nine signatures plus a new exported symbol, to serve a fairly narrow need — arguably the caller's own get_* probe is the honest place to pay for that knowledge.
  • Nine methods change shape at once. Doing only the one I needed would be worse (arbitrary inconsistency), but the all-or-nothing framing makes the smallest useful version of this change fairly wide.
  • A future true-upsert endpoint (single PUT, no pre-GET) would make the pre-check — and this return value — obsolete or harder to compute.

For what it's worth on intent: I could find no recorded rationale for -> None here. Returns: None is a blanket docstring template across ~80 catalog methods including plain voids like delete_workspace; there's no comment or doc note explaining it; and -> None dates to the original NAS-3058 implementation rather than a later removal. The closest relative, set_hll_type, documents its idempotency but is silent on the outcome. That's absence of evidence, not evidence of absence — the deliberate-choice reading above is still entirely possible, and you'd know better than the history does.

Notes on the implementation

  • UpsertOutcome(str, Enum) rather than enum.StrEnumrequires-python is >=3.10 and StrEnum landed in 3.11. Matches the existing SortDirection(str, Enum) precedent.
  • An enum rather than a bool: for an "or update" operation the bool is the entire return value, so if sdk.catalog_workspace.create_or_update(ws): reads as nonsense. It also leaves room for UNCHANGED later.
  • The three methods with an "id is None" path (workspace_setting, user_data_filter, filter_view) return CREATED on that branch, but see the pre-existing defect below — that branch cannot currently execute, so the docstrings describe it as unreachable rather than promising a creation.

Testing

  • Assertions on both branches added to the existing cassette tests (workspace, workspace setting, user data filter, jwk, user, user group, data source). No cassette needed re-recording — the outcome is derived from the already-recorded interactions, so the assertions double as a check that CREATED/UPDATED matches what the server actually did.
  • New tests/catalog/test_upsert_outcome.py adds mock-based unit tests for what no cassette reaches: filter views, export templates, and the update branch of user data filters. These stub only the branch-deciding getter, since the branch is chosen by whether that GET raises NotFoundException.
  • ruff 0.15.1 (the pinned pre-commit version) check + format clean; packages/gooddata-sdk: 498 passed, 2 skipped, 3 xfailed.

A pre-existing defect this turned up

Writing those tests showed that the id is None create branches cannot run at all — in create_or_update_filter_view, create_or_update_user_data_filter, and create_or_update_workspace_setting.

Each serializes through a generated model that requires a str id, and passing None fails type validation before any request is made:

ApiTypeError: Invalid type for variable 'id'. Required value type is str and passed type was NoneType at ['id']

This holds for the PostOptionalId variants too, where "optional" means omit the key, not accept None. It predates this PR — the branches were already unreachable; this change only added a return to them.

I have not fixed it here, as it's a separate concern in the entity models / generated client. It's pinned by strict xfail tests so that fixing it trips them and prompts a revisit, and the docstrings no longer claim those paths create anything. Happy to split it into its own issue or PR if you'd like.

Docs

Five create_or_update* pages had a hardcoded Returns: None and are updated (two different legacy formats in use). The remaining four methods have no docs pages.

Summary by CodeRabbit

  • New Features

    • Create-or-update operations now report whether an item was created or updated.
    • Added the public UpsertOutcome type with CREATED and UPDATED values.
    • Applies across data sources, users, user groups, workspaces, settings, filters, views, JWKs, and export templates.
  • Documentation

    • Updated SDK documentation to describe the new return values and possible outcomes.
  • Tests

    • Added and updated coverage verifying created and updated results across supported operations.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6420ecaf-a13e-499a-bea6-5e4952008cae

📥 Commits

Reviewing files that changed from the base of the PR and between 17bcb5c and b8993c4.

📒 Files selected for processing (16)
  • docs/content/en/latest/administration/organization/create_or_update_jwk.md
  • docs/content/en/latest/administration/user-groups/create_or_update_user_group.md
  • docs/content/en/latest/administration/users/create_or_update_user.md
  • docs/content/en/latest/data/data-source/create_or_update_data_source.md
  • docs/content/en/latest/workspace/workspaces/create_or_update.md
  • packages/gooddata-sdk/src/gooddata_sdk/__init__.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/data_source/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/organization/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/types.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/workspace/service.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_data_source.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_organization.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_user_service.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_workspace.py
  • packages/gooddata-sdk/tests/catalog/test_upsert_outcome.py
🚧 Files skipped from review as they are similar to previous changes (16)
  • packages/gooddata-sdk/src/gooddata_sdk/init.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_organization.py
  • docs/content/en/latest/administration/users/create_or_update_user.md
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/data_source/service.py
  • docs/content/en/latest/administration/organization/create_or_update_jwk.md
  • docs/content/en/latest/data/data-source/create_or_update_data_source.md
  • packages/gooddata-sdk/tests/catalog/test_catalog_data_source.py
  • packages/gooddata-sdk/tests/catalog/test_upsert_outcome.py
  • docs/content/en/latest/administration/user-groups/create_or_update_user_group.md
  • packages/gooddata-sdk/tests/catalog/test_catalog_workspace.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/organization/service.py
  • docs/content/en/latest/workspace/workspaces/create_or_update.md
  • packages/gooddata-sdk/tests/catalog/test_catalog_user_service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/types.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/workspace/service.py

📝 Walkthrough

Walkthrough

The SDK adds the public UpsertOutcome enum and returns CREATED or UPDATED from catalog upsert methods. Documentation and tests cover data sources, organization resources, users, user groups, workspaces, and related resources.

Changes

Upsert outcome reporting

Layer / File(s) Summary
Outcome contract and service implementations
packages/gooddata-sdk/src/gooddata_sdk/catalog/types.py, packages/gooddata-sdk/src/gooddata_sdk/__init__.py, packages/gooddata-sdk/src/gooddata_sdk/catalog/{data_source,organization,user}/service.py, docs/content/en/latest/administration/..., docs/content/en/latest/data/...
Added and exported UpsertOutcome. Data-source, JWK, export-template, user, and user-group upserts return CREATED or UPDATED.
Workspace resource outcomes
packages/gooddata-sdk/src/gooddata_sdk/catalog/workspace/service.py, docs/content/en/latest/workspace/workspaces/create_or_update.md
Workspace, workspace-setting, user-data-filter, and filter-view upserts return documented CREATED or UPDATED outcomes.
Outcome behavior validation
packages/gooddata-sdk/tests/catalog/test_catalog_*.py, packages/gooddata-sdk/tests/catalog/test_upsert_outcome.py
Tests verify returned outcomes, selected API operations, string enum values, and current ID-less creation failures.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: lupko, pcerny

Poem

A rabbit checks each upsert trail,
“CREATED” and “UPDATED” mark the tale.
The enum hops into the SDK,
Tests check each branch carefully.
Docs record the result.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 48.08% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: create-or-update methods now report whether they created or updated an entity.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.44444% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.59%. Comparing base (17bcb5c) to head (b8993c4).

Files with missing lines Patch % Lines
...-sdk/src/gooddata_sdk/catalog/workspace/service.py 90.32% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1705      +/-   ##
==========================================
+ Coverage   78.40%   78.59%   +0.19%     
==========================================
  Files         271      271              
  Lines       18741    18772      +31     
==========================================
+ Hits        14693    14754      +61     
+ Misses       4048     4018      -30     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vondravl
vondravl force-pushed the feat/upsert-outcome branch from 54285a1 to d9fb3ba Compare August 4, 2026 09:05
@vondravl
vondravl marked this pull request as ready for review August 4, 2026 09:13

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py`:
- Around line 37-39: Document that UpsertOutcome returned by
create_or_update_user, create_or_update_user_group, create_or_update,
create_or_update_workspace_setting, create_or_update_user_data_filter, and
create_or_update_filter_view is best-effort and reports the SDK-selected branch
rather than authoritative resource existence. Apply the same qualification in
packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py lines 37-39 and
103-105; packages/gooddata-sdk/src/gooddata_sdk/catalog/workspace/service.py
lines 71-73, 164-170, 1234-1240, and 1444-1450. Update the result documentation
in docs/content/en/latest/administration/organization/create_or_update_jwk.md
lines 21-23,
docs/content/en/latest/data/data-source/create_or_update_data_source.md lines
23-26,
docs/content/en/latest/administration/user-groups/create_or_update_user_group.md
lines 22-25,
docs/content/en/latest/administration/users/create_or_update_user.md lines
22-25, and docs/content/en/latest/workspace/workspaces/create_or_update.md lines
18-21 to state that each result is best-effort.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 48de74c1-5569-47e0-91c8-c42851ee5c16

📥 Commits

Reviewing files that changed from the base of the PR and between acfcc1a and d9fb3ba.

📒 Files selected for processing (16)
  • docs/content/en/latest/administration/organization/create_or_update_jwk.md
  • docs/content/en/latest/administration/user-groups/create_or_update_user_group.md
  • docs/content/en/latest/administration/users/create_or_update_user.md
  • docs/content/en/latest/data/data-source/create_or_update_data_source.md
  • docs/content/en/latest/workspace/workspaces/create_or_update.md
  • packages/gooddata-sdk/src/gooddata_sdk/__init__.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/data_source/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/organization/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/types.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py
  • packages/gooddata-sdk/src/gooddata_sdk/catalog/workspace/service.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_data_source.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_organization.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_user_service.py
  • packages/gooddata-sdk/tests/catalog/test_catalog_workspace.py
  • packages/gooddata-sdk/tests/catalog/test_upsert_outcome.py

Comment thread packages/gooddata-sdk/src/gooddata_sdk/catalog/user/service.py
ValidObjects = dict[str, set[str]]


class UpsertOutcome(str, Enum):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we would like to eventually switch this to StrEnum – we cannot right now because we have a support for Python 3.10 which does not support StrEnum.

Consider adding the following:

__str__ = str.__str__

It should make switch to StrEnum a noop.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in c618ee2 — added __str__ = str.__str__ on UpsertOutcome.

Verified it makes the eventual StrEnum swap a no-op: str(UpsertOutcome.CREATED) / f"{...}" / "%s" % / json.dumps all yield "created" instead of "UpsertOutcome.CREATED", while repr(), == "created" and UpsertOutcome("created") are unchanged. Pinned it in test_upsert_outcome.py::test_outcome_is_a_plain_string (str(outcome) == outcome.value + f-string), so a future base-class change that regresses this trips the test.

vondravl and others added 2 commits August 5, 2026 07:58
Every create_or_update* method already performs an existence check and
branches on it, then discards the answer and returns None. Callers that need
to know -- to log accurately, to count creations, to skip follow-up work --
have to repeat the same GET the SDK just made, because the create/update
entity calls are reachable only through the private _entities_api.

Return a new UpsertOutcome (CREATED / UPDATED) from all nine of them so the
information the SDK already computed is no longer thrown away. Adding a
return value to a method previously annotated -> None is backward compatible:
every in-repo caller (gooddata-pipelines, gooddata-dbt, gooddata-eval)
ignores the result and is unaffected.

The outcome is best-effort by nature -- the existence check is not atomic
with the write that follows -- which the enum docstring states so it is not
mistaken for an authoritative audit record.

Assertions on both branches are added to the existing cassette tests, and
mock-based unit tests cover what no cassette reaches: filter views, export
templates, and the update branch of user data filters.

Writing those tests turned up a pre-existing defect: the `id is None` create
branches of create_or_update_filter_view, create_or_update_user_data_filter
and create_or_update_workspace_setting cannot run at all. Each serializes
through a generated model requiring a str id, and passing None fails type
validation before any request -- including the PostOptionalId variants, where
"optional" means "omit the key" rather than "accept None". That is left
unfixed here (separate concern), but is pinned by strict xfail tests and the
docstrings no longer claim those paths create anything.
str(UpsertOutcome.CREATED) returned "UpsertOutcome.CREATED"; with
__str__ = str.__str__ it returns "created", so swapping the base class
for StrEnum once py3.10 support is dropped is a no-op for callers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vondravl
vondravl force-pushed the feat/upsert-outcome branch from c618ee2 to b8993c4 Compare August 5, 2026 05:58
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@hkad98
hkad98 merged commit 2f1a82f into gooddata:master Aug 5, 2026
14 of 15 checks passed
@vondravl
vondravl deleted the feat/upsert-outcome branch August 5, 2026 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants