Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 72 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
A minimal Python SDK to use Microsoft Dataverse as a database for Azure AI Foundry–style apps.

- Read (SQL) — Execute constrained read-only SQL via the Dataverse Web API `?sql=` parameter. Returns `list[dict]`.
- OData CRUD — Thin wrappers over Dataverse Web API (create/get/update/delete).
- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs. If `@odata.type` is absent the SDK resolves the logical name from metadata (cached).
- Bulk update — Call `update_multiple(entity_set, records)` to invoke the bound `UpdateMultiple` action; returns nothing. Each record must include the real primary key attribute (e.g. `accountid`).
- OData CRUD — Unified methods `create(entity, record|records)`, `update(entity, id|ids, patch|patches)`, `delete(entity, id|ids)` plus `get` / `get_multiple`.
- Bulk create — Pass a list of records to `create(...)` to invoke the bound `CreateMultiple` action; returns `list[str]` of GUIDs. If any payload omits `@odata.type` the SDK resolves and stamps it (cached).
- Bulk update — Provide a list of IDs with a single patch (broadcast) or a list of per‑record patches to `update(...)`; internally uses the bound `UpdateMultiple` action; returns nothing. Each record must include the primary key attribute when sent to UpdateMultiple.
- Retrieve multiple (paging) — Generator-based `get_multiple(...)` that yields pages, supports `$top` and Prefer: `odata.maxpagesize` (`page_size`).
- Metadata helpers — Create/inspect/delete simple custom tables (EntityDefinitions + Attributes).
- Pandas helpers — Convenience DataFrame oriented wrappers for quick prototyping/notebooks.
Expand All @@ -17,7 +17,7 @@ A minimal Python SDK to use Microsoft Dataverse as a database for Azure AI Found
- SQL-over-API: Constrained SQL (single SELECT with limited WHERE/TOP/ORDER BY) via native Web API `?sql=` parameter.
- Table metadata ops: create simple custom tables with primitive columns (string/int/decimal/float/datetime/bool) and delete them.
- Bulk create via `CreateMultiple` (collection-bound) by passing `list[dict]` to `create(entity_set, payloads)`; returns list of created IDs.
- Bulk update via `UpdateMultiple` by calling `update_multiple(entity_set, records)` with primary key attribute present in each record; returns nothing.
- Bulk update via `UpdateMultiple` (invoked internally) by calling unified `update(entity_set, ids, patch|patches)`; returns nothing.
- Retrieve multiple with server-driven paging: `get_multiple(...)` yields lists (pages) following `@odata.nextLink`. Control total via `$top` and per-page via `page_size` (Prefer: `odata.maxpagesize`).
- Optional pandas integration (`PandasODataClient`) for DataFrame based create / get / query.

Expand All @@ -26,6 +26,36 @@ Auth:
- You can pass any `azure.core.credentials.TokenCredential` you prefer; examples use `InteractiveBrowserCredential` for local runs.
- Token scope used by the SDK: `https://<yourorg>.crm.dynamics.com/.default` (derived from `base_url`).

## API Reference (Summary)

| Method | Signature (simplified) | Returns | Notes |
|--------|------------------------|---------|-------|
| `create` | `create(entity_set, record_dict)` | `list[str]` (len 1) | Single create; GUID from `OData-EntityId`. |
| `create` | `create(entity_set, list[record_dict])` | `list[str]` | Uses `CreateMultiple`; stamps `@odata.type` if missing. |
| `get` | `get(entity_set, id)` | `dict` | One record; supply GUID (with/without parentheses). |
| `get_multiple` | `get_multiple(entity_set, ..., page_size=None)` | `Iterable[list[dict]]` | Pages yielded (non-empty only). |
| `update` | `update(entity_set, id, patch)` | `None` | Single update; no representation returned. |
| `update` | `update(entity_set, list[id], patch)` | `None` | Broadcast; same patch applied to all IDs. Calls UpdateMultiple web API internally. |
| `update` | `update(entity_set, list[id], list[patch])` | `None` | 1:1 patches; lengths must match. Calls UpdateMultiple web API internally. |
| `delete` | `delete(entity_set, id)` | `None` | Delete one record. |
| `delete` | `delete(entity_set, list[id])` | `None` | Delete many (sequential). |
| `query_sql` | `query_sql(sql)` | `list[dict]` | Constrained read-only SELECT via `?sql=`. |
| `create_table` | `create_table(name, schema)` | `dict` | Creates custom table + columns. |
| `get_table_info` | `get_table_info(name)` | `dict | None` | Basic table metadata. |
| `list_tables` | `list_tables()` | `list[dict]` | Lists non-private tables. |
| `delete_table` | `delete_table(name)` | `None` | Drops custom table. |
| `PandasODataClient.create_df` | `create_df(entity_set, series)` | `str` | Returns GUID (wrapper). |
| `PandasODataClient.update` | `update(entity_set, id, series)` | `None` | Ignores empty Series. |
| `PandasODataClient.get_ids` | `get_ids(entity_set, ids, select=None)` | `DataFrame` | One row per ID (errors inline). |
| `PandasODataClient.query_sql_df` | `query_sql_df(sql)` | `DataFrame` | DataFrame for SQL results. |

Guidelines:
- `create` always returns a list of GUIDs (1 for single, N for bulk).
- `update`/`delete` always return `None` (single and multi forms).
- Bulk update chooses broadcast vs per-record by the type of `changes` (dict vs list).
- Paging and SQL operations never mutate inputs.
- Metadata lookups for logical name stamping cached per entity set (in-memory).

## Install

Create and activate a Python 3.13+ environment, then install dependencies:
Expand Down Expand Up @@ -69,7 +99,8 @@ The quickstart demonstrates:
- Creating a simple custom table (metadata APIs)
- Creating, reading, updating, and deleting records (OData)
- Bulk create (CreateMultiple) to insert many records in one call
- Retrieve multiple with paging (contrasting `$top` vs `page_size`)
- Bulk update via unified `update` (multi-ID broadcast & per‑record patches)
- Retrieve multiple with paging (`$top` vs `page_size`)
- Executing a read-only SQL query (Web API `?sql=`)

## Examples
Expand All @@ -92,22 +123,28 @@ from dataverse_sdk import DataverseClient
base_url = "https://yourorg.crm.dynamics.com"
client = DataverseClient(base_url=base_url, credential=DefaultAzureCredential())

# Create (returns created record)
created = client.create("accounts", {"name": "Acme, Inc.", "telephone1": "555-0100"})
account_id = created["accountid"]
# Create (returns list[str] of new GUIDs)
account_id = client.create("accounts", {"name": "Acme, Inc.", "telephone1": "555-0100"})[0]

# Read
account = client.get("accounts", account_id)

# Update (returns updated record)
updated = client.update("accounts", account_id, {"telephone1": "555-0199"})
# Update (returns None)
client.update("accounts", account_id, {"telephone1": "555-0199"})

# Bulk update (collection-bound UpdateMultiple)
# Each record must include the primary key attribute (accountid). The call returns None.
client.update_multiple("accounts", [
{"accountid": account_id, "telephone1": "555-0200"},
# Bulk update (broadcast) – apply same patch to several IDs
ids = client.create("accounts", [
{"name": "Contoso"},
{"name": "Fabrikam"},
])
print({"bulk_update": "ok"})
client.update("accounts", ids, {"telephone1": "555-0200"}) # broadcast patch

# Bulk update (1:1) – list of patches matches list of IDs
client.update("accounts", ids, [
{"telephone1": "555-1200"},
{"telephone1": "555-1300"},
])
print({"multi_update": "ok"})

# Delete
client.delete("accounts", account_id)
Expand All @@ -119,7 +156,7 @@ for r in rows:

## Bulk create (CreateMultiple)

Pass a list of payloads to `create(entity_set, payloads)` to invoke the collection-bound `Microsoft.Dynamics.CRM.CreateMultiple` action. The method returns a `list[str]` of created record IDs.
Pass a list of payloads to `create(entity_set, payloads)` to invoke the collection-bound `Microsoft.Dynamics.CRM.CreateMultiple` action. The method returns `list[str]` of created record IDs.

```python
# Bulk create accounts (returns list of GUIDs)
Expand All @@ -133,34 +170,31 @@ assert isinstance(ids, list) and all(isinstance(x, str) for x in ids)
print({"created_ids": ids})
```

## Bulk update (UpdateMultiple)
## Bulk update (UpdateMultiple under the hood)

Use `update_multiple(entity_set, records)` for a transactional batch update. The method returns `None`.
Use the unified `update` method for both single and bulk scenarios:

```python
ids = client.create("accounts", [
{"name": "Fourth Coffee"},
{"name": "Tailspin"},
])
# Broadcast
client.update("accounts", ids, {"telephone1": "555-0200"})

client.update_multiple("accounts", [
{"accountid": ids[0], "telephone1": "555-1111"},
{"accountid": ids[1], "telephone1": "555-2222"},
# 1:1 patches (length must match)
client.update("accounts", ids, [
{"telephone1": "555-1200"},
{"telephone1": "555-1300"},
])
print({"bulk_update": "ok"})
```

Notes:
- Each record must include the primary key attribute (e.g. `accountid`). No `id` alias yet.
- If any payload omits `@odata.type`, the logical name is resolved once and stamped (same as bulk create).
- Entire request fails (HTTP error) if any individual update fails; no partial success list is returned.
- If you need refreshed records post-update, issue individual `get` calls or a `get_multiple` query.
- Returns `None` (same as single update) to keep semantics consistent.
- Broadcast vs per-record determined by whether `changes` is a dict or list.
- Primary key attribute is injected automatically when constructing UpdateMultiple targets.
- If any payload omits `@odata.type`, it's stamped automatically (cached logical name lookup).

Notes:
- The bulk create response typically includes IDs only; the SDK returns the list of GUID strings.
- Single-record `create` still returns the full entity representation.
- `@odata.type` handling: If any payload in the list omits `@odata.type`, the SDK performs a one-time metadata query (`EntityDefinitions?$filter=EntitySetName eq '<entity_set>'`) to resolve the logical name, caches it, and stamps each missing item with `Microsoft.Dynamics.CRM.<logical>`. If **all** payloads already include `@odata.type`, no metadata call is made.
- The metadata lookup is per entity set and reused across subsequent multi-create calls in the same client instance (in-memory cache only).
Bulk create notes:
- Response includes only IDs; the SDK returns those GUID strings.
- Single-record `create` returns a one-element list of GUIDs.
- Metadata lookup for `@odata.type` is performed once per entity set (cached in-memory).


## Retrieve multiple with paging
Expand Down Expand Up @@ -267,7 +301,8 @@ client.delete_table("SampleItem") # delete the table
```

Notes:
- `create/update` return the full record using `Prefer: return=representation`.
- `create` always returns a list of GUIDs (length 1 for single input).
- `update` and `delete` return `None` for both single and multi.
- Passing a list of payloads to `create` triggers bulk create and returns `list[str]` of IDs.
- Use `get_multiple` for paging through result sets; prefer `select` to limit columns.
- For CRUD methods that take a record id, pass the GUID string (36-char hyphenated). Parentheses around the GUID are accepted but not required.
Expand All @@ -285,7 +320,7 @@ VS Code Tasks
- No general-purpose OData batching, upsert, or association operations yet.
- `DeleteMultiple` not yet exposed.
- Minimal retry policy in library (network-error only); examples include additional backoff for transient Dataverse consistency.
- Entity naming conventions in Dataverse: for multi-create the SDK resolves logical names from entity set metadata.
- Entity naming conventions in Dataverse: for bulk create the SDK resolves logical names from entity set metadata.

## Contributing

Expand Down
44 changes: 18 additions & 26 deletions examples/quickstart.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,33 +176,25 @@ def print_line_summaries(label: str, summaries: list[dict]) -> None:
)

record_ids: list[str] = []
created_recs: list[dict] = []

try:
# Single create (always returns full representation)
# Single create returns list[str] (length 1)
log_call(f"client.create('{entity_set}', single_payload)")
# Retry in case the custom table isn't fully provisioned immediately (404)
rec1 = backoff_retry(lambda: client.create(entity_set, single_payload))
created_recs.append(rec1)
rid1 = rec1.get(id_key)
if rid1:
record_ids.append(rid1)

# Multi create (list) now returns list[str] of IDs
single_ids = backoff_retry(lambda: client.create(entity_set, single_payload))
if not (isinstance(single_ids, list) and len(single_ids) == 1):
raise RuntimeError("Unexpected single create return shape (expected one-element list)")
record_ids.extend(single_ids)

# Multi create returns list[str]
log_call(f"client.create('{entity_set}', multi_payloads)")
multi_ids = backoff_retry(lambda: client.create(entity_set, multi_payloads))
if isinstance(multi_ids, list):
for mid in multi_ids:
if isinstance(mid, str):
record_ids.append(mid)
record_ids.extend([mid for mid in multi_ids if isinstance(mid, str)])
else:
print({"multi_unexpected_type": type(multi_ids).__name__, "value_preview": str(multi_ids)[:300]})

print({"entity": logical, "created_ids": record_ids})
summaries = []
for rec in created_recs:
summaries.append({"id": rec.get(id_key), **summary_from_record(rec)})
print_line_summaries("Created record summaries (single only; multi-create returns IDs only):", summaries)
print_line_summaries("Created record summaries (IDs only; representation not fetched):", [{"id": rid} for rid in record_ids[:1]])
except Exception as e:
# Surface detailed info for debugging (especially multi-create failures)
print(f"Create failed: {e}")
Expand Down Expand Up @@ -273,16 +265,16 @@ def print_line_summaries(label: str, summaries: list[dict]) -> None:

# Update only the chosen record and summarize
log_call(f"client.update('{entity_set}', '{target_id}', update_data)")
new_rec = backoff_retry(lambda: client.update(entity_set, target_id, update_data))
# Verify string/int/bool fields
# Perform update (returns None); follow-up read to verify
backoff_retry(lambda: client.update(entity_set, target_id, update_data))
verify_rec = backoff_retry(lambda: client.get(entity_set, target_id))
for k, v in expected_checks.items():
assert new_rec.get(k) == v, f"Field {k} expected {v}, got {new_rec.get(k)}"
# Verify decimal with tolerance
got = new_rec.get(amount_key)
assert verify_rec.get(k) == v, f"Field {k} expected {v}, got {verify_rec.get(k)}"
got = verify_rec.get(amount_key)
got_f = float(got) if got is not None else None
assert got_f is not None and abs(got_f - 543.21) < 1e-6, f"Field {amount_key} expected 543.21, got {got}"
print({"entity": logical, "updated": True})
print_line_summaries("Updated record summary:", [{"id": target_id, **summary_from_record(new_rec)}])
print_line_summaries("Updated record summary:", [{"id": target_id, **summary_from_record(verify_rec)}])
except Exception as e:
print(f"Update/verify failed: {e}")
sys.exit(1)
Expand All @@ -300,9 +292,9 @@ def print_line_summaries(label: str, summaries: list[dict]) -> None:
id_key: rid,
count_key: 100 + idx, # new count values
})
log_call(f"client.update_multiple('{entity_set}', <{len(bulk_updates)} records>)")
# update_multiple returns nothing (fire-and-forget success semantics)
backoff_retry(lambda: client.update_multiple(entity_set, bulk_updates))
log_call(f"client.update('{entity_set}', <{len(bulk_updates)} ids>, <patches>)")
# Unified update handles multiple via list of patches (returns None)
backoff_retry(lambda: client.update(entity_set, subset, bulk_updates))
print({"bulk_update_requested": len(bulk_updates), "bulk_update_completed": True})
# Verify the updated count values by refetching the subset
verification = []
Expand Down
Loading