Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.DEFAULT_GOAL := help

.PHONY: help install lint typecheck test test-all test-remote clean build publish
.PHONY: help install lint typecheck test test-cli test-all test-remote clean build publish

help: ## Show this help
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-15s\033[0m %s\n", $$1, $$2}'
Expand All @@ -22,6 +22,9 @@ typecheck: ## Run mypy type checking
test: ## Run unit tests only (no remote API calls)
pytest tests/ -k "not remote" --ignore=tests/integration

test-cli: ## Run CLI unit tests only (no remote API calls)
python tests/cli/run_all.py

test-all: ## Run all tests including integration (needs tests/.env)
pytest tests/

Expand Down
54 changes: 29 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ api.upload_folder("my-org/my-model", "model", "./output", path_in_repo="")
### Create a Repository

```bash
ms repo create my-org/my-model --repo-type model --visibility private
ms create my-org/my-model --repo-type model --visibility private
```

```python
Expand All @@ -116,13 +116,13 @@ api.create_repo("my-org/my-model", "model", visibility="private", license="apach

```bash
ms deploy my-org/chat-demo --repo-type studio
ms logs my-org/chat-demo --log-type runtime
ms logs my-org/chat-demo --log-type run
ms stop my-org/chat-demo --repo-type studio
```

```python
api.deploy_repo("my-org/chat-demo", "studio")
api.get_repo_logs("my-org/chat-demo", log_type="runtime")
api.get_repo_logs("my-org/chat-demo", log_type="run")
api.stop_repo("my-org/chat-demo", "studio")
```

Expand All @@ -132,14 +132,17 @@ api.stop_repo("my-org/chat-demo", "studio")

The CLI is available as both `ms` and `modelscope`.

**Global options** (placed before the subcommand):
**Global options** (placed before or after the subcommand):

| Option | Description |
|--------|-------------|
| `--token TOKEN` | API token (overrides env and persisted token) |
| `--endpoint URL` | API endpoint (default: `https://modelscope.cn`) |
| `-v, --verbose` | Enable DEBUG logging |
| `-V, --version` | Print version and exit |
| `-v, --verbose` | Enable DEBUG logging (global only) |
| `-V, --version` | Print version and exit (global only) |

> `--token` and `--endpoint` can be placed either before or after the subcommand:
> `ms --token xxx download ...` and `ms download ... --token xxx` are equivalent.

### `ms login`

Expand All @@ -160,6 +163,7 @@ Show the user associated with the current token.

```bash
ms whoami
ms whoami --token $MY_TOKEN # check a specific token without logging in
```

### `ms download`
Expand Down Expand Up @@ -206,9 +210,6 @@ ms download Qwen/Qwen3-0.6B --cache-dir /data/hub-cache --max-workers 8
# Force re-download even if already cached
ms download Qwen/Qwen3-0.6B config.json --force

# Download a Studio space
ms download my-org/chat-demo --repo-type studio

# Download all skills from a collection (legacy flag)
ms download --collection my-org/skill-collection

Expand Down Expand Up @@ -283,25 +284,25 @@ ms upload my-org/my-model ./output --disable-tqdm

</details>

### `ms repo`
### `ms create` / `ms info` / `ms list` / `ms delete`

Repository management (create, info, list, delete).
Repository management.

```bash
ms repo create my-org/my-model --repo-type model --visibility private
ms repo create my-org/demo --repo-type studio --sdk-type gradio
ms repo info my-org/my-model --repo-type model
ms repo list --repo-type model --owner my-org --page-size 20
ms repo delete my-org/my-model --repo-type model --yes
ms create my-org/my-model --repo-type model --visibility private
ms create my-org/demo --repo-type studio --sdk-type gradio
ms info my-org/my-model --repo-type model
ms list --repo-type model --owner my-org --page-size 20
ms delete my-org/my-model --repo-type model --yes
```

<details>
<summary><code>ms repo create</code> options</summary>
<summary><code>ms create</code> options</summary>

| Argument / Option | Required | Description |
|-------------------|----------|-------------|
| `repo_id` | yes | Repository identifier |
| `--repo-type` | yes | `model`, `dataset`, `studio`, `skill`, or `mcp` |
| `--repo-type` | yes | `model`, `dataset`, `studio`, or `skill` |
| `--visibility` | no | `public`, `private`, or `internal` |
| `--license` | no | SPDX license identifier (e.g. `apache-2.0`) |
| `--chinese-name` | no | Display name in Chinese |
Expand All @@ -321,20 +322,23 @@ Manage Studio and MCP deployments.

```bash
ms deploy my-org/chat-demo --repo-type studio
ms logs my-org/chat-demo --log-type runtime --keyword ERROR --page-size 50
ms logs my-org/chat-demo --log-type run --keyword ERROR --page-size 50
ms settings my-org/chat-demo cpu=4 memory=8192
ms stop my-org/chat-demo --repo-type studio
```

<details>
<summary>Options</summary>

| Command | Key Options |
|---------|-------------|
| `ms deploy <repo_id>` | `--repo-type {studio,mcp}` |
| `ms stop <repo_id>` | `--repo-type {studio,mcp}` |
| `ms logs <repo_id>` | `--log-type {runtime,build}`, `--keyword`, `--page`, `--page-size` |
| `ms settings <repo_id> key=val...` | Key-value pairs passed to backend |
| Command | `--repo-type` | Key Options |
|---------|---------------|-------------|
| `ms deploy <repo_id>` | `{studio,mcp}` (default: `studio`) | — |
| `ms stop <repo_id>` | `{studio,mcp}` (default: `studio`) | — |
| `ms logs <repo_id>` | `{studio}` only | `--log-type {run,build}`, `--keyword`, `--page`, `--page-size` |
| `ms settings <repo_id> key=val...` | `{studio,skill}` (default: `studio`) | Key-value pairs passed to backend |

> **Note:** `ms logs` only supports Studio spaces. MCP server logs are not available via this command.
> `ms settings` supports Studio and Skill repos; for MCP servers use `ms mcp deploy` with configuration payload.

</details>

Expand Down
35 changes: 28 additions & 7 deletions src/modelscope_hub/_legacy_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ def _request(
params: dict[str, Any] | None = None,
json_body: Any | None = None,
data: Any | None = None,
files: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
timeout: int | None = None,
stream: bool = False,
Expand All @@ -157,8 +158,9 @@ def _request(
self._ensure_session_auth()
url = self._build_url(path)
merged_headers = self._headers(headers)
# Remove Content-Type for non-json payloads
if data is not None and json_body is None:
# Remove Content-Type for non-json payloads so requests can set
# the correct boundary for multipart or the right encoding for data.
if (data is not None or files is not None) and json_body is None:
merged_headers.pop("Content-Type", None)

logger.debug("%s %s", method, url)
Expand All @@ -169,6 +171,7 @@ def _request(
params=params,
json=json_body,
data=data,
files=files,
headers=merged_headers,
timeout=timeout or self._timeout,
stream=stream,
Expand All @@ -179,6 +182,15 @@ def _request(
raise RequestTimeoutError(f"Request timed out: {exc}") from exc

logger.debug("%s %s -> %s", method, url, resp.status_code)
if resp.status_code >= 400:
logger.debug(
"Request failed: %s %s params=%s status=%s body=%s",
method,
url,
params,
resp.status_code,
resp.text[:500] if resp.text else "",
)
raise_for_status(resp)
return resp

Expand Down Expand Up @@ -215,9 +227,16 @@ def create_repo(self, repo_type: str, body: dict[str, Any]) -> dict:
"""POST /api/v1/{type}s — create a new repository.

``body`` is the fully-constructed request payload (PascalCase keys).

The dataset endpoint requires multipart form data (matching the
upstream server implementation), while model uses JSON.
"""
segment = _resolve_segment(repo_type)
resp = self._request("POST", segment, json_body=body)
if repo_type in (RepoType.DATASET, "dataset"):
files = {k: (None, str(v)) for k, v in body.items() if v is not None}
resp = self._request("POST", segment, files=files)
else:
resp = self._request("POST", segment, json_body=body)
return self._json_data(resp)

def delete_repo(self, repo_id: str, repo_type: str) -> None:
Expand All @@ -242,7 +261,8 @@ def list_repo_files(
) -> list[dict]:
"""List files in a repository.

GET /api/v1/{type}s/{repo_id}/repo/files?Revision=&Recursive=&Root=
Models/studios/etc: GET /api/v1/{type}s/{repo_id}/repo/files
Datasets: GET /api/v1/datasets/{repo_id}/repo/tree
"""
segment = _resolve_segment(repo_type)
params: dict[str, Any] = {
Expand All @@ -252,7 +272,8 @@ def list_repo_files(
if root:
params["Root"] = root

resp = self._request("GET", f"{segment}/{repo_id}/repo/files", params=params)
suffix = "repo/tree" if repo_type in (RepoType.DATASET, "dataset", "datasets") else "repo/files"
resp = self._request("GET", f"{segment}/{repo_id}/{suffix}", params=params)
data = self._json_data(resp)
if isinstance(data, list):
return data
Expand All @@ -271,7 +292,7 @@ def list_dataset_files_paginated(
"""List files in a dataset repo using pagination.

Datasets can have millions of files, so this method pages through
``GET /api/v1/datasets/{repo_id}/repo/files`` with
``GET /api/v1/datasets/{repo_id}/repo/tree`` with
``PageNumber``/``PageSize`` params.
"""
all_files: list[dict] = []
Expand All @@ -287,7 +308,7 @@ def list_dataset_files_paginated(
params["Root"] = root_path
resp = self._request(
"GET",
f"datasets/{repo_id}/repo/files",
f"datasets/{repo_id}/repo/tree",
params=params,
)
data = self._json_data(resp)
Expand Down
9 changes: 9 additions & 0 deletions src/modelscope_hub/_openapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,15 @@ def _request(
last_exc = NetworkError(f"Request failed: {exc}")
else:
_logger.debug("%s %s -> %s", method_upper, url, response.status_code)
if response.status_code >= 400:
_logger.debug(
"Request failed: %s %s params=%s status=%s body=%s",
method_upper,
url,
params,
response.status_code,
response.text[:500] if response.text else "",
)
try:
raise_for_status(response)
except _RETRYABLE_EXC as exc: # type: ignore[misc]
Expand Down
66 changes: 60 additions & 6 deletions src/modelscope_hub/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@

# Routing tables — declarative dispatch keeps :class:`HubApi` free of long
# if/elif chains and makes adding new repo types a one-line change.
_CREATABLE_TYPES: frozenset[RepoType] = frozenset(
{RepoType.MODEL, RepoType.DATASET, RepoType.STUDIO, RepoType.SKILL}
)
_OPENAPI_CREATE_TYPES: frozenset[RepoType] = frozenset({RepoType.STUDIO, RepoType.SKILL})
_OPENAPI_DETAIL_TYPES: frozenset[RepoType] = frozenset(
{RepoType.MODEL, RepoType.DATASET, RepoType.STUDIO, RepoType.SKILL}
Expand All @@ -82,6 +85,11 @@
}


_STUDIO_FIELD_RENAMES: dict[str, str] = {
"cover_image": "coverImage",
}


class HubApi:
"""Unified client for ModelScope Hub operations.

Expand Down Expand Up @@ -284,6 +292,20 @@ def _repo_info_from_payload(
for key, value in data.items():
normalised[aliases.get(key, key)] = value

# The OpenAPI surface encodes visibility as a ``private`` bool (with an
# optional ``gated`` flag) instead of the legacy ``Visibility`` integer.
# Translate it so downstream code sees a uniform ``Visibility`` enum.
if normalised.get("visibility") is None:
private_flag = normalised.pop("private", None)
gated_flag = normalised.pop("gated", None)
if isinstance(private_flag, bool):
if private_flag:
normalised["visibility"] = Visibility.PRIVATE
elif gated_flag:
normalised["visibility"] = Visibility.INTERNAL
else:
normalised["visibility"] = Visibility.PUBLIC

normalised.setdefault("owner", owner_hint)
normalised.setdefault("name", name_hint)
normalised["repo_type"] = repo_type
Expand Down Expand Up @@ -506,6 +528,12 @@ def create_repo(
>>> api.create_repo("alice/chat-demo", repo_type="studio", visibility="public")
"""
rt = self._normalize_repo_type(repo_type)
if rt not in _CREATABLE_TYPES:
supported = ", ".join(sorted(t.value for t in _CREATABLE_TYPES))
raise NotSupportedError(
f"create_repo does not support repo_type={rt.value!r}. "
f"Supported types: {supported}."
)
owner, name = self._parse_repo_id(repo_id)
vis = self._normalize_visibility(visibility)
if license is not None:
Expand Down Expand Up @@ -535,6 +563,9 @@ def create_repo(
payload["license"] = license
if description is not None:
payload["description"] = description
for old_key, new_key in _STUDIO_FIELD_RENAMES.items():
if old_key in extra:
extra[new_key] = extra.pop(old_key)
payload.update(extra)
data = (
self.openapi.create_studio(payload)
Expand All @@ -545,12 +576,20 @@ def create_repo(
data, rt, owner_hint=owner, name_hint=name
)

body: dict[str, Any] = {
"Path": owner,
"Name": name,
"Visibility": vis if vis is not None else int(Visibility.PUBLIC),
"License": license or "Apache-2.0",
}
if rt is RepoType.DATASET:
body: dict[str, Any] = {
"Owner": owner,
"Name": name,
"Visibility": vis if vis is not None else int(Visibility.PUBLIC),
"License": license or "Apache-2.0",
}
else:
body = {
"Path": owner,
"Name": name,
"Visibility": vis if vis is not None else int(Visibility.PUBLIC),
"License": license or "Apache-2.0",
}
if chinese_name is not None:
body["ChineseName"] = chinese_name
if description is not None:
Expand Down Expand Up @@ -1054,6 +1093,12 @@ def download_file(
'{\n "architectures": [\n "Ll'
"""
rt = self._normalize_repo_type(repo_type)
if rt is RepoType.STUDIO:
raise NotSupportedError(
"File download is not supported for studio repositories. "
"Studios are application containers without a file listing API. "
f"To access studio source code, use: git clone https://modelscope.cn/studios/{repo_id}.git"
)
return self.downloader.download_file(
repo_id=repo_id,
repo_type=str(rt),
Expand Down Expand Up @@ -1128,6 +1173,12 @@ def download_repo(
['config.json', 'tokenizer.json', 'tokenizer_config.json']
"""
rt = self._normalize_repo_type(repo_type)
if rt is RepoType.STUDIO:
raise NotSupportedError(
"File download is not supported for studio repositories. "
"Studios are application containers without a file listing API. "
f"To access studio source code, use: git clone https://modelscope.cn/studios/{repo_id}.git"
)
return self.downloader.download_repo(
repo_id=repo_id,
repo_type=str(rt),
Expand Down Expand Up @@ -1463,6 +1514,9 @@ def update_repo_settings(
"""
rt = self._normalize_repo_type(repo_type)
owner, name = self._parse_repo_id(repo_id)
for old_key, new_key in _STUDIO_FIELD_RENAMES.items():
if old_key in settings:
settings[new_key] = settings.pop(old_key)
if rt is RepoType.STUDIO:
return self.openapi.update_studio_settings(owner, name, settings)
if rt is RepoType.SKILL:
Expand Down
Loading