build: add pyproject.toml for pip/uv installable packaging#54
Open
zhangxin81 wants to merge 1 commit into
Open
build: add pyproject.toml for pip/uv installable packaging#54zhangxin81 wants to merge 1 commit into
zhangxin81 wants to merge 1 commit into
Conversation
DeepSpec shipped as a plain repo with only requirements.txt, so it could not be installed as a package (pip install ., or a git dependency from another project). Add a PEP 621 pyproject.toml with a setuptools backend so the deepspec library is importable after a standard install. - Packages the deepspec* library only; scripts/, config/, tests/, and eval_datasets/ stay as repo tooling and are not shipped. - Runtime dependencies mirror requirements.txt; data-preparation-only deps (datasets, openai) move to an optional [data] extra. - requires-python >=3.11 (matches the numpy==2.4.4 floor).
zhangxin81
added a commit
to zhangxin81/VeOmni
that referenced
this pull request
Jul 6, 2026
Install DeepSpec as a pinned git dependency instead of a sys.path-only bootstrap, and address reviewer comments on the draft trainer. - Add an opt-in `deepspec` extra: git source pinned by commit in [tool.uv.sources] (the commit adds DeepSpec's pyproject.toml, proposed upstream in deepseek-ai/DeepSpec#54; repoint to upstream once merged) and an empty [[tool.uv.dependency-metadata]] so DeepSpec's own torch/ transformers pins do not conflict with VeOmni's stack. DEEPSPEC_PATH / sibling checkout stays as a dev fallback. - draft_trainer: report the true global-token loss (mean all-reduce over the fsdp group inverts DeepSpec's world-size backward scaling) instead of DeepSpec's per-rank mean-of-means metric. - draft_trainer: freeze embed_tokens / lm_head by substring match so nested HF param names (model.embed_tokens.weight) are matched in the fallback path. - Update integration docs, uv.md, and integration docstrings to describe the deepspec extra as the primary install path.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add a PEP 621
pyproject.tomlwith a setuptools build backend so DeepSpec is installable as a package.Today DeepSpec ships as a plain repo with only
requirements.txt, so there is no build backend:pip install ., building a wheel, or depending on DeepSpec as a git dependency from another project all fail (nothing to build). This adds standard packaging while keeping the repo layout unchanged.Details
setuptools>=77.0.0/setuptools.build_meta.deepspec*only.scripts/,config/,tests/, andeval_datasets/remain repo tooling/data and are not shipped in the wheel.requirements.txtsopip install .andpip install -r requirements.txtstay in lockstep. Data-preparation-only deps (datasets,openai) move to an optional[data]extra since they are not needed for training/eval imports.requires-python = ">=3.11": matches thenumpy==2.4.4floor.Testing
Built a wheel locally with
uv build --wheel:deepspec/modeling/{dspark,eagle3}/**,deepspec/data,deepspec/utils/**, etc.).scripts/,config/,tests/,eval_datasets/are correctly excluded.METADATAreports the expected name/version,Requires-Python, and pinnedRequires-Dist.Why
Enables downstream projects to depend on DeepSpec via a standard install (e.g. a git dependency in another project's
pyproject.toml) instead of manualsys.path/PYTHONPATHwiring.