fix(hy_worldplay): load base Wan checkpoint into the HY network (no --ckpt-path path)#227
Open
wenqingw-nv wants to merge 1 commit into
Open
Conversation
Running the HY pipeline without `--ckpt-path` (README's "conditioners stay zero-init, parity-safe vs base Wan" path) raised `RuntimeError: Missing key(s)` — the base Wan checkpoint has no `action_embedding` / `o_prope` params and `Wan21Transformer.__init__` does a strict `load_state_dict`. (Latent because CI/GPU runs always pass `--ckpt-path`; the smoke tests only assert the static config, never a real base load.) Override `HyWorldPlayWanDiTNetwork.load_state_dict` to tolerate *only* those HY-specific zero-init keys when absent (they keep their zero-init identity); any other missing key, or any unexpected key, still raises. HY's distilled checkpoint carries the keys, so that path is unaffected. Verified end-to-end: `flashdreams-run hy-worldplay-wan-i2v-5b --example-data True --num-chunk 1 --pose w-3` (no `--ckpt-path`) now produces a valid mp4. Adds CPU tests for the tolerate / still-reject behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 tasks
Collaborator
Author
|
/ok to test 0270385 |
Collaborator
|
Why would it load Wan2.2 base checkpoint? We only consider inference so it should directly load the HY ckpt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spun off from #203 (surfaced while GPU-verifying the HY-WorldPlay follow-ups).
Bug: running the HY pipeline without
--ckpt-path— the path the README documents as "loads base Wan 2.2, HY conditioners stay zero-init, strict identity / parity-safe vs base Wan" — actually raises:The base Wan checkpoint has no
action_embedding/o_propeparams, andWan21Transformer.__init__does a strictload_state_dict. Latent because CI/GPU runs always pass--ckpt-path(distilled), and the smoke tests only assert the static config — never a real base load.Fix: override
HyWorldPlayWanDiTNetwork.load_state_dictto tolerate only the HY-specific zero-init keys (action_embedding.*,*.o_prope.*) when absent — they keep their zero-init values (the intended identity). Any other missing key, or any unexpected key, still raises (not a blanketstrict=False). The distilled checkpoint carries these keys, so that path is unchanged.Verified:
flashdreams-run hy-worldplay-wan-i2v-5b --example-data True --num-chunk 1 --pose w-3(no--ckpt-path) → valid 13-frame 704×1280 mp4.Part of #203.