Running on windows?

[pmapumba]

How do I run this project on Windows?

[svlandeg]

Hi @pmapumba! You'd probably be best off spinning up a virtual Linux box with a proper GPU on your favorite cloud provider, as detailed in the readme. That said, I believe you can get it running on a Windows machine on CPU (cf comment here and readme here) but it's probably not ideal.

[zakki]

With the following changes, scripts.base_train is now working on Windows 11 with an RTX 4090.

• Manually install uv and rust
• Remove dependency on fcntl
• uv pip install "triton-windows<3.5"

Since base_train isn't finished yet, I don't know if it works as a whole.

https://github.com/zakki/nanochat/tree/windows

Overriding: depth = 20
Overriding: device_batch_size = 4
Overriding: run = dummy
Autodetected device type: cuda
2025-11-02 00:43:12,843 - nanochat.common - INFO - Distributed world size: 1
D:\share\nanochat\.venv\lib\site-packages\torch\_utils.py:101: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:35.)
  untyped_storage = torch.UntypedStorage(self.size(), device=device)
Vocab size: 65,536
num_layers: 20
model_dim: 1280
num_heads: 10
num_kv_heads: 10
Tokens / micro-batch / rank: 4 x 2048 = 8,192
Tokens / micro-batch: 8,192
Total batch size 524,288 => gradient accumulation steps: 64
Number of parameters: 560,988,160
Estimated FLOPs per token: 3.491758e+09
Calculated number of iterations from target data:param ratio: 21,400
Total number of training tokens: 11,219,763,200
Tokens : Params ratio: 20.00
Total training FLOPs estimate: 3.917670e+19
Scaling the LR for the AdamW parameters ∝1/√(1280/768) = 0.774597
Step 00000 | Validation bpb: 3.3017
step 00000/21400 (0.00%) | loss: 11.090355 | lrm: 1.00 | dt: 45387.65ms | tok/sec: 11,551 | mfu: 4.08 | total time: 0.00m
step 00001/21400 (0.00%) | loss: 10.807708 | lrm: 1.00 | dt: 20939.16ms | tok/sec: 25,038 | mfu: 8.84 | total time: 0.00m
step 00002/21400 (0.01%) | loss: 10.227508 | lrm: 1.00 | dt: 23994.30ms | tok/sec: 21,850 | mfu: 7.71 | total time: 0.00m
step 00003/21400 (0.01%) | loss: 9.486226 | lrm: 1.00 | dt: 24323.24ms | tok/sec: 21,555 | mfu: 7.61 | total time: 0.00m
step 00004/21400 (0.02%) | loss: 8.915292 | lrm: 1.00 | dt: 24294.44ms | tok/sec: 21,580 | mfu: 7.62 | total time: 0.00m

[zakki]

Also I need to set PYTHONUTF8=1, because eval_data jsonl files contain non-ascii characters.

[pmapumba]

Thank you. I also managed to get it work on Windows 11, with a few changes. I used portalocker instead of fcntl and manually installed uv and rust.

[svlandeg]

Also I need to set PYTHONUTF8=1, because eval_data jsonl files contain non-ascii characters.

@zakki: I hope #243 has fixed that particular issue!

[svlandeg]

Remove dependency on fcntl

This should now be fixed by #247