NeMo Integrate

[cat-state]

Currently implemented:

• Training for ILQL
• Generation using ILQL (i.e validation)
• Inference script for ILQL

Future issues:

• More reusable nemo abstraction
• bf16 appears unstable compared to fp16

[jon-tow]

If _reconfigure for generate is uncommented, as currently is, num_workers=self.cfg.data.num_workers seems to work fine. Do you think we should change it back from 0?

[jon-tow]

I think this PR is in a really good state right now. Any updates on the remaining issues you've listed?

• Verify checkpointing
• ILQL loss spikes near the start of training (this doesn't seem to be the case now?)

[cat-state]

I think this PR is in a really good state right now. Any updates on the remaining issues you've listed?

* [ ]  Verify checkpointing

* [ ]  ILQL loss spikes near the start of training (this doesn't seem to be the case now?)

Yeah, I think the last one isn't a case anymore. I'll add a script showing how to load a checkpoint and infer it.

[cat-state]

I added the inference script and made the checkpointing save to a reloadable name (doesn't work with metrics with / in their name)

[jon-tow]

Leaving some very minor nits

[jon-tow]

Amazing work 🥳 🥳 🥳 Thanks @cat-state !!