Add dolly v2 instruction tuning ray train

[yutsai84]

Why are these changes needed?

This pr creates a demo for instruction fine-tuning the dolly v2 3B model with deepspeed and Ray Train into a ray's doc repo.

This is a migration from the previous pr using Ray AIR

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[yutsai84]

@@xwjiang2010, this is the migration of previous notebook from Ray AIR to Ray Train. :)

[kevin85421]

cc @woshiyyya @justinvyu

Would you mind taking a look when you have time? Thanks!

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[aslonnie]

is this still relevant? should we close this?

[anyscalesam]

I know we reworked all of the docs for Train GA; is this still something that makes sense to be merged @matthewdeng ?