Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Guide For Multi-Node Distributed Finetuning #1477

Closed
wants to merge 6 commits into from
Closed

Guide For Multi-Node Distributed Finetuning #1477

wants to merge 6 commits into from

Commits on Apr 4, 2024

  1. Create Distributed_Finetuning_Multi-Node.md 

    Title: Distributed Finetuning Setup Guide

Description:
This PR introduces a comprehensive guide for setting up a distributed finetuning environment using Axolotl and Accelerate. The guide covers the following steps:

1. Configuring SSH for passwordless access across multiple nodes
2. Generating and exchanging public keys for secure communication
3. Configuring Axolotl with shared settings and host files
4. Configuring Accelerate for multi-node training with Deepspeed
5. Running distributed finetuning using Accelerate
    @shahdivax
    shahdivax committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    9c75d6c View commit details
    Browse the repository at this point in the history

  2. Merge pull request #1 from shahdivax/shahdivax-patch-1 

    Create Distributed_Finetuning_Multi-Node.md
    @shahdivax
    shahdivax committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    71798ad View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. Merge branch 'main' into main

    @shahdivax
    shahdivax committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    724ff48 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Update and rename Distributed_Finetuning_Multi-Node.md to Distributed… 

    …_Finetuning_Multi-Node.qmd
    @shahdivax
    shahdivax committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    3d73d36 View commit details
    Browse the repository at this point in the history

  2. Update Distributed_Finetuning_Multi-Node.qmd

    @shahdivax
    shahdivax committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    8d0fd87 View commit details
    Browse the repository at this point in the history

  3. Update Distributed_Finetuning_Multi-Node.qmd

    @shahdivax
    shahdivax committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    ac8c142 View commit details
    Browse the repository at this point in the history