Introducing auto_install.sh

[monk1337]

Objective:

This PR introduces an automated setup script (auto_install.sh) designed to streamline the installation process of Axolotl. It addresses common challenges faced by users not utilizing Docker, particularly in configuring the correct CUDA, PyTorch versions, and related dependencies. This enhancement is motivated by community feedback highlighting difficulties in manual setup processes.

Features and Implementation:

• Conda Environment Setup: Automatically checks for an existing Conda installation and proceeds to install Conda if it's not found, ensuring a controlled setup environment.
• CUDA Version Management: Installs a specific CUDA version within the Conda environment tailored for compatibility, addressing common version mismatch issues.
• PyTorch Ecosystem Compatibility: Ensures the installation of PyTorch, TorchAudio, and TorchVision that are compatible with the selected CUDA version, mitigating dependency conflicts.
• Axolotl Dependencies: Automatically installs all required dependencies for Axolotl, facilitating a smoother setup experience.

Planned Enhancements:

• OS-Specific Installation: Currently it supports Linux only however, future iterations will aim to automatically identify the operating system and adjust the CUDA installation and other dependencies accordingly.
• Docker Support Argument: Incorporate an option to install Docker if required by the user, enhancing flexibility for users preferring containerized environments.
• Installation Validation: Introduce a simple test script capable of running a small model on a dataset of 500 samples to verify the correctness of the installation and operational integrity of Axolotl post-setup.

Note: This script presents an additional option for users seeking an easier installation route without mandating its use. Users retain the freedom to choose their preferred installation method. The primary goal of this script is to reduce setup time and complexity, offering an alternative to manual configuration.

[maximegmd]

I am a bit opposed to the auto install being at the root, while it covers the "common" use case, there is a growing variety of setups (ROCm, MPS, XPU...)

I also think that there is no need for such a script as the docker images cover the difficult part of setting it up. So I am not sure who this helps, power users already have a conda install and most likely the correct pytorch version. End users should just pull the docker image.

[monk1337]

I am a bit opposed to the auto install being at the root, while it covers the "common" use case, there is a growing variety of setups (ROCm, MPS, XPU...)

I also think that there is no need for such a script as the docker images cover the difficult part of setting it up. So I am not sure who this helps, power users already have a conda install and most likely the correct pytorch version. End users should just pull the docker image.

You make a fair case that the Docker images likely cover much of the complexity for many users. However, as you rightly mentioned, not all users currently utilize Docker. I recently came across several incidents on Discord and elsewhere where users faced issues installing Axolotl, for example this popular Reddit post Reddit post, where a user faced frustration while installing Axolotl. This inspired me to consider simplifying the initial setup process for those who could benefit.

It can also support different platforms. I just created a very initial version; there is more to go. Moreover, it would just serve as an additional option for users - they could follow other methods if they prefer.

[mhenrichsen]

Gemma config change and auto setup should be two different PR, so they don't go in together.

Also, please add some documentation that this autoconfig is only for linux.

[monk1337]

Gemma config change and auto setup should be two different PR, so they don't go in together.

Also, please add some documentation that this autoconfig is only for linux.

Done, Thank you!

[winglian]

Thanks @monk1337, I do find value in having this script as having docker adds another layer of complexity over a native install. I do worry that having a script adds another point of maintenance/testing/support. Currently our docker images are well tested as we typically run tests against the built images themselves.

I think a good compromise would be to indicate that the install script itself is "beta" and have a date somewhere where it was last updated/tested, just for visibility to end users.

[NanoCode012]

If this is the start of a one-stop installer (like how text-webui supports several options), this may be a good starting point. Otherwise, I believe it would be better to improve the current pip/conda installation instructions.

Edit: I also agree with suggestion above where this has to be stated for Nvidia-only.

[NanoCode012]

Some systems do not support 12.1, so this may be an issue? One alternative could be like ooba where they allow option to use 11.8?

[NanoCode012]

Suggested change

	pip3 uninstall deepspeed -y
	pip3 install deepspeed==0.13.1

This will not be necessary anymore. There's a separate PR which addresses this.

[NanoCode012]

This section may be better integrated with the pip/conda section?