Document rust compiler as a requirement

[ppf2]

OS: MacOS Monterey 12.6

In order for the python -m pip install eland script to install the packages successfully, the rust compiler is required. Let's update our documentation to include this pre-requisite.

[davidkyle]

I have not been able to recreate this on my machine but watching a this video about setting the presenter describes the same problem. The cause is that the fast tokenisers in Hugging Face transformers are written in Rust. The suggested work around is to install Rust.

The relevant section is around 7mins45s.

https://youtu.be/uYas6ysyjgY?t=471

Installing transformers on my mac (macOS Monterey 12.6.1 M1 max) does not require Rust (which is not installed)

This succeeds

/usr/bin/python3 -m pip install transformers

As does using a Miniconda virtual env

~/miniconda3/bin/python -m venv .env
source .env/bin/activate
pip install transformers

[davidkyle]

The requirement was documented in #507 and recent versions of Transformers tokenizers work with M1 macs. If using a version of Eland where the Transformers requirement is pinned to a version > 4.10 a Rust compiler is not required.