[Unified TorchTrainer] Add HF Transformers TorchTrainer Utilities

doc/source/_toc.yml

@@ -65,10 +65,12 @@ parts:
             title: PyTorch
           - file: train/getting-started-pytorch-lightning
             title: PyTorch Lightning
+          - file: train/getting-started-transformers
+            title: Hugging Face Transformers
           - file: train/more-frameworks
             sections:
               - file: train/huggingface-transformers-accelerate
-                title: HuggingFace Transformers & Accelerate
+                title: Hugging Face Transformers & Accelerate
               - file: train/distributed-tensorflow-keras
                 title: TensorFlow & Keras
               - file: train/distributed-xgboost-lightgbm
@@ -86,6 +88,8 @@ parts:
             sections:
               - file: train/examples/pytorch/torch_fashion_mnist_example
                 title: "PyTorch Fashion MNIST Example"
+              - file: train/examples/transformers/transformers_torch_trainer_basic
+                title: "Hugging Face Transformers Basic Example"
               - file: train/examples/lightning/lightning_mnist_example
                 title: "PyTorch Lightning Basic Example"
               - file: train/examples/lightning/lightning_cola_advanced

doc/source/train/api/api.rst

@@ -19,7 +19,7 @@ Ray Train Integrations
 PyTorch Ecosystem
 ~~~~~~~~~~~~~~~~~
 
-Scale out your PyTorch, Lightning, HuggingFace code with Ray TorchTrainer.
+Scale out your PyTorch, Lightning, Hugging Face code with Ray TorchTrainer.
 
 .. autosummary::
     :toctree: doc/
@@ -72,17 +72,28 @@ PyTorch Lightning
     ~train.lightning.LightningCheckpoint
     ~train.lightning.LightningPredictor
 
-HuggingFace Transformers
-************************
+Hugging Face Transformers
+*************************
+
+.. autosummary::
+    :toctree: doc/
+
+    ~train.huggingface.transformers.prepare_trainer
+    ~train.huggingface.transformers.RayTrainReportCallback
+
+.. note::
+
+    We will deprecate `TransformersTrainer`, `TransformersCheckpoint` in Ray 2.8. Please 
+    refer to the :ref:`migration guide <transformers-trainer-migration-guide>` for more info.
 
 .. autosummary::
     :toctree: doc/
 
     ~train.huggingface.TransformersTrainer
     ~train.huggingface.TransformersCheckpoint
 
-HuggingFace Accelerate
-**********************
+Hugging Face Accelerate
+***********************
 
 .. autosummary::
     :toctree: doc/

doc/source/train/examples/transformers/transformers_torch_trainer_basic.rst

@@ -0,0 +1,8 @@
+:orphan:
+
+.. _transformers_torch_trainer_basic_example :
+
+Ray Train Basic Example for HuggingFace Transformers
+====================================================
+
+.. literalinclude:: /../../python/ray/train/examples/transformers/transformers_torch_trainer_basic.py