By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.
Major Features and Improvements
Arch
- Some bugs fixed for spark computing engine
Component
- Unified IO keys naming format for all components
- Add LLMLoader to support running FATE-LLM v2.0 with pipeline
OSX
- Compatible with eggroll-v2.x
- add 2.x api backport support
- bug fix
-
Improved the display issue of output data.
-
Enhanced the PyPI package: configuration files have been relocated to the user's home directory, and the relative paths for uploading data are based on the user's home directory.
-
Added support for running FATE algorithms with Spark + Hadoop.
-
Fixed an issue where failed tasks could not be retried.
-
Fixed an issue where the system couldn't run when the task cores exceeded the system total cores.
- Pipeline: add supports for fate-llm 2.0
- newly added LLMModelLoader, LLMDatasetLoader, LLMDataFuncLoader
- newly added configuration parsing of seq2seq_runner and ot_runner
- Pipeline: unified input interface of components
- Adapt to fate-v2.0 framework:
- Migrate parameter-efficient fine-tuning training methods and models.
- Migrate Standard Offsite-Tuning and Extended Offsite-Tuning（Federated Offsite-Tuning+)
- Newly trainer，dataset, data_processing function design
- New FedKSeed Federated Tuning Algorithm: train large language models in a federated learning setting with extremely low communication cost
- Add Support for Job Runtime Configuration