DVC example project for checkpoints

This project is a showcase for the checkpoints features in DVC 2.0 and dvclive. It shows how to use checkpoints in various ways by training a a CNN model to classify Fashion MNIST dataset in Tensorflow.

All four branches are cloned and installed similarly:

git clone https://github.com/iterative/dvc-example-checkpoints-tensorflow -b basic
cd dvc-example-checkpoints-tensorflow
virtualenv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt

You can also clone https://github.com/iterative/dvc-example-checkpoints-tensorflow with all the tags and use git checkout to navigate among them.

Branches

• basic: Shows how to use checkpoints by modifying dvc.yaml.

In dvc.yaml, the following changes are done. You can also specify this by using --checkpoints/-c option to dvc stage add.

    outs:
      - models/fashion-mnist/model.h5:
          checkpoint: true

• dvclive: Shows how to use dvclive callback for Keras to record the model and the metrics automatically in every checkpoint.

    def on_epoch_end(self, epoch, logs=None):
        logs = logs or {}
        for metric, value in logs.items():
            dvclive.log(metric, value)
        dvclive.next_step()

• python-api: Uses make_checkpoint() API call in a custom Tensorflow callback to record a checkpoint in train.py.

    def on_epoch_end(self, epoch, logs=None):
        if (epoch % self.frequency) == 0:
            make_checkpoint()

• signal-file: This branch shows language-independent way of producing checkpoints. Instead of using DVC Python API or DVClive, a signal file is created to set the checkpoint.

    def dvc_signal(self):
        "Generates a DVC signal file and blocks until it's deleted"
        dvc_root = os.getenv("DVC_ROOT") # Root dir of dvc project.
        if dvc_root: # Skip if not running via dvc.
            signal_file = os.path.join(dvc_root, ".dvc", "tmp",
                "DVC_CHECKPOINT")
            with open(signal_file, "w") as f: # Write empty file.
                f.write("")
            while os.path.exists(signal_file): # Wait until dvc deletes file.
                # Wait 10 milliseconds
                time.sleep(0.01)

You can run the experiments similar to the examples in documentation.

Contributing

This repository is generated by example-repos-dev. For Pull Requests regarding the fixes, please use that repository.