Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Probabilistically True and Tight Bounds for Robust DNN Training

Training Deep Neural Networks (DNNs) that are robust to norm bounded adversarial attacks remains an elusive problem. While verification based methods are generally too expensive to robustly train large networks, it was demonstrated in Gowal et. al. that bounded input intervals can be inexpensively propagated per layer through large networks. This interval bound propagation (IBP) approach lead to high robustness and was the first to be employed on large networks. However, due to the very loose nature of the IBP bounds, particularly for large networks, the required training procedure is complex and involved. In this paper, we closely examine the bounds of a block of layers composed of an affine layer followed by a ReLU nonlinearity followed by another affine layer. In doing so, we propose probabilistic bounds, true bounds with overwhelming probability, that are provably tighter than IBP bounds in expectation. We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP. With such tight bounds, we demonstrate that a simple standard training procedure can achieve the best robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.


Install it using poetry 0.12.11 (the recommended way):

git clone && cd ptb
poetry install

For Jupyter support:

poetry install --extras jupyter
poetry run python -m ipykernel install --user --name ptb

Or using pip:

pip install git+

Then, check this out:

$ ptb basic --help
Usage: ptb basic [OPTIONS]

  Start basic neural network training.

  -v, --validate / -t, --train    Whether to run in validation or training
                                  mode.  [default: True]
  -d, --dataset TEXT              Which dataset to use.
  -m, --model TEXT                Which model architecture to use.
  -p, --pretrained / -s, --scratch
                                  Whether to load a pretrained model.
                                  [default: False]
  -lr, --learning-rate FLOAT RANGE
                                  Learning rate.
  -mm, --momentum FLOAT RANGE     SGD momentum.
  -w, --weight-decay FLOAT RANGE  SGD weight decay.
  -f, --factor FLOAT RANGE        The trade-off coefficient.
  -t, --temperature FLOAT RANGE   Softmax temperature.
  -n, --number-of-epochs INTEGER RANGE
                                  The maximum number of epochs.
  -b, --batch-size INTEGER RANGE  Mini-batch size.
  -j, --jobs INTEGER RANGE        Number of threads for data loading when
                                  using cuda.
  -c, --checkpoint PATH           A checkpoint file to save the best model.
  -r, --resume PATH               A checkpoint file to resume from.
  -l, --log-dir PATH              A tensorboard logs directory.
  -sd, --seed INTEGER RANGE       Seed the random number generators (slow!).
  -e, --epsilon FLOAT RANGE       Epsilon used for training with interval
  --help                          Show this message and exit.

Please, note that in this current version, training using PTB on multiple GPUs is not supported. However, the code will try to use all available GPUs which will lead to an error with DataParallel. In case you have more than one GPU, don't forget to set the environment variable CUDA_VISIBLE_DEVICES to only one GPU.


This is the official implementation of the method described in this paper:

    title={Expected Tight Bounds for Robust Deep Neural Network Training},
    author={Salman Alsubaihi and Adel Bibi and Modar Alfadly and Abdullah Hamdi and Bernard Ghanem},




Modar M. Alfadly


I would gladly accept any pull request that improves any aspect of this repository.


Probabilistically True and Tight Bounds for Robust Deep Neural Network Training







No releases published


No packages published