-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement estimator for Hugging Face models #2245
Changes from all commits
b0b5be0
2e32345
7a33398
8faa487
b7ed033
7728cb2
9e0246f
101a309
c314c11
b6d3fb0
5005e08
f42e115
1f3bfcc
11c8d91
8d96eea
df40e1c
9399ccd
551da0b
c3d52ef
4249610
17f4f46
f49978c
ef805e8
d209c0f
ae3b343
6498386
126460a
d9326ff
c1c5b94
2bab5d6
de1d477
86645ec
fffed5a
e2f2ab7
88a2791
ed65928
ad3d581
d178ab6
5d7b08d
daa7c62
9208af9
124f6c8
2e7a29a
e67388f
e520782
41ac37d
5cccfba
f9fcc3f
0178cbe
0914f99
9ed5e32
d0f3fa1
b3cbf95
5d6f11a
89bd452
44bdefe
6f37944
25d357a
fe91036
3d4fe0d
6a6f094
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
name: CI Huggingface | ||
on: | ||
# Run on manual trigger | ||
workflow_dispatch: | ||
|
||
# Run on pull requests | ||
pull_request: | ||
paths-ignore: | ||
- '*.md' | ||
|
||
# Run on merge queue | ||
merge_group: | ||
|
||
# Run when pushing to main or dev branches | ||
push: | ||
branches: | ||
- main | ||
- dev* | ||
|
||
# Run scheduled CI flow daily | ||
schedule: | ||
- cron: '0 8 * * 0' | ||
|
||
jobs: | ||
test: | ||
runs-on: ubuntu-20.04 | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
include: | ||
- name: Huggingface 4.30 | ||
framework: huggingface | ||
python: 3.9 | ||
torch: 1.13.1+cpu | ||
torchvision: 0.14.1+cpu | ||
torchaudio: 0.13.1 | ||
transformers: 4.30.2 | ||
|
||
name: ${{ matrix.name }} | ||
steps: | ||
- name: Checkout Repo | ||
uses: actions/checkout@v3 | ||
- name: Setup Python | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: ${{ matrix.python }} | ||
- name: Install Dependencies | ||
run: | | ||
sudo apt-get update | ||
sudo apt-get -y -q install ffmpeg libavcodec-extra | ||
python -m pip install --upgrade pip setuptools wheel | ||
pip3 install -r requirements_test.txt | ||
pip install tensorflow==2.10.1 | ||
pip install keras==2.10.0 | ||
pip install torch==${{ matrix.torch }} -f https://download.pytorch.org/whl/cpu/torch_stable.html | ||
pip install torchvision==${{ matrix.torchvision }} -f https://download.pytorch.org/whl/cpu/torch_stable.html | ||
pip install torchaudio==${{ matrix.torchaudio }} -f https://download.pytorch.org/whl/cpu/torch_stable.html | ||
pip install transformers==${{ matrix.transformers }} | ||
pip list | ||
- name: Run Tests | ||
run: ./run_tests.sh ${{ matrix.framework }} | ||
- name: Upload coverage to Codecov | ||
uses: codecov/codecov-action@v3 | ||
with: | ||
fail_ci_if_error: true |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -49,18 +49,29 @@ def add_single_bd(x: np.ndarray, distance: int = 2, pixel_value: int = 1) -> np. | |
return x | ||
|
||
|
||
def add_pattern_bd(x: np.ndarray, distance: int = 2, pixel_value: int = 1) -> np.ndarray: | ||
def add_pattern_bd(x: np.ndarray, distance: int = 2, pixel_value: int = 1, channels_first: bool = False) -> np.ndarray: | ||
GiulioZizzo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
""" | ||
Augments a matrix by setting a checkerboard-like pattern of values some `distance` away from the bottom-right | ||
edge to 1. Works for single images or a batch of images. | ||
|
||
:param x: A single image or batch of images of shape NWHC, NHW, or HC. Pixels will be added to all channels. | ||
:param distance: Distance from bottom-right walls. | ||
:param pixel_value: Value used to replace the entries of the image matrix. | ||
:param channels_first: If the data is provided in channels first format we transpose to NWHC or HC depending on | ||
input shape | ||
:return: Backdoored image. | ||
""" | ||
x = np.copy(x) | ||
original_dtype = x.dtype | ||
shape = x.shape | ||
if channels_first: | ||
if len(shape) == 4: | ||
# Transpose the image putting channels last | ||
x = np.transpose(x, (0, 2, 3, 1)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the transpose with 4 indices work if There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point: that should only apply to 4D inputs and an additional one is needed for 2D to go from HC to CH. |
||
if len(shape) == 2: | ||
# HC to CH | ||
x = np.transpose(x) | ||
|
||
if len(shape) == 4: | ||
height, width = x.shape[1:3] | ||
x[:, height - distance, width - distance, :] = pixel_value | ||
|
@@ -81,7 +92,15 @@ def add_pattern_bd(x: np.ndarray, distance: int = 2, pixel_value: int = 1) -> np | |
x[height - distance - 2, width - distance] = pixel_value | ||
else: | ||
raise ValueError(f"Invalid array shape: {shape}") | ||
return x | ||
|
||
if channels_first: | ||
if len(shape) == 4: | ||
# Putting channels first again | ||
x = np.transpose(x, (0, 3, 1, 2)) | ||
if len(shape) == 2: | ||
x = np.transpose(x) | ||
|
||
return x.astype(original_dtype) | ||
|
||
|
||
def insert_image( | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this change required?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There could be several items in
dist
with the same minimum value. Hence,dist_min_index
, rather than being an expected index, is a tensor with several values. To avoid this crashing the attack, we pick the first returned index. This bug was discovered when adapting the attacks to run with HF.