find_critical_para() returns empty minus_row_cos, minus_col_cos

[vietvo89]

Hi, when I ran find_critical_para(), it always returns empty minus_row_cos, minus_col_cos. Can you double-check this code? It seems that modules are associated with map or self, has no param.grad or param.grad=None.

[vietvo89]

When I printed print(name, param.grad)

The output looks like this

model.embed_tokens.weight tensor([[ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
        [ 0.2190, -0.0189, -0.0076,  ..., -0.0893,  0.0321,  0.0510],
        [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
        ...,
        [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  ...,  0.0000,  0.0000,  0.0000]],
       device='cuda:0', dtype=torch.float16)
model.layers.0.self_attn.q_proj.weight None
model.layers.0.self_attn.k_proj.weight None
model.layers.0.self_attn.v_proj.weight None
model.layers.0.self_attn.o_proj.weight None
model.layers.0.mlp.gate_proj.weight None
model.layers.0.mlp.up_proj.weight None
model.layers.0.mlp.down_proj.weight None
model.layers.0.input_layernorm.weight tensor([-0.0141,  0.0554,  0.0344,  ...,  0.0705, -0.0140, -0.0198],
       device='cuda:0', dtype=torch.float16)
model.layers.0.post_attention_layernorm.weight tensor([-0.0043,  0.0043, -0.0031,  ...,  0.0039, -0.0089,  0.0064],
       device='cuda:0', dtype=torch.float16)
model.layers.1.self_attn.q_proj.weight None
model.layers.1.self_attn.k_proj.weight None
model.layers.1.self_attn.v_proj.weight None
model.layers.1.self_attn.o_proj.weight None
model.layers.1.mlp.gate_proj.weight None
model.layers.1.mlp.up_proj.weight None
model.layers.1.mlp.down_proj.weight None
model.layers.1.input_layernorm.weight tensor([-0.0075, -0.0037,  0.0098,  ..., -0.0129, -0.0085,  0.0042],
       device='cuda:0', dtype=torch.float16)
model.layers.1.post_attention_layernorm.weight tensor([-0.0084,  0.0184, -0.0048,  ...,  0.0079, -0.0100, -0.0099],
       device='cuda:0', dtype=torch.float16)

If this is normal, I do not know how your code below works.

            if  param.grad is not None and ("mlp" in name or "self" in name):
                grad_norm = param.grad.to(gradient_norms_compare[name].device)
                row_cos = torch.nan_to_num(F.cosine_similarity(grad_norm, (gradient_norms_compare[name]), dim=1))
                col_cos = torch.nan_to_num(F.cosine_similarity(grad_norm, (gradient_norms_compare[name]), dim=0))
                if name not in row_coss:
                    row_coss[name] = row_cos 
                    col_coss[name] = col_cos
                else:
                    row_coss[name] += row_cos 
                    col_coss[name] += col_cos

[xyq7]

Hi, may I know what command you are running?

[vietvo89]

hi, I ran test_xstest.py. I used meta-llama/Llama-2-7b-chat-hf. Since your stuff occupies a lot of GPU when running find_critical_para() and my machine has insufficient GPU, I used the quantized 8b of that model instead. You can replicate this problem by using load_in_8bit = True when loading the model.

[vietvo89]

One more thing not related to this issue, it seems your method is designed to detect human-crafted jailbreak prompts. It cannot detect jailbreak prompts generated by jailbreak attack algorithms. Is it right? I did not see experiments in the main paper to evaluate against jailbreak attacks.

[xyq7]

Hi, Yes, we didn't test jailbreak prompts generated by jailbreak attack algorithms, but only those included in the dataset we use in the paper. May test the effectiveness to see whether it works.

As for the issue, sorry that I'm busy with projects these days, I will try my best to see what's the problem related to the issue this week.