FileNotFoundError: [Errno 2] No such file or directory: 'pks/DocTamperV1-TrainingSet/_75.pk'

[256-7421142]

Traceback (most recent call last):
File "/mnt/data/experiments/DocTamper-main/models/train.py", line 114, in
train_data = TamperDataset(train_imgs_dir, 'train')
File "/mnt/data/experiments/DocTamper-main/models/train.py", line 36, in init
with open('pks/'+roots+'_%d.pk'%minq,'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'pks/DocTamperV1-TrainingSet/_75.pk'

I followed your hints and wrote train.py, but I don't know how to generate the pk file for the training set

[256-7421142]

qt_table.pk DocTamperV1-FCD_75.pk... What is the purpose of these files, how are they generated, and how do you pick the right quantisation table?

[qcf-568]

The purpose of them is to record the random selected compression factors for a fair comparison, it can also be seen as keeping the random seed same all the test time. We pick the right quant-table by choosing a compress factor then get the corresponding standard quant-table from OpenCV.

[256-7421142]

The purpose of them is to record the random selected compression factors for a fair comparison, it can also be seen as keeping the random seed same all the test time. We pick the right quant-table by choosing a compress factor then get the corresponding standard quant-table from OpenCV.

Can you elaborate on how the corresponding quant_table is generated based on Curriculum learning (CL) during training, or can you provide the pk file of the trained quant_table, I don't know how to generate the corresponding pk file using OpenCV.

[qcf-568]

I didn't use any .pk file to conduct CL, the compression factors are randomly and dynamically generated in the dataset class. I am forbidden to show the actual codes publicly now.

[qcf-568]

The training process doesn't need such .pk file (e.g. xxx_75.pk xxx_90.pk), some of the codes of the dataset class need to be adjusted before training stage.

[256-7421142]

        record = self.record[index]
        choicei = len(record)-1
        q = int(record[-1]) 
        use_qtb = self.pks[q]
        if choicei>1:
            q2 = int(record[-3])
            use_qtb2 = self.pks[q2]
        if choicei>0:
            q1 = int(record[-2])
            use_qtb1 = self.pks[q1]

Why is the length of the record inconsistent?

[256-7421142]

The purpose of them is to record the random selected compression factors for a fair comparison, it can also be seen as keeping the random seed same all the test time. We pick the right quant-table by choosing a compress factor then get the corresponding standard quant-table from OpenCV.

record = self.record[index]
    choicei = len(record)-1
    q = int(record[-1]) 
    use_qtb = self.pks[q]
    if choicei>1:
        q2 = int(record[-3])
        use_qtb2 = self.pks[q2]
    if choicei>0:
        q1 = int(record[-2])
        use_qtb1 = self.pks[q1]

Why is the length of the record inconsistent?

[qcf-568]

This part of codes is to choice the times (1,2 or 3) for image compression. Sometimes images are compressed more than once. The times for compression are randomly generated before being recorded.

[256-7421142]

The purpose of them is to record the random selected compression factors for a fair comparison, it can also be seen as keeping the random seed same all the test time. We pick the right quant-table by choosing a compress factor then get the corresponding standard quant-table from OpenCV.

How is the quantised table generated during training, the range of values in the quantised table is in 0-63?When I am reproducing the CL training strategy, the quantised table generated based on qtable = jpg.quant_tables[0] exceeds this range, and the input to self.qtembed = nn.Embedding(64,16) gives error RuntimeError: CUDA error: device-side assert triggered

[qcf-568]

Yes, the range of values is forced to be limited to 0-63.