AU detectors 'JAANET' and 'DRML' not working out-of-the-box

[E1qU]

For me, all other AU detectors work just fine, but when I use JAANET for AU detection (au_model = "jaanet"), the detector.detect_image() function only outputs to the terminal "exception occurred" and its output is a DataFrame object that only consists of NaN values.

Also, when I use DRML for AU detection (au_model = "drml"), the following error occurs:

Traceback (most recent call last):

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 187, in nti
    n = int(s.strip() or "0", 8)

ValueError: invalid literal for int() with base 8: 'nsor_v2'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 2289, in next
    tarinfo = self.tarinfo.fromtarfile(self)

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 1095, in fromtarfile
    obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 1037, in frombuf
    chksum = nti(buf[148:156])

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 189, in nti
    raise InvalidHeaderError("invalid header")

InvalidHeaderError: invalid header


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\XXX\Anaconda3\lib\site-packages\torch\serialization.py", line 555, in _load
    return legacy_load(f)

  File "C:\Users\XXX\Anaconda3\lib\site-packages\torch\serialization.py", line 466, in legacy_load
    with closing(tarfile.open(fileobj=f, mode='r:', format=tarfile.PAX_FORMAT)) as tar, \

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 1593, in open
    return func(name, filemode, fileobj, **kwargs)

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 1623, in taropen
    return cls(name, mode, fileobj, **kwargs)

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 1486, in __init__
    self.firstmember = self.next()

  File "C:\Users\XXX\Anaconda3\lib\tarfile.py", line 2301, in next
    raise ReadError(str(e))

ReadError: invalid header


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\code\python_files\pyfeat\pyfeat_visualization.py", line 13, in <module>
    detector = Detector(face_model = face_model, landmark_model = landmark_model, au_model = au_model, emotion_model = emotion_model)

  File "C:\Users\XXX\Anaconda3\lib\site-packages\feat\detector.py", line 202, in __init__
    self.au_model = DRMLNet()

  File "C:\Users\XXX\Anaconda3\lib\site-packages\feat\au_detectors\DRML\DRML_test.py", line 26, in __init__
    self.drml_net.load_state_dict(torch.load(self.params["config_write_path_prefix"]))

  File "C:\Users\XXX\Anaconda3\lib\site-packages\torch\serialization.py", line 386, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)

  File "C:\Users\XXX\Anaconda3\lib\site-packages\torch\serialization.py", line 559, in _load
    raise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))

RuntimeError: C:\Users\XXX\Anaconda3\lib\site-packages\feat\resources\DRMLNetParams.pth is a zip archive (did you mean to use torch.jit.load()?)

I installed py-feat using pip install and I have not made any modifications to the source code. I am using a Windows 10 machine and the Anaconda Python environment.

[jcheong0428]

@E1qU : Thank you for bringing this to our attention. On JAANet, can you try a different image and let us know if you still get no predictions (nans, with the error)? The exception returning a nan dataframe could be due to the detectors not being able to find a face. If it also fails on other images including the test image we have on our tutorial, it might indicate another issue

On DRML, thanks for raising this which appears to be an error associated with loading the model. Can you share what PyTorch version you are using? Hopefully that can help us replicate and fix this problem.

[E1qU]

Thank you for the response. It was a good suggestion to try out if JAANet fails on the test image, I was able to find the issue: if I have

au_model = "jaanet"
emotion_model = "resmasknet"

everything works fine. But for some reason, if I have

au_model = "jaanet"
emotion_model = "rf"

the error that I was getting previously occurs. I do not need to use emotion detections in my current task at all, so changing to ResMaskNet is not an issue for me. I only changed to using random forests since I thought that processing with RFs would be slightly faster than using ResMaskNet. However, it is good for you to know that this kind of issue exists if au_model = "jaanet" and emotion_model = "rf" are being used together.

For the second issue regarding DRML, I am currently using version 1.2.0 of PyTorch.