Generated image for CoreML is all black

[RahulBhalley]

Hi @bes-dev, thanks for your work!

I am actually not able to convert MobileStyleGAN correctly. I am using the following command:

python train.py --cfg configs/mobile_stylegan_ffhq.json --checkpoint_dir . --ckpt mobilestylegan_ffhq_v2.ckpt --export-model coreml

But the generated image is all black. If I replace the synthesis network with the one provided here then the image does get generated fine. Meaning that MappingNetwork.mlmodel's parameters are loaded but not of SynthesisNetwork.mlmodel. Note that I do download the mobilestylegan_ffhq_v2.ckpt checkpoint and save it in root directory of this repository.

[RahulBhalley]

Could you please help me resolve this issue?

[bes-dev]

@RahulBhalley it works well on my side. Please, provide more information about your environment to reproduce this issue.

[RahulBhalley]

Thanks for replying @bes-dev! (Mentioning you to notify you.)

I muted & changed some code to conquer some Neptune logger related error I mentioned in #30 issue. I made minor changes to pl.callbacks.ModelCheckpoint for pytorch_lightning 1.5.9 as shown in screenshot below.

Here's my system information (package list is long!):

OS: macOS Monterey 12.2
Language: Python 3.7.12
Pip-managed packages:

Package                  Version
------------------------ ---------
absl-py                  1.0.0
addict                   2.2.1
aiohttp                  3.8.1
aiosignal                1.2.0
appnope                  0.1.2
argon2-cffi              21.3.0
argon2-cffi-bindings     21.2.0
arrow                    1.2.2
astor                    0.8.1
astunparse               1.6.3
async-timeout            4.0.2
asynctest                0.13.0
attrs                    21.4.0
backcall                 0.2.0
beautifulsoup4           4.10.0
bleach                   4.1.0
boto3                    1.20.49
botocore                 1.23.49
bravado                  11.0.3
bravado-core             5.17.0
cached-property          1.5.2
cachetools               4.2.4
certifi                  2021.10.8
cffi                     1.15.0
charset-normalizer       2.0.11
click                    8.0.3
cma                      3.1.0
coremltools              5.1.0
cycler                   0.11.0
debugpy                  1.5.1
decorator                5.1.1
defusedxml               0.7.1
dlib                     19.23.0
entrypoints              0.3
filelock                 3.4.2
fonttools                4.29.1
fqdn                     1.5.1
frozenlist               1.3.0
fsspec                   2022.1.0
future                   0.18.2
gast                     0.3.3
gdown                    4.2.1
gitdb                    4.0.9
GitPython                3.1.26
google-api-core          2.5.0
google-api-python-client 2.36.0
google-auth              1.35.0
google-auth-httplib2     0.1.0
google-auth-oauthlib     0.4.6
google-pasta             0.2.0
googleapis-common-protos 1.54.0
grpcio                   1.43.0
gudhi                    3.5.0
h5py                     2.10.0
httplib2                 0.20.4
huggingface-hub          0.4.0
idna                     3.3
importlib-metadata       4.10.1
importlib-resources      5.4.0
ipykernel                6.7.0
ipython                  7.31.1
ipython-genutils         0.2.0
ipywidgets               7.6.5
isoduration              20.11.0
jedi                     0.18.1
Jinja2                   3.0.3
jmespath                 0.10.0
joblib                   1.1.0
jsonpointer              2.2
jsonref                  0.2
jsonschema               4.4.0
jupyter                  1.0.0
jupyter-client           7.1.2
jupyter-console          6.4.0
jupyter-core             4.9.1
jupyterlab-pygments      0.1.2
jupyterlab-widgets       1.0.2
Keras                    2.0.0
Keras-Applications       1.0.8
Keras-Preprocessing      1.1.2
kiwisolver               1.3.2
kornia                   0.4.1
Markdown                 3.3.6
MarkupSafe               2.0.1
matplotlib               3.5.1
matplotlib-inline        0.1.3
mistune                  0.8.4
mock                     4.0.3
monotonic                1.6
mpmath                   1.2.1
msgpack                  1.0.3
multidict                6.0.2
nbclient                 0.5.10
nbconvert                6.4.1
nbformat                 5.1.3
neptune-client           0.4.132
nest-asyncio             1.5.4
notebook                 6.4.8
numpy                    1.21.5
oauth2client             4.1.3
oauthlib                 3.2.0
opencv-python            4.5.5.62
opt-einsum               3.3.0
packaging                21.3
pandas                   1.3.5
pandocfilters            1.5.0
parso                    0.8.3
pexpect                  4.8.0
pickleshare              0.7.5
Pillow                   9.0.0
pip                      21.2.4
piq                      0.5.2
prometheus-client        0.13.1
prompt-toolkit           3.0.26
protobuf                 3.19.4
psutil                   5.9.0
ptyprocess               0.7.0
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pycparser                2.21
pyDeprecate              0.3.1
PyDrive                  1.3.1
Pygments                 2.11.2
PyJWT                    2.3.0
pyparsing                3.0.7
pyrsistent               0.18.1
PySocks                  1.7.1
python-dateutil          2.8.2
pytorch-fid              0.2.1
pytorch-lightning        1.5.9
pytorch-wavelets         1.3.0
pytz                     2021.3
PyWavelets               1.1.1
PyYAML                   6.0
pyzmq                    22.3.0
qibo                     0.1.6
qtconsole                5.2.2
QtPy                     2.0.0
regex                    2022.1.18
requests                 2.27.1
requests-oauthlib        1.3.1
rfc3339-validator        0.1.4
rfc3987                  1.3.8
rsa                      4.8
s3transfer               0.5.1
sacremoses               0.0.47
scipy                    1.4.1
Send2Trash               1.8.0
setuptools               59.5.0
simplejson               3.17.6
six                      1.16.0
smmap                    5.0.0
soupsieve                2.3.1
swagger-spec-validator   2.7.4
sympy                    1.9
tensorboard              2.2.2
tensorboard-data-server  0.6.1
tensorboard-plugin-wit   1.8.1
tensorflow               2.2.0
tensorflow-estimator     2.2.0
termcolor                1.1.0
terminado                0.13.1
testpath                 0.5.0
tokenizers               0.11.4
torch                    1.10.2
torchaudio               0.10.2
torchmetrics             0.7.1
torchtext                0.11.2
torchvision              0.11.3
tornado                  6.1
tqdm                     4.62.3
traitlets                5.1.1
transformers             4.16.2
typing_extensions        4.0.1
uri-template             1.1.0
uritemplate              4.1.1
urllib3                  1.26.8
wcwidth                  0.2.5
webcolors                1.11.1
webencodings             0.5.1
websocket-client         1.2.3
Werkzeug                 2.0.2
wheel                    0.37.0
widgetsnbextension       3.5.2
wrapt                    1.13.3
yarl                     1.7.2
zipp                     3.7.0

[RahulBhalley]

I created a virtual environment with Python 3.7.12 and then install all the requirements. I also reverted all the code back to original except for commenting out the logger arguments. Now the package list is as follows:

absl-py==1.0.0
addict==2.2.1
arrow==1.2.2
attrs==21.4.0
bravado==11.0.3
bravado-core==5.17.0
cached-property==1.5.2
cachetools==5.0.0
certifi==2021.10.8
charset-normalizer==2.0.11
click==8.0.3
coremltools==5.1.0
filelock==3.4.2
fqdn==1.5.1
fsspec==2022.1.0
future==0.18.2
gdown==3.12.2
gitdb==4.0.9
GitPython==3.1.26
google-auth==2.6.0
google-auth-oauthlib==0.4.6
grpcio==1.43.0
gudhi==3.5.0
idna==3.3
importlib-metadata==4.11.0
importlib-resources==5.4.0
isoduration==20.11.0
jsonpointer==2.2
jsonref==0.2
jsonschema==4.4.0
kornia==0.4.1
Markdown==3.3.6
monotonic==1.6
mpmath==1.2.1
msgpack==1.0.3
neptune-client==0.4.132
numpy==1.17.5
oauthlib==3.2.0
packaging==21.3
pandas==1.3.5
Pillow==9.0.1
piq==0.5.2
protobuf==3.19.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
PyJWT==2.3.0
pyparsing==3.0.7
pyrsistent==0.18.1
PySocks==1.7.1
python-dateutil==2.8.2
pytorch-fid==0.2.1
pytorch-lightning==1.0.2
pytorch-wavelets @ git+git://github.com/fbcotter/pytorch_wavelets.git@8d2e3b4289beaea9aa89f7b1dbb290e448331197
pytz==2021.3
PyWavelets==1.1.1
PyYAML==6.0
requests==2.27.1
requests-oauthlib==1.3.1
rfc3339-validator==0.1.4
rfc3987==1.3.8
rsa==4.8
scipy==1.7.3
simplejson==3.17.6
six==1.16.0
smmap==5.0.0
swagger-spec-validator==2.7.4
sympy==1.9
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
torch==1.9.1
torchvision==0.11.3
tqdm==4.62.3
typing_extensions==4.0.1
uri-template==1.1.0
urllib3==1.26.8
webcolors==1.11.1
websocket-client==1.2.3
Werkzeug==2.0.3
zipp==3.7.0

Pinging you once again @bes-dev because it's really important for me.

[RahulBhalley]

The observations are same as the previous one. Weird, why doesn't conversion work for me?

[RahulBhalley]

I figured out by using your tensor_to_img function that I'm not post-processing the output tensor.

So I'm closing this issue. Thanks for quick response btw and your work too!