Can't run examples #2735

albertoxamin · 2019-10-15T19:00:12Z

Describe the bug
Completely unable to execute examples from the documentation

To Reproduce
Steps to reproduce the behavior:

Go to any example scene
Click on play
See error

Console logs / stack traces

DllNotFoundException: macblas
MacBLAS.SGEMM (System.Single* ap, System.Int32 an, System.Int32 am, System.Single* bp, System.Int32 bn, System.Int32 bm, System.Single* cp, System.Int32 cn, System.Int32 cm, System.Int32 bs, System.Boolean transposeA, System.Boolean transposeB) (at Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/MacBLAS.cs:26)
Barracuda.UnsafeArrayCPUOps.Dense (Barracuda.Tensor X, Barracuda.Tensor W, Barracuda.Tensor B) (at <68dd6e4acc004c7c8bb12c0e7eb964f9>:0)
Barracuda.GenericWorker+<ExecuteAsync>d__26.MoveNext () (at <68dd6e4acc004c7c8bb12c0e7eb964f9>:0)
Barracuda.GenericWorker.Execute () (at <68dd6e4acc004c7c8bb12c0e7eb964f9>:0)
Barracuda.GenericWorker.Execute (System.Collections.Generic.IDictionary`2[TKey,TValue] inputs) (at <68dd6e4acc004c7c8bb12c0e7eb964f9>:0)
MLAgents.LearningBrain.DecideAction () (at Assets/ML-Agents/Scripts/LearningBrain.cs:158)
MLAgents.Brain.BrainDecideAction () (at Assets/ML-Agents/Scripts/Brain.cs:98)
MLAgents.Academy.EnvironmentStep () (at Assets/ML-Agents/Scripts/Academy.cs:558)
MLAgents.Academy.FixedUpdate () (at Assets/ML-Agents/Scripts/Academy.cs:583)

Screenshots

Environment (please complete the following information):

OS + version: Mac OS Catalina 10.15 (19A583)
ML-Agents version: ML-Agents v0.10.1
Environment: 3DBall

The text was updated successfully, but these errors were encountered:

albertoxamin · 2019-10-15T20:55:51Z

Could it be that this problem is caused by macOS Catalina that dropped 32 bit support and the plugin was not compiled for 64bit? @mantasp

chriselion · 2019-10-15T22:40:24Z

Hi @albertoxamin - we'll look into this. I don't think 32-bit is the problem; if you run

$ file UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS/macblas
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS/macblas: Mach-O 64-bit bundle x86_64

but there might be a problem with the code signing.

surfnerd · 2019-10-15T23:32:26Z

Hey @albertoxamin,
After a bit of inspection, it seems like macOS Catalina may be a bit more strict about checking that a .bundle code signature is valid.

codesign --verify -v "UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle"
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: a sealed resource is missing or invalid
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS/macblas.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/Info.plist.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/_CodeSignature.meta

Since a .bundle is actually just a directory, Unity adds .meta files where, maybe, it shouldn't. After removing the .meta files from the bundle I see:

codesign --verify -v "UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle"
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: valid on disk
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: satisfies its Designated Requirement

I will test later tonight on my personal laptop which has Catalina installed to gather more information. Thanks for the report!

surfnerd · 2019-10-16T15:33:53Z

Hi @albertoxamin,
I tried to reproduce your issue on Catalina (19B68f) and the previous version, with unity versions 2017.4.33f and 2019.3.b06. I was able to run training on both with no issues on ML-Agents 0.10.1. Can you give us any more information about your personal setup?

albertoxamin · 2019-10-16T16:25:13Z

I'm using Unity 2017.4.32f1 downloaded from Unity Hub
PATH

/usr/local/opt/qt/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Users/alberto/Library/Android//tools:/Users/alberto/Library/Android//platform-tools:/opt/local/bin:/opt/local/sbin:/usr/local/opt/libxml2/bin:/Users/alberto/.pyenv/shims:/Users/alberto/.nvm/versions/node/v10.10.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/alberto/Development/flutter/bin:/Library/TeX/texbin:/usr/local/share/dotnet:/opt/X11/bin:~/.dotnet/tools:/Applications/Wireshark.app/Contents/MacOS:/Users/alberto/.antigen/bundles/ael-code/zsh-colored-man-pages:/Users/alberto/gocode/bin:/usr/local/bin:/usr/local/opt/fzf/bin

PYTHON

λ alberto [Barracuda/Plugins/OSX] → python --version
Python 3.7.0

λ alberto [Barracuda/Plugins/OSX] → python -m pip list
Package               Version
--------------------- --------
absl-py               0.8.1
appnope               0.1.0
asciinema             2.0.2
astor                 0.8.0
astroid               2.3.1
attrs                 19.3.0
autopep8              1.4.4
backcall              0.1.0
bleach                3.1.0
cloudpickle           1.2.2
cycler                0.10.0
decorator             4.4.0
defusedxml            0.6.0
entrypoints           0.3
gast                  0.3.2
google-pasta          0.1.7
grpcio                1.24.1
h5py                  2.10.0
importlib-metadata    0.23
ipykernel             5.1.2
ipython               7.8.0
ipython-genutils      0.2.0
ipywidgets            7.5.1
isort                 4.3.21
jedi                  0.15.1
Jinja2                2.10.3
jsonschema            3.1.1
jupyter               1.0.0
jupyter-client        5.3.4
jupyter-console       6.0.0
jupyter-core          4.6.0
Keras-Applications    1.0.8
Keras-Preprocessing   1.1.0
kiwisolver            1.1.0
lazy-object-proxy     1.4.2
Markdown              3.1.1
MarkupSafe            1.1.1
matplotlib            3.1.1
mccabe                0.6.1
mistune               0.8.4
mlagents              0.10.1
mlagents-envs         0.10.1
more-itertools        7.2.0
nbconvert             5.6.0
nbformat              4.4.0
notebook              6.0.1
numpy                 1.17.2
opencv-contrib-python 4.1.1.26
opencv-python         4.1.1.26
pandocfilters         1.4.2
parso                 0.5.1
pexpect               4.7.0
pickleshare           0.7.5
Pillow                6.2.0
pip                   19.3
prometheus-client     0.7.1
prompt-toolkit        2.0.10
protobuf              3.10.0
psycopg2              2.7.6.1
psycopg2-binary       2.7.6.1
ptyprocess            0.6.0
pycodestyle           2.5.0
Pygments              2.4.2
pylint                2.4.2
pyparsing             2.4.2
pyrsistent            0.15.4
python-dateutil       2.8.0
PyYAML                5.1.2
pyzmq                 18.1.0
qtconsole             4.5.5
Send2Trash            1.5.0
setuptools            39.0.1
six                   1.12.0
tensorboard           1.14.0
tensorflow            1.14.0
tensorflow-estimator  1.14.0
termcolor             1.1.0
terminado             0.8.2
testpath              0.4.2
tornado               6.0.3
traitlets             4.3.3
typed-ast             1.4.0
wcwidth               0.1.7
webencodings          0.5.1
Werkzeug              0.16.0
wheel                 0.33.6
widgetsnbextension    3.5.1
wrapt                 1.11.2
zipp                  0.6.0

SYSTEM REPORT

Hardware Overview:

  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro14,2
  Processor Name:	Dual-Core Intel Core i5
  Processor Speed:	3,1 GHz
  Number of Processors:	1
  Total Number of Cores:	2
  L2 Cache (per Core):	256 KB
  L3 Cache:	4 MB
  Hyper-Threading Technology:	Enabled
  Memory:	16 GB
  Boot ROM Version:	202.0.0.0.0
  SMC Version (system):	2.44f2

albertoxamin · 2019-10-16T16:33:24Z

I just tried with unity 2019.2.3f1 and got the same error.

I even tried to do this and got your same results

Hey @albertoxamin,
After a bit of inspection, it seems like macOS Catalina may be a bit more strict about checking that a .bundle code signature is valid.

codesign --verify -v "UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle"
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: a sealed resource is missing or invalid
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/MacOS/macblas.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/Info.plist.meta
file added: UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle/Contents/_CodeSignature.meta

Since a .bundle is actually just a directory, Unity adds .meta files where, maybe, it shouldn't. After removing the .meta files from the bundle I see:

codesign --verify -v "UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle"
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: valid on disk
UnitySDK/Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle: satisfies its Designated Requirement

I will test later tonight on my personal laptop which has Catalina installed to gather more information. Thanks for the report!

surfnerd · 2019-10-16T16:59:58Z

The only major difference that I see potentially is that I'm on a beta version of Catalina. I can try to partition my drive tonight or tomorrow and install the public release in order to get closer to your setup. Thanks for the extra info.

albertoxamin · 2019-10-16T18:15:40Z

Apple just pushed an update on Catalina, I'm downloading it right now and I will report if anything changes.

Edit: No luck I'm still encountering this problem in build 19A602

albertoxamin · 2019-10-16T19:09:23Z

UPDATE, I was able to make it work.
Apparently downloading the repository as zip marks the bundle as untrusted, but cloning the repo with git doesn't.

For anyone in the future that encounters this problem, just do a git clone.

If you don't want to download again the repo you could also run this on the terminal (from the Plugins/OSX folder): xattr -d com.apple.quarantine macblas.bundle

surfnerd · 2019-10-16T19:40:41Z

Wow, interesting. Thanks for the update @albertoxamin.

Hylus · 2019-11-22T23:41:28Z

Same problem here but i downloaded repository as zip, run terminal and receive this messageMacBook-Pro:OSX maciej$ xattr -d com.apple.quarantine macblas.bundle xattr: macblas.bundle: No such xattr: com.apple.quarantine

albertoxamin · 2019-11-23T07:36:38Z

Which version of MacOS are you using?

…

On 23 Nov 2019, 00:41 +0100, Maciej Hyla ***@***.***>, wrote: Same problem here but i downloaded repository as zip, run terminal and receive this messageMacBook-Pro:OSX maciej$ xattr -d com.apple.quarantine macblas.bundle xattr: macblas.bundle: No such xattr: com.apple.quarantine — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

jussikuosa · 2020-12-15T14:00:49Z

@Hylus I got the project to work with the above instrucions on 2019.4.9f1.

I downloaded the project from drive, removed the .meta files with

cd /Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX/macblas.bundle
find . -name '*.meta' -delete

and ran xattr in ./Assets/ML-Agents/Plugins/Barracuda.Core/Barracuda/Plugins/OSX

github-actions · 2021-01-29T20:08:25Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

albertoxamin added the bug Issue describes a potential bug in ml-agents. label Oct 15, 2019

surfnerd added the barracuda Bugs related to Barracuda label Oct 15, 2019

chriselion self-assigned this Oct 15, 2019

surfnerd self-assigned this Oct 15, 2019

albertoxamin closed this as completed Oct 16, 2019

github-actions bot locked as resolved and limited conversation to collaborators Jan 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't run examples #2735

Can't run examples #2735

albertoxamin commented Oct 15, 2019

albertoxamin commented Oct 15, 2019

chriselion commented Oct 15, 2019 •

edited

surfnerd commented Oct 15, 2019 •

edited

surfnerd commented Oct 16, 2019

albertoxamin commented Oct 16, 2019

albertoxamin commented Oct 16, 2019

surfnerd commented Oct 16, 2019

albertoxamin commented Oct 16, 2019 •

edited

albertoxamin commented Oct 16, 2019 •

edited

surfnerd commented Oct 16, 2019

Hylus commented Nov 22, 2019

albertoxamin commented Nov 23, 2019 via email

jussikuosa commented Dec 15, 2020

github-actions bot commented Jan 29, 2021

Can't run examples #2735

Can't run examples #2735

Comments

albertoxamin commented Oct 15, 2019

albertoxamin commented Oct 15, 2019

chriselion commented Oct 15, 2019 • edited

surfnerd commented Oct 15, 2019 • edited

surfnerd commented Oct 16, 2019

albertoxamin commented Oct 16, 2019

albertoxamin commented Oct 16, 2019

surfnerd commented Oct 16, 2019

albertoxamin commented Oct 16, 2019 • edited

albertoxamin commented Oct 16, 2019 • edited

surfnerd commented Oct 16, 2019

Hylus commented Nov 22, 2019

albertoxamin commented Nov 23, 2019 via email

jussikuosa commented Dec 15, 2020

github-actions bot commented Jan 29, 2021

chriselion commented Oct 15, 2019 •

edited

surfnerd commented Oct 15, 2019 •

edited

albertoxamin commented Oct 16, 2019 •

edited

albertoxamin commented Oct 16, 2019 •

edited