Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

breeze kvstore topology does not work #13

Closed
cmpt-mnt opened this issue Dec 3, 2017 · 4 comments
Closed

breeze kvstore topology does not work #13

cmpt-mnt opened this issue Dec 3, 2017 · 4 comments

Comments

@cmpt-mnt
Copy link

cmpt-mnt commented Dec 3, 2017

Issue Description

The breeze kvstore topology does not work because matplotlib was not installed at build_openr.sh script.

Environment

  • Latest commit 54566cc
  • OS version: ubuntu-16.04

Minimal test code / Steps to reproduce the issue

  1. Install openr using one step build script build_openr.sh.
  2. Run openr.
  3. Execute following command.
$ breeze kvstore topology

What's the actual result?

$ breeze kvstore topology
matplotlib and networkx needed for drawing. Skipping

What's the expected result?

The command generates png file to /tmp/

solution I guess

  • add following lines to kvstore.py in openr/py/openr/cli/commands:
import matplotlib
matplotlib.use('Agg')
  • execute pip install matplotlib
  • Then, try build_openr.sh
@saifhhasan
Copy link
Contributor

Can you install matplotlib and run breeze kvstore topology ?

Reason for this error message is that we don't want to enforce matplotlib to be a hard dependency of breeze tool (as it is very big library). Hence we made it soft dependency

You don't need to rebuild anything. Just install matplotlib and run command again. I will change message to be more specific. Let me know if it works for you.

@zhanleewo
Copy link
Contributor

zhanleewo commented Dec 5, 2017

sudo apt-get install python-matplotlib

@saifhhasan
Copy link
Contributor

commit e4d9ff3#diff-304d33f4448afe5d095ce2e660da325b add better error description of how to fix it. Closing the task.

@cmpt-mnt
Copy link
Author

cmpt-mnt commented Dec 5, 2017

Thanks, I understood the reason why matplotlib was not installed. The better error description sounds good.
But my environment, the topology command cannot work after installing matplotlib...

root@u6:~/openr/build# breeze kvstore topology
Drawing topology requires `matplotlib` and `networkx` libraries. You can install them with following command and retry. 
  pip install matplotlib
  pip install networkx
root@u6:~/openr/build# pip install matplotlib
Collecting matplotlib
  Downloading matplotlib-2.1.0-cp27-cp27mu-manylinux1_x86_64.whl (14.9MB)
    100% |████████████████████████████████| 14.9MB 75kB/s 
Collecting numpy>=1.7.1 (from matplotlib)
  Downloading numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl (16.6MB)
    100% |████████████████████████████████| 16.7MB 70kB/s 
Collecting cycler>=0.10 (from matplotlib)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six>=1.10 in /usr/local/lib/python2.7/dist-packages/six-1.11.0-py2.7.egg (from matplotlib)
Collecting pytz (from matplotlib)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 2.5MB/s 
Collecting subprocess32 (from matplotlib)
  Downloading subprocess32-3.2.7.tar.gz (54kB)
    100% |████████████████████████████████| 61kB 8.4MB/s 
Collecting python-dateutil>=2.0 (from matplotlib)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 5.4MB/s 
Collecting backports.functools-lru-cache (from matplotlib)
  Downloading backports.functools_lru_cache-1.4-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 9.8MB/s 
Building wheels for collected packages: subprocess32
  Running setup.py bdist_wheel for subprocess32 ... done
  Stored in directory: /root/.cache/pip/wheels/7d/4c/a4/ce9ceb463dae01f4b95e670abd9afc8d65a45f38012f8030cc
Successfully built subprocess32
Installing collected packages: numpy, cycler, pytz, subprocess32, python-dateutil, backports.functools-lru-cache, pyparsing, matplotlib
Successfully installed backports.functools-lru-cache-1.4 cycler-0.10.0 matplotlib-2.1.0 numpy-1.13.3 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.3 subprocess32-3.2.7
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@u6:~/openr/build# pip install numpy
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/local/lib/python2.7/dist-packages
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@u6:~/openr/build# breeze kvstore topology
Drawing topology requires `matplotlib` and `networkx` libraries. You can install them with following command and retry. 
  pip install matplotlib
  pip install networkx

And I try the command after apt install python-matplotlib, then a different error has come.

root@u6:~/openr/build# apt install python-matplotlib
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  blt fontconfig fontconfig-config fonts-lyx hicolor-icon-theme
  javascript-common libatk1.0-0 libatk1.0-data libavahi-client3
  libavahi-common-data libavahi-common3 libblas-common libblas3 libcairo2
  libcups2 libdatrie1 libfontconfig1 libgdk-pixbuf2.0-0
  libgdk-pixbuf2.0-common libgfortran3 libglade2-0 libgraphite2-3
  libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libharfbuzz0b libjbig0
  libjpeg-turbo8 libjpeg8 libjs-jquery libjs-jquery-ui liblapack3
  liblcms2-2 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0
  libpixman-1-0 libtcl8.6 libthai-data libthai0 libtiff5 libtk8.6 libwebp5
  libwebpmux1 libxcb-render0 libxcb-shm0 libxcomposite1 libxcursor1
  libxdamage1 libxfixes3 libxft2 libxi6 libxinerama1 libxrandr2 libxrender1
  libxss1 python-cairo python-cycler python-dateutil python-glade2
  python-gobject-2 python-gtk2 python-imaging python-matplotlib-data
  python-numpy python-pil python-pyparsing python-six python-tk python-tz
  tk8.6-blt2.5 ttf-bitstream-vera x11-common
Suggested packages:
  blt-demo apache2 | lighttpd | httpd cups-common librsvg2-common gvfs
  libjs-jquery-ui-docs liblcms2-utils tcl8.6 tk8.6 python-gtk2-doc
  python-gobject-2-dbg dvipng ffmpeg gir1.2-gtk-3.0 ghostscript inkscape
  ipython python-cairocffi python-configobj python-excelerator
  python-gobject python-matplotlib-doc python-nose python-qt4 python-scipy
  python-sip python-tornado python-traits python-wxgtk3.0
  texlive-extra-utils texlive-latex-extra ttf-staypuft gfortran
  python-numpy-dbg python-numpy-doc python-pil-doc python-pil-dbg tix
  python-tk-dbg
The following NEW packages will be installed:
  blt fontconfig fontconfig-config fonts-lyx hicolor-icon-theme
  javascript-common libatk1.0-0 libatk1.0-data libavahi-client3
  libavahi-common-data libavahi-common3 libblas-common libblas3 libcairo2
  libcups2 libdatrie1 libfontconfig1 libgdk-pixbuf2.0-0
  libgdk-pixbuf2.0-common libgfortran3 libglade2-0 libgraphite2-3
  libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libharfbuzz0b libjbig0
  libjpeg-turbo8 libjpeg8 libjs-jquery libjs-jquery-ui liblapack3
  liblcms2-2 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0
  libpixman-1-0 libtcl8.6 libthai-data libthai0 libtiff5 libtk8.6 libwebp5
  libwebpmux1 libxcb-render0 libxcb-shm0 libxcomposite1 libxcursor1
  libxdamage1 libxfixes3 libxft2 libxi6 libxinerama1 libxrandr2 libxrender1
  libxss1 python-cairo python-cycler python-dateutil python-glade2
  python-gobject-2 python-gtk2 python-imaging python-matplotlib
  python-matplotlib-data python-numpy python-pil python-pyparsing
  python-six python-tk python-tz tk8.6-blt2.5 ttf-bitstream-vera x11-common
0 upgraded, 74 newly installed, 0 to remove and 16 not upgraded.
Need to get 20.0 MB of archives.
After this operation, 74.4 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

(snip: the install was success)
 
root@u6:~/openr/build# breeze kvstore topology
Traceback (most recent call last):
  File "/usr/local/bin/breeze", line 9, in <module>
    load_entry_point('py-openr==1.0', 'console_scripts', 'breeze')()
  File "build/bdist.linux-x86_64/egg/openr/cli/breeze.py", line 77, in main
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click-6.7-py2.7.egg/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "build/bdist.linux-x86_64/egg/openr/cli/clis/kvstore.py", line 258, in topology
  File "build/bdist.linux-x86_64/egg/openr/cli/commands/kvstore.py", line 517, in run
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 534, in figure
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/backend_bases.py", line 170, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/usr/local/lib/python2.7/dist-packages/matplotlib/backends/backend_tkagg.py", line 1049, in new_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1818, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

I imagine that a reason of the issue is that my enviroment is only CUI (not GUI)...
Many thanks.

facebook-github-bot pushed a commit that referenced this issue Feb 13, 2020
)

Summary:
Take the README.md from
https://github.com/facebookexperimental/mononoke/blob/7ead0e29e41aec0771531a4650b6170bc1ff6566/README.md
and apply it on Eden repo.

Re-add the Cargo.toml file that declares Cargo workspace.

Re-add fbcode_builder/getdeps manifest for Mononoke
Pull Request resolved: facebook/sapling#13

Test Plan:
./build/fbcode_builder/getdeps.py build mononoke
  ./build/fbcode_builder/getdeps.py test mononoke

Reviewed By: ahornby

Differential Revision: D19833059

Pulled By: lukaspiatkowski

fbshipit-source-id: fb37e13306c0b9969a7c4e52b05e1a66a577022f
facebook-github-bot pushed a commit that referenced this issue Feb 28, 2020
Summary:
There is a race condition when KvStoreWrapper and ZMQ background threads are
destroyed. This diff is an attempt to explicitly destroy KvStoreWrapper in TearDown

Also use unique_ptr instead of shared_ptr

```
WARNING: ThreadSanitizer: data race (pid=49594)
  Write of size 8 at 0x7ba000000220 by thread T1:
    #0 close <null> (link_monitor_test+0xdb1f6a)
    #1 zmq::signaler_t::~signaler_t() /home/engshare/third-party2/zeromq/4.3.1/src/zeromq-4.3.1/src/signaler.cpp:114:20 (link_monitor_test+0xbd1274)

  Previous read of size 8 at 0x7ba000000220 by main thread:
    #0 epoll_ctl <null> (link_monitor_test+0xd9bdef)
    #1 epoll_del /home/engshare/third-party2/libevent/1.4.14b_hphp/src/libevent-1.4.14b-stable/epoll.c:485 (link_monitor_test+0xec4253)
    #2 folly::EventBaseEvent::eb_event_del() <null> (link_monitor_test+0xa747ef)
    #3 folly::EventHandler::~EventHandler() <null> (link_monitor_test+0xa74a70)
    #4 std::_Hashtable<int, std::pair<int const, openr::OpenrEventBase::ZmqEventHandler>, std::allocator<std::pair<int const, openr::OpenrEventBase::ZmqEventHandler> >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::~_Hashtable() <null> (link_monitor_test+0x219620)
    #5 openr::OpenrEventBase::~OpenrEventBase() <null> (link_monitor_test+0x212d8d)
    #6 openr::KvStore::~KvStore() <null> (link_monitor_test+0x1bfbb7)
    #7 openr::KvStore::~KvStore() <null> (link_monitor_test+0x1bfbe9)
    #8 openr::KvStoreWrapper::~KvStoreWrapper() <null> (link_monitor_test+0x117533)
    #9 std::_Sp_counted_ptr_inplace<openr::KvStoreWrapper, std::allocator<openr::KvStoreWrapper>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() <null> (link_monitor_test+0x116fbd)
    #10 LinkMonitorTestFixture::~LinkMonitorTestFixture() <null> (link_monitor_test+0xf8285)
    #11 LinkMonitorTestFixture_BasicOperation_Test::~LinkMonitorTestFixture_BasicOperation_Test() <null> (link_monitor_test+0xf7ee9)
    #12 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) <null> (link_monitor_test+0xc4b2ae)
    #13 __libc_start_main /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/csu/libc-start.c:308:16 (libc.so.6+0x211a5)
```

Reviewed By: yi-xian

Differential Revision: D20148667

fbshipit-source-id: 71634c08df8ebbc98a6b8c4aa3329166834453f2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants