Building SLING on OSX

[ringgaard]

We currently only support Linux out-of-the-box. We do try to make SLING build with both GCC and clang, but some changes are needed to get it to build under OSX. We intend to use this issue to track and document problems with building SLING for OSX and solutions for this these. While we do not current intend to check these into the main code base, it can serve as a guide to those who want to use it on OSX anyway.

[ringgaard]

The CASPAR parser trainer rely on the SLING Python API component (pysling.so), which cannot compile on OSX for two primary reasons:

1. The SLING HTTP server uses epoll(7), which is not supported by OSX. The HTTP server is used by the task dashboard for monitoring the SLING task system built into pysling.so.
2. The util/elf-writer component is used for writing ELF files (i.e. .o file). The static web resources used by the web UIs are linked into the binary by putting them into .o files (see the embed_data rule) which are then linked into the binary. OSX uses Mach-O format for object files, which is not currently supported.

You can use the task system and the SLING Python API without the dashboard, but this requires some modifications. You have to remove //sling/task:dashboard and //sling/http:http-server from the build rule for //sling/pyapi. Also, the dashboard functionality needs to be removed from sling/pyapi/pytask.cc.

diff --git a/sling/pyapi/pytask.cc b/sling/pyapi/pytask.cc
index 77b73e7..1f5920b 100644
--- a/sling/pyapi/pytask.cc
+++ b/sling/pyapi/pytask.cc
@@ -14,17 +14,10 @@
 
 #include "sling/pyapi/pytask.h"
 
-#include "sling/http/http-server.h"
-#include "sling/task/dashboard.h"
-
 using namespace sling::task;
 
 namespace sling {
 
-// Task monitor.
-static HTTPServer *http = nullptr;
-static task::Dashboard *dashboard = nullptr;
-
 // Python type declarations.
 PyTypeObject PyJob::type;
 
@@ -170,11 +163,6 @@ PyObject *PyJob::Start() {
     Py_INCREF(this);
     running_ = true;
 
-    // Register job in dashboard.
-    if (dashboard != nullptr) {
-      job_->RegisterMonitor(dashboard);
-    }
-
     // Start job.
     job_->Start();
   }
@@ -276,38 +264,14 @@ PyObject *PyJob::PyAttr(PyObject *obj, const char *name) {
 }
 
 PyObject *StartTaskMonitor(PyObject *self, PyObject *args) {
-  // Get port number.
-  int port;
-  if (!PyArg_ParseTuple(args, "i", &port)) return nullptr;
-
-  // Start HTTP server.
-  bool start_http_server = false;
-  if (http == nullptr) {
-    LOG(INFO) << "Start HTTP server in port " << port;
-    HTTPServerOptions options;
-    http = new HTTPServer(options, port);
-    start_http_server = true;
-  }
-
-  // Start dashboard.
-  if (dashboard == nullptr) {
-    dashboard = new task::Dashboard();
-    dashboard->Register(http);
-  }
-
-  if (start_http_server) http->Start();
-
   Py_RETURN_NONE;
 }
 
 PyObject *GetJobStatistics() {
-  if (dashboard == nullptr) Py_RETURN_NONE;
-  string stats = dashboard->GetStatus();
-  return PyString_FromStringAndSize(stats.data(), stats.size());
+  Py_RETURN_NONE;
 }
 
 PyObject *FinalizeDashboard() {
-  if (dashboard != nullptr) dashboard->Finalize(60);
   Py_RETURN_NONE;
 }

[johndpope]

@ringgaard - you can use this repo https://github.com/sih4sing5hong5/sling_trying_out

Dockerfile

FROM ubuntu:16.04
MAINTAINER sih4sing5hong5

RUN apt-get update # 0529
RUN apt-get install -y python python-pip wget

RUN pip install --upgrade pip
RUN pip install http://www.jbox.dk/sling/sling-1.0.0-cp27-none-linux_x86_64.whl

RUN mkdir /usr/local/sling
WORKDIR /usr/local/sling
RUN wget http://www.jbox.dk/sling/sempar.flow

COPY trying_out.py trying_out.py
RUN echo SLING is a parser for annotating text with frame semantic annotations. | python trying_out.py

trying_out.py

import sling

parser = sling.Parser("sempar.flow")

text = raw_input("text: ")
doc = parser.parse(text)
print doc.frame.data(pretty=True)
for m in doc.mentions:
  print "mention", doc.phrase(m.begin, m.end)

docker build .

Sending build context to Docker daemon 37.98 MB
Step 1 : FROM ubuntu:16.04
 ---> 5e8b97a2a082
Step 2 : MAINTAINER sih4sing5hong5
 ---> Using cache
 ---> d92181c53f70
Step 3 : RUN apt-get update # 0529
 ---> Using cache
 ---> c9bdcbb20882
Step 4 : RUN apt-get install -y python python-pip wget
 ---> Using cache
 ---> d658546ad0e9
Step 5 : RUN pip install --upgrade pip
 ---> Using cache
 ---> 93a352b0c9b1
Step 6 : RUN pip install http://www.jbox.dk/sling/sling-1.0.0-cp27-none-linux_x86_64.whl
 ---> Using cache
 ---> a8794c64ac92
Step 7 : RUN mkdir /usr/local/sling
 ---> Using cache
 ---> 8360aa993e12
Step 8 : WORKDIR /usr/local/sling
 ---> Using cache
 ---> e3f6340268da
Step 9 : COPY sempar.flow sempar.flow
 ---> 5f5ef7e02ff1
Removing intermediate container 0f8b84eba327
Step 10 : COPY trying_out.py trying_out.py
 ---> 68d8b1f5da60
Removing intermediate container 81ab28504570
Step 11 : RUN echo SLING is a parser for annotating text with frame semantic annotations. | python trying_out.py
 ---> Running in 205c6d7ce0a1
text: {=#1
  :/s/document
  /s/document/text: "SLING is a parser for annotating text with frame semantic annotations."
  /s/document/tokens: [{=#2
    :/s/token
    /s/token/index: 0
    /s/token/text: "SLING"
    /s/token/start: 0
    /s/token/length: 5
    /s/token/break: 0
  }, {=#3
    :/s/token
    /s/token/index: 1
    /s/token/text: "is"
    /s/token/start: 6
    /s/token/length: 2
  }, {=#4
    :/s/token
    /s/token/index: 2
    /s/token/text: "a"
    /s/token/start: 9
    /s/token/length: 1
  }, {=#5
    :/s/token
    /s/token/index: 3
    /s/token/text: "parser"
    /s/token/start: 11
    /s/token/length: 6
  }, {=#6
    :/s/token
    /s/token/index: 4
    /s/token/text: "for"
    /s/token/start: 18
    /s/token/length: 3
  }, {=#7
    :/s/token
    /s/token/index: 5
    /s/token/text: "annotating"
    /s/token/start: 22
    /s/token/length: 10
  }, {=#8
    :/s/token
    /s/token/index: 6
    /s/token/text: "text"
    /s/token/start: 33
    /s/token/length: 4
  }, {=#9
    :/s/token
    /s/token/index: 7
    /s/token/text: "with"
    /s/token/start: 38
    /s/token/length: 4
  }, {=#10
    :/s/token
    /s/token/index: 8
    /s/token/text: "frame"
    /s/token/start: 43
    /s/token/length: 5
  }, {=#11
    :/s/token
    /s/token/index: 9
    /s/token/text: "semantic"
    /s/token/start: 49
    /s/token/length: 8
  }, {=#12
    :/s/token
    /s/token/index: 10
    /s/token/text: "annotations"
    /s/token/start: 58
    /s/token/length: 11
  }, {=#13
    :/s/token
    /s/token/index: 11
    /s/token/text: "."
    /s/token/start: 69
    /s/token/length: 1
    /s/token/break: 0
  }]
  /s/document/mention: {=#14
    :/s/phrase
    /s/phrase/begin: 0
    /s/phrase/evokes: {=#15
      :/saft/other
    }
  }
  /s/document/mention: {=#16
    :/s/phrase
    /s/phrase/begin: 1
    /s/phrase/evokes: {=#17
      :/pb/be-01
      /pb/arg1: #15
      /pb/arg2: {=#18
        :/saft/other
      }
    }
  }
  /s/document/mention: {=#19
    :/s/phrase
    /s/phrase/begin: 3
    /s/phrase/evokes: #18
  }
  /s/document/mention: {=#20
    :/s/phrase
    /s/phrase/begin: 6
    /s/phrase/evokes: {=#21
      :/saft/art
    }
  }
  /s/document/mention: {=#22
    :/s/phrase
    /s/phrase/begin: 8
    /s/phrase/length: 3
    /s/phrase/evokes: {=#23
      :/saft/other
    }
  }
}
mention SLING
mention is
mention parser
mention text
mention frame semantic annotations
 ---> aa87077a3aeb
Removing intermediate container 205c6d7ce0a1
Successfully built aa87077a3aeb

[ringgaard]

@johndpope: Can this Docker container run on OSX? If so, this is a simple way to run SLING under OSX.

[johndpope]

yes. here's another Dockerfile that could be merged which exposes drive to host via ssh so you can edit python code with osx / vscode.
/usr/bin/supervisord
https://gist.github.com/johndpope/b8a1f334e1d1769e73391ad6270bbff9

[johndpope]

found this now - it will mount volumes
https://github.com/tensorflow/models/tree/master/research/syntaxnet/docker-devel

[johndpope]

we can use the __ double underscore APPLE double underscore ___ compiler director to bypass code

USE CASPER BRANCH
https://gist.github.com/johndpope/f1ee8995094063e13db636bf31c21ce0

eg. #ifdef APPLE
#define DISABLE_DASHBOARD
#endif

#ifndef DISABLE_DASHBOARD
#include "sling/http/http-server.h"
#include "sling/task/dashboard.h"
#endif

[johndpope]

then bazel build -c opt sling/pyapi:pysling.so

[johndpope]

[johndpope]

not sure how to get this to work on mac. I wonder if it's possible to use once compiled without specifically installing the wheel? I looked at buildWheel script - and doesn't seem like there's anything specific to linux. (except the name ofcourse)