SEDE, short for Service Execution and Distribution Environment, provides a service-oriented solution to encapsulate software components and offer their functionality through a communication protocol over a network.
Clone this repository and export SEDE jar files.
cd SEDE_ROOT_DIRECTORY
./gradlew jarjar
tree deploy/SEDE
deploy/SEDE
>> ├── SEDE.core-0.0.3.jar
>> ├── SEDE.executor-0.0.3.jar
>> ├── SEDE.gateway-0.0.3.jar
>> ├── SEDE.http-0.0.3.jar
>> ...
One-liner (also works on Windows OS):
# deploy/SEDE/ contains the SEDE jars.
# PATH_TO_LOG_CONFIG is folder that contains log4j2 config.
# The first arguemnt is the port to be used, e.g.: 6000
# The remaining arguments are enterpreted as file paths to class and type-configurations:
java -cp deploy/SEDE/'*':PATH_TO_LOG_CONFIG \
de.upb.sede.gateway.GatewayServerStarter \
6000 \
deploy/run-gateway-template/configs/* \ # this folder contains all service and type configurations offered by this project.
Alternatively, one can create a run folder for a specific gateway by copying the template run folder:
cd SEDE_ROOT_DIRECTORY/deploy
cp -r run-gateway-template my-gateway
tree my-gateway/
>> my-gateway/
>> ├── configs
>> │ ├── demo-classconf.json
>> │ └── demo-typeconf.json
>> └── run.sh
Remove the demonstration class- and type-configurations and replace them with the configuration files of the services that should be supported:
rm my-gateway/configs/*
cp PATH_TO_CONFIGS/* my-gateway/configs/
tree my-gateway/
>> my-gateway/
>> ├── configs
>> │ ├── a-classconf.json
>> │ ├── a-typeconf.json
>> │ ├── b-classconf.json
>> │ └── b-typeconf.json
>> └── run.sh
Optionally, place a Log4j2-configuration file in the run directory:
cp ../SEDE.core/src/test/resources/log4j2.xml my-gateway/
Run the gateway using the run.sh
script:
bash my-gateway/run.sh
One-liner (also works on Windows OS):
# deploy/SEDE/ contains the SEDE jars.
# PATH_TO_LOG_CONFIG is folder that contains log4j2 config.
# Inputs:
# executor-config.json is the path to the executor configuration
# The machine's host address is: 100.90.80.70
# The port opened is: 9000
java -cp deploy/SEDE/'*':path-to-services/'*':PATH_TO_LOG_CONFIG \
de.upb.sede.exec.ExecutorServerStarter \
executor-config.json \
100.90.80.70 \
9000
One can also create a dedicated executor run directory, similarly to how the gateway run directory was created. (Copy and extend the deploy/run-executor-template
folder instead).
Run ./gradlew buildPython
to build the deploy/SEDE_python
directory. Each inner folder inside this directory contains the sources of a python sub-project from SEDE in addition to it's setup.py
file.
Install the python executor using pip
:
pip install deploy/SEDE_python/SEDE.executor/
After this the console command pyexecutor
is ready. This command takes 3 arguments:
- Path to the executor configuration file, like this one: config.json
- The host machine's address the executor is running on. It is important to note that other executors need to reach this executor when they use this address.
- The tcp-port that is opened by the http server.
An example:
pyexecutor config_path/config.json localhost 5000
Before running the executor though, make sure the services you want to use are also installed. For example the sub-project scikit.ml
offers algorithm from the scikit-learn library.
The following will install the scikit.ml
project in addition to its dependencies like scikit-learn
:
pip install deploy/SEDE_python/scikit.ml/
To create Docker SEDE images execute: ./gradlew buildDocker
. This gradle task assumes that the dockerd
daemon is running because it runs docker build
commands.
After completion run docker images
to confirm the newly created images:
REPOSITORY TAG IMAGE ID
sede Gateway 35a6812ee54b
sede pyExecutor c58e0aecc278
sede jExecutor e89adb7b7a10
sede java 4593ff808896
openjdk 8 ba56d4f23853
python 3.6-stretch d49c41b6e6c4
You can now run the start-up commands from above to run a SEDE component inside a container. The following example illustrates this for the gateway server:
$ ./gradlew buildDocker
# -p 6000:8080 Publish the gateway᾿s port 8080 to the host's port 6000.
# -p 6001:2200 Publish the telnet server port 2200 to host's port 6001.
# -dt detach (run container in background) and tty (Allocate a pseudo-TTY)
$ docker run \
-p 6000:8080 \
-p 6001:2200 \
-dt sede:Gateway \
java \
-cp SEDE/'*':. \
de.upb.sede.gateway.GatewayServerStarter \
8080 \
/configs # class and type configuration files.
>> ed9720a1d7c86bf1c47097d3169eea04e1e523917ca105a9e955ba49ff69cc84
# connect to the gateway command shell with telnet:
$ telnet 127.0.0.1 6001
>> Enter command: services ls
>> Known services are:
>> Catalano.Imaging.Filters.CannyEdgeDetector
>> Catalano.Imaging.Filters.CannyEdgeDetectorFactory
>> Catalano.Imaging.Filters.Crop
>> Catalano.Imaging.Filters.GrayScale
>> Catalano.Imaging.Filters.GrayScaleFactory
>> ...
>> Enter command: exit session
>> Closing session...
TODO
For more examples and usage, please refer to the wiki. TODO
SEDE uses Java 8 and Python 3.5 +. We use Eclipse, IntelliJ IDEA and PyCharm to work on SEDE. These sections show how to setup each development environment.
Import the gradle project:
- File > Import...
- Gradle > Existing Gradle Project
- Choose the path to your
SEDE_ROOT_DIRECTORY
as the Project root directory - Finish
There is now a set of errors due to cyclic dependencies between gradle sub-projects. In reality SEDE only contains cyclic dependencies between test modules of sub-projects, so this error is easily fixed by configuring Eclipse:
- Preferences
- Java > Compiler > Building
- Under Build path problems set Circular dependencies to
Warning
.
Now select (right-click) the 'SEDE' project and choose Refresh gradle project
under Gradle.
Import the gradle project:
- File > New > Project from existing sources...
- Choose your
SEDE_ROOT_DIRECTORY
. - Import project from external model > Gradle
- Finish
- In the Gradle view click the Refresh all Gradle projects button.
When developing the python modules in SEDE install each python project using pip with editable mode (i.e. setuptools "develop mode") enabled:
pip install -e SEDE.executor/src/main/python/
pip install -e SEDE.services/scikit.ml/src/main/python/
The usual procedure for contribution to open source projects:
- Fork it
- Create your feature branch (
git checkout -b feature/fooBar
) - Commit your changes (
git commit -am 'Add some fooBar'
) - Push to the branch (
git push origin feature/fooBar
) - Create a new Pull Request