Skip to content

Commit

Permalink
Fixing github tests, cmake files, readme, ...
Browse files Browse the repository at this point in the history
  • Loading branch information
krulis-martin committed Aug 19, 2023
1 parent 60a9bd0 commit 3ed3762
Show file tree
Hide file tree
Showing 8 changed files with 32 additions and 38 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/linux-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on: [push, pull_request]

jobs:
tests:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v2
with:
Expand All @@ -30,7 +30,7 @@ jobs:
run: sudo apt-get install -y libcap-dev

# install isolate
- run: git clone https://github.com/ioi/isolate.git
- run: git clone https://github.com/ReCodEx/isolate
- run: make isolate
working-directory: isolate
- run: sudo make install
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/windows-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ jobs:
- name: Install dependencies and prepare directory structure for cmake
working-directory: libs
run: |
nuget install boost -ExcludeVersion -Version 1.72.0
nuget install boost -ExcludeVersion -Version 1.74.0
mv -Force ./boost/lib/native/include/ ./boost/
nuget install boost_system-vc142 -ExcludeVersion -Version 1.72.0
nuget install boost_system-vc142 -ExcludeVersion -Version 1.74.0
mv -Force ./boost_system-vc142/lib/native/* ./boost/lib/
nuget install boost_program_options-vc142 -ExcludeVersion -Version 1.72.0
nuget install boost_program_options-vc142 -ExcludeVersion -Version 1.74.0
mv -Force ./boost_program_options-vc142/lib/native/* ./boost/lib/
nuget install rmt_zlib -ExcludeVersion -Version 1.2.8.7
Expand Down
9 changes: 3 additions & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
cmake_minimum_required(VERSION 3.11.0)
project(recodex-worker)
set(RECODEX_VERSION 1.7.2)
set(RECODEX_VERSION 1.8.0)
enable_testing()

# Hack for old CentOS and new GCC with uncompatible ABI
# add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)

set(EXEC_NAME ${PROJECT_NAME})
set(gtest_force_shared_crt ON CACHE BOOL "Use shared (DLL) run-time lib even when Google Test is built as static lib.")

# Find installed packages on system
# Find installed packages on the system
# -- LibArchive
find_package(LibArchive)

Expand All @@ -21,7 +18,7 @@ else()
endif()
# -- load Boost
set (Boost_USE_MULTITHREADED ON)
find_package(Boost 1.66.0 REQUIRED COMPONENTS filesystem system program_options)
find_package(Boost 1.74.0 REQUIRED COMPONENTS system program_options)
include_directories(${Boost_INCLUDE_DIRS})

# -- cURL
Expand Down
41 changes: 19 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,39 +10,37 @@
[![COPR](https://copr.fedorainfracloud.org/coprs/semai/ReCodEx/package/recodex-worker/status_image/last_build.png)](https://copr.fedorainfracloud.org/coprs/semai/ReCodEx/)

The job of the worker is to securely execute a job according to its
configuration and upload results back for latter processing. After receiving an
evaluation request, worker has to do following:
configuration and upload results back for later processing. After receiving an
evaluation request, a worker has to do the following:

- download the archive containing submitted source files and configuration file
- download the archive containing the submitted source files and configuration file
- download any supplementary files based on the configuration file, such as test
inputs or helper programs (this is done on demand, using a `fetch` command
in the assignment configuration)
- evaluate the submission according to job configuration
- during evaluation progress messages can be sent back to broker
- evaluate the submission according to the job configuration
- during evaluation progress messages can be sent back to the broker
- upload the results of the evaluation to the fileserver
- notify broker that the evaluation finished
- notify the broker that the evaluation finished

## Installation

### COPR Installation

Follows description for CentOS which will do all steps as described in _Manual Installation_.
Follows the description for RHEL-like systems which will do all steps as described in _Manual Installation_.

```
# yum install yum-plugin-copr
# yum copr enable semai/ReCodEx
# yum install recodex-worker
# dnf install dnf-plugin-copr
# dnf copr enable semai/ReCodEx
# dnf install recodex-worker
```

### Manual Installation

#### Dependencies

Worker specific requirements are written in this section. It covers only basic
Worker-specific requirements are written in this section. It covers only basic
requirements, additional runtimes or tools may be needed depending on type of
use. The package names are for CentOS if not specified otherwise.
use. The package names are for CentOS if not specified otherwise.

- Boost 1.70 development libs (`boost-devel` package)
- Boost 1.74 development libs or newer (`boost-devel` package, or `libboost-all-dev` on Debian/Ubuntu)
- ZeroMQ in version at least 4.0, packages `zeromq` and `zeromq-devel`
(`libzmq3-dev` on Debian)
- YAML-CPP library, `yaml-cpp` and `yaml-cpp-devel` (`libyaml-cpp0.5v5` and
Expand All @@ -55,21 +53,20 @@ use. The package names are for CentOS if not specified otherwise.

**Isolate** (only for Linux installations)

First, we need to compile sandbox Isolate from source and install it. Current
worker is tested against version 1.3, so this version needs to be checked out.
Assume that we keep source code in `/opt/src` dir. For building man page you
First, we need to compile sandbox Isolate from the source and install it. The current
worker is tested against our local clone which has some patches in it, so this version needs to be checked out.
Assume that we keep the source code in `/opt/src` dir. For building man page you
need to have package `asciidoc` installed.

```
$ cd /opt/src
$ git clone https://github.com/ioi/isolate.git
$ git clone https://github.com/ReCodEx/isolate
$ cd isolate
$ git checkout v1.3
$ make
# make install && make install-doc
```

For proper work Isolate depends on several advanced features of the Linux
For proper work, Isolate depends on several advanced features of the Linux
kernel. Make sure that your kernel is compiled with `CONFIG_PID_NS`,
`CONFIG_IPC_NS`, `CONFIG_NET_NS`, `CONFIG_CPUSETS`, `CONFIG_CGROUP_CPUACCT`,
`CONFIG_MEMCG`. If your machine has swap enabled, also check
Expand Down Expand Up @@ -107,7 +104,7 @@ worker source codes.
that `rpm` and `deb` packages are build in the same time. You may need to have
`rpmbuild` command (usually as `rpmbuild` or `rpm` package) or edit
CPACK_GENERATOR variable in _CMakeLists.txt_ file in root of source code tree.
- Install generated package through your package manager (`yum`, `dnf`, `dpkg`).
- Install generated package through your package manager (`dnf`, `dnf`, `dpkg`).

The worker installation process is composed of following steps:

Expand Down
2 changes: 1 addition & 1 deletion src/archives/archivator.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ void archivator::compress(const std::string &dir, const std::string &destination
// find out where the two paths diverge - std::filesystem::relative() is too new now to use it
fs::path::const_iterator itr_dir = dir_path.begin();
fs::path::const_iterator itr_file = file.begin();
while (*itr_dir == *itr_file && itr_dir != dir_path.end()) {
while (itr_dir != dir_path.end() && itr_file != file.end() && *itr_dir == *itr_file) {
++itr_dir;
++itr_file;
}
Expand Down
2 changes: 1 addition & 1 deletion src/fileman/cache_manager.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ void cache_manager::get_file(const std::string &src_name, const std::string &dst
fs::perms::owner_write | fs::perms::group_write | fs::perms::others_write,
fs::perm_options::add);
// change last modification time of the file
fs::last_write_time(source_file, std::filesystem::file_time_type::min());
fs::last_write_time(source_file, fs::file_time_type::clock::now());
} catch (fs::filesystem_error &e) {
auto message = "Failed to copy file '" + source_file.string() + "' to '" + dst_path + "'. Error: " + e.what();
logger_->warn(message);
Expand Down
2 changes: 1 addition & 1 deletion tests/isolate_sandbox.cpp
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#ifndef _WIN32
#ifdef TEST_ISOLATE

#include <gtest/gtest.h>
#include <gmock/gmock.h>
Expand Down
4 changes: 2 additions & 2 deletions tests/job.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -784,8 +784,8 @@ TEST(job_test, job_variables)

auto bnd_dirs = limits->bound_dirs;
ASSERT_EQ(bnd_dirs.size(), 1u);
ASSERT_EQ(path(std::get<0>(bnd_dirs[0])).string(), (temp_directory_path() / "recodex").string());
ASSERT_EQ(path(std::get<1>(bnd_dirs[0])).string(), (dir / "tmp").string());
ASSERT_EQ(path(std::get<0>(bnd_dirs[0])), (temp_directory_path() / "recodex"));
ASSERT_EQ(path(std::get<1>(bnd_dirs[0])), (dir / "tmp"));

// cleanup after yourself
remove_all(dir_root);
Expand Down

0 comments on commit 3ed3762

Please sign in to comment.