Skip to content

Commit

Permalink
Juke Build Reforged (#4861)
Browse files Browse the repository at this point in the history
* # This is a combination of 7 commits.
# This is the 1st commit message:

Revert "z"

This reverts commit 3c2b584aa7803965e56be472463a508ff1f90932.

# This is the commit message #2:

Revert "jesus im dum"

This reverts commit 079e7c6b3bb9e4b6bad58d663a804011cd1cebb0.

# This is the commit message #3:

text

# This is the commit message #4:

Revert "ccc"

This reverts commit e7ef0248502f0abeb68749abc73440494b52b9f0.

# This is the commit message #5:

Revert "prec1s"

This reverts commit e217e183d551bdf3e5130c644590b59ec7b36882.

# This is the commit message #6:

Revert "reverttoclassic"

This reverts commit 591e333742d9507bd86e741fefe055120886a395.

# This is the commit message #7:

Revert "v"

This reverts commit bbd3c05ae182bfbb934899aabc25eb93b490bcda.

* Readds the old CBT

* Juke Build (#59390)

* Juke Build Hotfix 1 (#59643)

* Juke Build Fix

* More fixes

* Juke Build Hotfix 2 - PreCompile script compatibility (#59649)

* Juke Build Hotfix 2 - PreCompile script compatibility

* Pass arguments from bat to build.js

* Pass arguments in BUILD.bat as well

* Quick tweak

* Juke Build 0.4.0 (#59796)

Updated Juke Build to version 0.4.0
Main new feature: dependsOn, inputs, outputs, and onlyWhen fields now all accept a function of format ({ get }) => ..., and all of them can be async. This means that you can easily parametrize every part of the build.
Added references to Juke Build to README in the build folder

* Stop compiling DM if compiler outputs are locked. (#60022)

Basically, saves developer's time by yelling that the compiler can't write to dmb/rsc, because they are locked by Dream Daemon.

Added myself as a code owner for /tools/build.

* Spring Cleaning

* Adapt CI to work with this.

* Update continuous_integration, no more ENV:

* merge mistake

* Clear out dist

A mistake made when removing CBT from the old PR, this folder was meant to be untracked and became tracked.

* Docker experiment. Not done. Too lazy to stash

Sorry ~

* Possibly temporary use of tgstation's docker image

* Actually grab auxmos btw, bee's dockerfile doesn't

* Quickfix for Linux Auxtools

Co-authored-by: Aleksej Komarov <stylemistake@gmail.com>
  • Loading branch information
Sinestia and stylemistake committed Aug 7, 2021
1 parent 34da7bc commit 261160d
Show file tree
Hide file tree
Showing 49 changed files with 8,296 additions and 423 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,4 @@ tgstation.lk
tgstation.dyn.rsc
*.dll
Dockerfile
tools/bootstrap/.cache
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@ indent_style = space
[/tgui/**/*.{js,styl,ract,json,html}]
indent_style = space
indent_size = 2

[Dockerfile]
indent_style = space
12 changes: 7 additions & 5 deletions .github/workflows/continuous_integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Install Tools
run: |
pip3 install setuptools
bash tools/ci/install_build_tools.sh
bash tools/ci/install_node.sh
bash tools/ci/install_auxmos.sh
pip3 install -r tools/mapmerge2/requirements.txt
- name: Run Linters
Expand All @@ -38,7 +38,8 @@ jobs:
bash tools/ci/check_changelogs.sh
find . -name "*.php" -print0 | xargs -0 -n1 php -l
find . -name "*.json" -not -path "*/node_modules/*" -print0 | xargs -0 python3 ./tools/json_verifier.py
bash tools/ci/build_tgui.sh
tgui/bin/tgui --lint
tgui/bin/tgui --test
bash tools/ci/check_grep.sh
python3 tools/mapmerge2/dmi.py --test
~/dreamchecker
Expand All @@ -59,7 +60,7 @@ jobs:
run: |
source $HOME/BYOND/byond/bin/byondsetup
python3 tools/ci/template_dm_generator.py
bash tools/ci/dm.sh -DCIBUILDING -DCITESTING -DALL_MAPS beestation.dme
tools/build/build dm -DCIBUILDING -DCITESTING -DALL_MAPS
run_all_tests:
runs-on: ubuntu-latest
services:
Expand Down Expand Up @@ -98,11 +99,12 @@ jobs:
- name: Compile and run tests
run: |
source $HOME/BYOND/byond/bin/byondsetup
bash tools/ci/dm.sh -DCIBUILDING beestation.dme
bash tools/ci/run_server.sh
tools/build/build -DCIBUILDING
test_windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- name: Compile
run: pwsh tools/ci/build.ps1
env:
DM_EXE: "C:\\byond\\bin\\dm.exe"
56 changes: 11 additions & 45 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@
#Ignore byond config folder.
/cfg/**/*

# Ignore compiled linux libs in the root folder, e.g. librust_g.so
/*.so

#Ignore compiled files and other files generated during compilation.
*.mdme
*.dmb
Expand Down Expand Up @@ -49,36 +52,14 @@ __pycache__/
*.py[cod]
*$py.class

# C extensions
#*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt
pip-*.txt

# Unit test / coverage reports
htmlcov/
Expand All @@ -88,7 +69,6 @@ htmlcov/
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/

# Translations
Expand All @@ -97,20 +77,13 @@ coverage.xml

# Django stuff:
*.log
local_settings.py

# Flask instance folder
instance/

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# IPython Notebook
.ipynb_checkpoints

Expand All @@ -123,10 +96,6 @@ celerybeat-schedule
# dotenv
.env

# virtualenv
venv/
ENV/

# IntelliJ IDEA / PyCharm (with plugin)
.idea

Expand All @@ -149,12 +118,6 @@ Desktop.ini
# Recycle Bin used on file shares
$RECYCLE.BIN/

# Windows Installer files
#*.cab
#*.msi
#*.msm
#*.msp

# Windows shortcuts
*.lnk

Expand Down Expand Up @@ -195,10 +158,10 @@ Temporary Items

#Visual studio stuff
*.vscode/*
!/.vscode/extensions.json
!/.vscode/settings.json
tools/MapAtmosFixer/MapAtmosFixer/obj/*
tools/MapAtmosFixer/MapAtmosFixer/bin/*
/tools/MapAtmosFixer/MapAtmosFixer/obj/*
/tools/MapAtmosFixer/MapAtmosFixer/bin/*
/tools/CreditsTool/bin/*
/tools/CreditsTool/obj/*

#GitHub Atom
.atom-build.json
Expand All @@ -216,3 +179,6 @@ tools/MapAtmosFixer/MapAtmosFixer/bin/*
!/config/jukebox_music/sounds/exclude
/config/title_music/sounds/*
!/config/title_music/sounds/exclude

# Common build tooling, C B T
!/tools/build
4 changes: 4 additions & 0 deletions .tgs4.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
linux_scripts:
PreCompile.sh: tools/tgs4_scripts/PreCompile.sh
windows_scripts:
PreCompile.bat: tools/tgs4_scripts/PreCompile.bat
12 changes: 12 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "byond",
"request": "launch",
"name": "Launch DreamSeeker",
"preLaunchTask": "Build All",
"dmb": "${workspaceFolder}/${command:CurrentDMB}"
}
]
}
55 changes: 55 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"version": "2.0.0",
"tasks": [
{
"type": "process",
"command": "tools/build/build",
"windows": {
"command": ".\\tools\\build\\build.bat"
},
"options": {
"env": {
"DM_EXE": "${config:dreammaker.byondPath}"
}
},
"problemMatcher": [
"$dreammaker",
"$tsc",
"$eslint-stylish"
],
"group": {
"kind": "build",
"isDefault": true
},
"dependsOn": "dm: reparse",
"label": "Build All"
},
{
"type": "dreammaker",
"dme": "beestation.dme",
"problemMatcher": [
"$dreammaker"
],
"group": "build",
"label": "dm: build - beestation.dme"
},
{
"type": "shell",
"command": "tgui/bin/tgui",
"windows": {
"command": ".\\tgui\\bin\\tgui.bat"
},
"problemMatcher": [
"$tsc",
"$eslint-stylish"
],
"group": "build",
"label": "tgui: build"
},
{
"command": "${command:dreammaker.reparse}",
"group": "build",
"label": "dm: reparse"
}
]
}
3 changes: 3 additions & 0 deletions BUILD.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
@echo off
call "%~dp0\tools\build\build.bat" %*
pause
115 changes: 67 additions & 48 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,72 +1,91 @@
FROM beestation/byond:514.1554 as base
ONBUILD ENV BYOND_MAJOR=514
ONBUILD ENV BYOND_MINOR=1554
# base = ubuntu + full apt update
FROM ubuntu:xenial AS base

FROM base as build_base

RUN apt-get update \
RUN dpkg --add-architecture i386 \
&& apt-get update \
&& apt-get upgrade -y \
&& apt-get dist-upgrade -y \
&& apt-get install -y --no-install-recommends \
git \
dos2unix\
ca-certificates
ca-certificates

FROM build_base as rust_g

WORKDIR /rust_g
# byond = base + byond installed globally
FROM base AS byond
WORKDIR /byond

RUN apt-get install -y --no-install-recommends \
libssl-dev \
pkg-config \
curl \
gcc-multilib \
&& curl https://sh.rustup.rs -sSf | sh -s -- -y --default-host i686-unknown-linux-gnu \
&& git init \
&& git remote add origin https://github.com/BeeStation/rust-g
curl \
unzip \
make \
libstdc++6:i386

COPY dependencies.sh .

RUN dos2unix dependencies.sh \
&& /bin/bash -c "source dependencies.sh \
&& git fetch --depth 1 origin \$RUST_G_VERSION" \
&& git checkout FETCH_HEAD \
&& ~/.cargo/bin/cargo build --release --all-features \
&& apt-get --purge remove -y dos2unix

FROM base as dm_base

RUN . ./dependencies.sh \
&& curl "http://www.byond.com/download/build/${BYOND_MAJOR}/${BYOND_MAJOR}.${BYOND_MINOR}_byond_linux.zip" -o byond.zip \
&& unzip byond.zip \
&& cd byond \
&& sed -i 's|install:|&\n\tmkdir -p $(MAN_DIR)/man6|' Makefile \
&& make install \
&& chmod 644 /usr/local/byond/man/man6/* \
&& apt-get purge -y --auto-remove curl unzip make \
&& cd .. \
&& rm -rf byond byond.zip

# build = byond + beestation compiled and deployed to /deploy. Hijacking it for auxtools to prevent making another layer.
FROM byond AS build
WORKDIR /beestation

FROM dm_base as build
RUN apt-get install -y --no-install-recommends \
curl

COPY . .

RUN apt-get update \
&& apt-get install -y --no-install-recommends dos2unix \
&& rm -rf /var/lib/apt/lists/* \
&& DreamMaker -max_errors 0 beestation.dme && dos2unix tools/deploy.sh && tools/deploy.sh /deploy
RUN env TG_BOOTSTRAP_NODE_LINUX=1 tools/build/build \
&& tools/deploy.sh /deploy \
&& cd auxtools \
&& curl -O "https://github.com/BeeStation/auxmos/releases/download/${AUXMOS_VERSION}/libauxmos.so" \
&& chmod +x libauxmos.so

FROM dm_base
# rust = base + rustc and i686 target
FROM base AS rust
RUN apt-get install -y --no-install-recommends \
curl && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal \
&& ~/.cargo/bin/rustup target add i686-unknown-linux-gnu

EXPOSE 1337
# rust_g = base + rust_g compiled to /rust_g
FROM rust AS rust_g
WORKDIR /rust_g

RUN apt-get update \
&& apt-get install -y --no-install-recommends software-properties-common \
&& add-apt-repository ppa:ubuntu-toolchain-r/test \
&& apt-get update \
&& apt-get upgrade -y \
&& apt-get dist-upgrade -y \
&& apt-get install -y --no-install-recommends \
mariadb-client \
libssl1.0.0 \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /root/.byond/bin
RUN apt-get install -y --no-install-recommends \
pkg-config:i386 \
libssl-dev:i386 \
gcc-multilib \
git \
&& git init \
&& git remote add origin https://github.com/beestation/rust-g

COPY dependencies.sh .

RUN . ./dependencies.sh \
&& git fetch --depth 1 origin "${RUST_G_VERSION}" \
&& git checkout FETCH_HEAD \
&& env PKG_CONFIG_ALLOW_CROSS=1 ~/.cargo/bin/cargo build --release --all-features --target i686-unknown-linux-gnu

# final = byond + runtime deps + rust_g + build
FROM byond
WORKDIR /beestation

RUN apt-get install -y --no-install-recommends \
libssl1.0.0:i386 \
zlib1g:i386

COPY --from=rust_g /rust_g/target/release/librust_g.so /root/.byond/bin/rust_g
COPY --from=build /deploy ./
COPY --from=rust_g /rust_g/target/i686-unknown-linux-gnu/release/librust_g.so /root/.byond/bin/rust_g

#auxtools fexists memes
RUN ln -s /beestation/auxtools/libauxmos.so /root/.byond/bin/libauxmos.so

VOLUME [ "/beestation/config", "/beestation/data" ]

ENTRYPOINT [ "DreamDaemon", "beestation.dmb", "-port", "1337", "-trusted", "-close", "-verbose" ]
EXPOSE 1337
Loading

0 comments on commit 261160d

Please sign in to comment.