tgstation-server-v4.4.3

Please refer to the README for setup instructions.

Component Versions

Core: 4.4.3
Configuration: 2.0.0
HTTP API: 7.1.0
DreamMaker API: 5.2.3
Web Control Panel: 0.4.0
Host Watchdog: 1.1.0

The recommended client is currently the legacy Tgstation.Server.ControlPanel. This will be phased out as the web client is completed.

Patch 3

Core

• Fixed requests failing if instance online state did not match with the database. (#1096)
• Fixed sending invalid revision information over the initial bridge. (#1096)
• Fixed a rare watchdog issue that came from incorrectly measuring the time it took for DreamDaemon to become responsive. (#1096)
• If all test merges have been merged on remote at the time of an auto-update, a hard reset will be performed. Previously, a custom merge commit would be created. (#1096)
• Improved tracking of most recent origin commit when performing local merges. (#1096)
• Fixed EventNotification type 0 not being serialized over topic requests. (#1096)

HTTP API

• Entries returned from GET /Administration/Logs are now returned from newest to oldest. (#1096)

Patch 2

Core

• Fixed a mountain of potential race conditions involving instance online state and API requests. (#1075)
• Fixed an rare issue where a reattach startup would terminate DreamDaemon if the server could not bind to the API port. (#1075)
• Fixed a rare NullReferenceException when reattaching. (#1075)
• Fixed an potential issue regarding DreamDaemon log reading on Windows. (#1075)
• Instance move jobs now only rename the directory. (#1075)
• Fixed an issue where repository deletion could be cancelled if the request to start the job was aborted right as the response came in. (#1075)
• Creating instances no longer uppercases the path on disk on Windows (Known issue #1074). (#1075)
• Jobs will no longer start until instance manager startup is complete. (#1075)
• Fixed disabled chat bots connecting when their instance onlined. (#1075)
• Fixed an issue that manifested as the instance's chat bot being online and non-responsive. (#1075)
• Fixed the configuration option General:RestartTimeout not being respected in some cases. (#1075)
• Fixed a rare issue with the chat message handler crashing if messages were processed to quickly. (#1075)
• Fixed an issue where process suspension failures would leak thread handles on Windows. (#1075)
• Fixed merged pull requests continuing to appear in server metadata after an automatic update. (#1075)
• Fixed being able to download any file on the system with the log downloading endpoint. (#1075)
• Fixed a potential InvalidOperationException when in process dump jobs that should have been translated as ErrorCode 89. (#1075)

DMAPI

• Fixed an issue where immutable lists could be changed by consumer code. (#1075)
• Fix OnInitializationComplete breaking sleep_offline. (#1087)

HTTP API

• Added a new builtin user "TGS". This user cannot be accessed via the API and is used by the server to create startup/reattach and auto update jobs as well as create the Admin user. (#1075)
• Fixed an issue where ErrorCode 78 could occur for non-socket address in use related errors. (#1075)
• Attempting to access an offline instance now results in a 409 response with the new ErrorCode 90 as opposed to 403. 403 Will still be returned if the user isn't authorized to access that instance. (#1075)
• Added ErrorCode 91 for when chat bots fail to connect. (#1075)
• Fixed issues where an instance could be reported as offline when it was actually online and vice-versa. (#1075)
• Chat provider (re)connection is now done as a job, there is no way to retrieve this job upon creation through the API currently. Though it provides a roundabout way to see why it is failing. (#1075)
• Fixed a rare 500 error on GET /DreamDaemon as the server was crashing. (#1075)

Patch 1

Core

• Fixed BYOND topic timeouts potentially being constantly set to 0 from bad configs. (#1072)
• Fixed an issue where exceptions that crash the server would fail to be re-thrown at the top level. (#1072)
• Fixed a potential NullReferenceException when on-lining instances. (#1072)
• Instance on-lining errors at server startup will no longer prevent the server from starting. (#1072)
• Fixed a rare issue where reattaching would fail instance startup due to an ArgumentNullException. (#1072)

HTTP API

• Instance queries will now always reflect online state as it is in the service. (#1072)

Update 4.X

Configuration

• Configuration is now versioned in the field General:ConfigVersion. Having this up-to-date will prevent warnings during server startup. (#1062)
• The initial config version is 2.0.0. (#1062)
• Future release notes will contain this section to indicate changes to be made to the configuration file. These changes should be performed BEFORE updating to ensure server stability. (#1062)
• PLEASE ENSURE YOU HAVE RENAMED THE FIELD Database:MySqlServerVersion to Database:ServerVersion and it is set correctly. Failure to do so may interfere with migrations.
• Renamed General:UseBasicWatchdogOnWindows to General:UseBasicWatchdog. (#1070)
• The default POSIX log directory has been changed to /var/log/tgstation-server on POSIX and C:/ProgramData/tgstation-server on Windows. This change only affects you if your FileLogging:Directory is set to null. (#1070)
• The option General:UseExperimentalWatchdog has been removed. (#1066)

Core

• Added an event script for when the watchdog launches: WatchdogLaunch. (#1053)
• Fixed watchdog shutdown and reattach event scripts not being invoked. (#1053)
• Improved overall performance by pooling usage of the internal ORM object. (#1049)
• Fixed some inconsistencies with watchdog reattaching. (#1057)
• The experimental watchdog has been removed. (#1066)
• Deployment will no longer duplicate the compiled game as it is no longer necessary. (#1066)
• Added the ability to create process mini/core dumps. These are stored in an instances Diagnostics/ProcessDumpFiles directory. (#1051)
• The gdb package (specifically the gcore executable) is now a POSIX dependency to support creating process dump files. (#1051)
• The setup wizard will now automatically fill in the current General:ConfigVersion. (#1062)
• Aborted HTTP requests that result in database query abortion no longer finish as 409. (#1069)
• GitHub API requests no longer make some requests/jobs take longer to abort than they should. (#1069)
• Added the PosixWatchdog which brings the Windows style seamless updates to Linux. (#1070)
• Fixed deploying .dmbs with a different name as the current .dmb causing errors on Windows. (#1070)
• Monitor crashes, though rare, will no longer unnecessarily reboot DreamDaemon. (#1070)
• Fixed automatic .dme discovery for files not in the repository root. (#1070)
• Cancelled deployments now have a nicer error message in their Discord embeds. (#1070)
• Topic request timeouts are now stored in the database as a per-instance setting. Initial value will be based off the config which will remain as a default value for new instances. (#1055)
• Fixed manually restarting the watchdog sending two redundant chat messages. (#1055)

HTTP API

• Custom BYOND versions may now be installed and used. Upload a zip as a base64 encoded string in the new content field of the Byond model when initially installing. The custom version will have a patch value >= 1 applied to it to indicate that it is not official. Use this version to switch back to it without reuploading. (#1061)
• Repository recursive submodule cloning may now be disabled by explicitly setting recurseSubmodules to false in the PUT request to start the job. (#1050)
• POST /Instance will no longer create/grant instance user permissions on that instance for the current user. (#1052)
• PATCH /Instance/{id} will now give full instance user permissions on the target instance. This uses the new instance manager right 1024. (#1052)
• Added error code 87 for attempting to test merge with an unsupported remote. Currently the only supported remote is GitHub.com. (#1052)
• POST /Chat will now return 204 as opposed to 200 if no data can be returned due to insufficient rights. (#1046)
• Added more response documentation to the OpenAPI spec to be more accurate. (#1046)
• Added a new model, LogFile, for accessing server logs. (#1058)
• GET /Administration/Logs now returns a list of LogFile models without their contents. GET /Administration/Logs/ can be used to retrieve the contents. (#1058)
• Added new Administration right 32 for retrieving LogFiles. (#1058)
• DreamDaemon running field replaced with more verbose status enum. (#1057)
• Deprecated ErrorCode 40. (#1066)
• DreamDaemon field secondaryPort removed and primaryPort has been renamed to port. (#1066)
• Renamed the DreamDaemon right SetPorts to SetPort. (#1066)
• PATCH /DreamDaemon/Diagnostics will start a job to create a process dump. Added DreamDaemon permission 8192 to support this. (#1051)
• Having an incorrect Accept header should now properly return 406 as opposed to 403. (#1069)
• Improved Accept header parsing, any header that is a subtype of application/json will now be accepted. This includes the likes of */* and application/*. (#1069)
• Removed legacy Username/Password header authentication scheme. (#1069)
• All header errors will now be included in the 4XX ErrorMessage.additionalData. (#1069)
• Added the DreamMaker field requireDMApiValidation, defaulting to true. If false deployments that time out on DMAPI validation will still be available for use. This comes with the restriction that all DMAPI functionality, heartbeats, and graceful restarts (including those required by BYOND version switches) will be unavailable for the deployment. Deploying new code with the same BYOND version will still work seamlessly. (#1070)
• Added DreamMaker permission 128 for enabling/disabling the DMAPI requirement for deployments. (#1070)
• Fixed DreamMaker permission 64 not granting access to POST /DreamMaker. (#1070)
• Fixed failed Jobs always returning null for their errorCode. (#1070)
• Error code 54 deprecated in favor of error code 52. (#1070)
• Added DreamDaemon right 16384 for changing the new topicRequestTimeout value in the model. (#1055)

Update 3.X

Core

• The default config value for General:RestartTimeout has been changed from 5s to 60s. It is recommended server operators make this change as well since if it is hit, running DreamDaemon processes may become orphaned during reboot/update operations and must be killed manually before being launched again with TGS. (#1021)
• Added failed query retrying to the MSSQL database backend. (#1021)
• Deployment messages with Discord bots are now delivered via an updating rich embed. (#1019)
• HTTP/2 has been enabled in addition to HTTP/1 for the web server. (#1022)
• The API port has a new configuration location. Please migrate from the legacy Kestrel configuration section to the new General:ApiPort option. (#1022)
• This release includes the framework for supporting PostgresSql as a DB backend. It is currently disabled due to an upstream bug that will be fixed in a few days (npgsql/efcore.pg#1392). (#1029)

HTTP API

• Removed recursion from some API model definitions. (#1026)

Update 2.X

• Added DreamDaemon heartbeats and automatic restarts (Along with associated right). The watchdog will automatically restart if 4 heartbeats are missed. (#975)
• Minor DMAPI update to properly support heartbeats though it is not mandatory. (#975)
• Fixed being unable to suspend and resume processes on POSIX systems. (#975)
• Fixed a crash that could occur if the server was shutdown with watchdogs running waiting on topic requests to complete. (#975)
• Minor C# client patch for moving some API properties across their inheritance tree. (#975)
• Fixed consistency issues surrounding graceful watchdog actions. (#975)
• Attempting to download invalid BYOND versions will no longer create empty directories for them. (#975)
• Fixed being able to make API requests before initial instance startup had completed. (#975)
• Fixed a rare NullReferenceException when launching DreamDaemon. (#977)
• Enabled connection resiliency for MySQL connections. This was causing heavy instability across the server. (#977)
• Fixed a potential issue where chat bots could report the watchdog was starting instead of restarting. (#977)
• Fixed deployment messages showing an extra .0 after the BYOND version. (#977)

RTM Update 1.X

• Most failed requests now come with an error code defined in the API spec. (#920)
• We now independently version several internal components such as the REST and DM APIs. (#920)
• A hard limit of 1000 UTF-8 characters have been added to most strings in models. (#920)
• Improved server-side validation of many models. (#920)
• It's no longer possible to set rights enums to invalidly high values. (#920)
• It's no longer possible to create multiple users with the same system identifier. (#920)
• Fixed chat bot names conflicting with those in separate instances. (#920)
• Added configurable limits for the maximum number of users and instances in the server. (#920)
• The dotnet framework used has been upgraded to .NET Core 3.1. (#927)
• SQLite support has been added, though a full relational database is still the recommended option. (#927)
• Added a startup error if the server detects it cannot create symbolic links (#929)
• Added a startup error if the server detects it cannot create or load git repositories (#929)
• Added configurable limits for the number of chat bots in an instance and the number of channels in a chat bot. There are new permissions for these API fields. (#930)
• Fixed an issue where custom commands could become "de-synced". (#935)
• TGS now uses DMAPI v5.0.0. It has been rewritten to communicate via exclusively via HTTP, /world/Topic(), and world.params. Because of this, the ultrasafe security level is now supported. (#935)
• Reattaching now remembers the launch security level. (#935)
• Fixed the workarounds for trusted mode .dmbs that regressed. (#935)
• CompileJob models now have the received DMAPI version in their dataset. (#935)
• If an instance is renamed, this updated will be reflected in the DMAPI. (#935)
• Added some logging to the script used when starting docker containers. (#936)
• The ServerInformation schema object now includes the minimum password length, instance, and user limits. (#940)
• All version strings returned from the API are now in semantic versioning format. Returned versions in the BYOND model will have their patch number set to zero. (#940)
• You can no longer downgrade to server versions below v4.1.0. (#940)
• Failed job may now include error codes. (#942)
• Fixed not being able to checkout remote branches other than the initially cloned branch. (#943)
• Fixed an issue where checking out a specific SHA in the repository would be a no-op. (#943)
• You can now limit where instances may be installed in the configuration. (#945)
• Detaching an instance will no longer fail due to database integrity issues. (#946)
• Added support for joining IRC channels with keys. (#950)
• Automatic .dme detection is now recursive within a repository. (#956)
• Fixed changing the reboot mode with the basic watchdog preventing staged compile jobs from being applied. (#956)
• On POSIX, BYOND cache cleanup will now target the correct directory. (#956)
• Retrieving compile job details after its job completes will now no longer possibly return out of date data. (#956)
• Fixed issues with receiving BYOND topic responses. (#956)
• Console logging is now asynchronous. This is good for performance. (#956)
• Reduced memory requirements while sending BYOND topics. (#956)

Initial Release

Whats New:

• Agnostic HTTP API: The replaces the WCF service calls used in TGS3. This helps avoid Windows vendor lock-in and get away from the SOAP API that literally no one understood (not even me). With it, it's much easier to expose TGS to the internet, all you need is a HTTPS reverse proxy in front of it. A rundown of the new API exists here: https://tgstation.github.io/tgstation-server/api.html.
• Granular Access Controls: Windows users are no longer (required to be) the basis for authentication to the server. We now have database-backed users as a login option. These use a combined Basic/JWT authentication scheme with industry standard password hashing and salting. Users are fully customizable and can be given granular access to every bit of the server via the new permissions system. From changing the BYOND version, to test merging a PR, to restarting the server, every action may now be granted or revoked on a per user basis.
  • Limitation: Users can be disabled but not deleted.
• Proper Long Running Operation Support: Server actions take a long time, from a git pull to a DreamMaker compile. TGS now internally allows for them to be run in parallel with each other and provides an audit record via the database. This is an improvement over the old system where connections had to be held open for the duration of operations.
• Database Backend: TGS requires an SQL database to operate. This allows for much better concurrency and is just overall much cleaner than the old single json file storage blob per instance.
  • Limitation: There is a one-to-one relationship with a TGS server and a database. DO NOT SHARE TGS DATABASES.
• Linux/Docker Support: TGS4 is Linux and docker compatible. (Note this does not mean that rust-g and BSQL work out of the box, they must be compiled using event scripts like PreCompile.sh).
  • Limitation: TGS4 has a dependency on the native library libgit2 which is known to cause issues on Linux. The binaries distributed with TGS are kept up to date with the upstream repository, but out of the box Linux support can't be assured in every environment. Docker is guaranteed to always work, however. See the repository for the distributed binary here: https://github.com/libgit2/libgit2sharp.nativebinaries.
  • Limitation: System based logins are not supported on Linux. #709
• Incredibly Detailed Logging: Various log levels exist now (Trace/Debug/Info/Warning/Error/Critical) and are sanely output to a rolling file on the host. Significant improvement over having to use the Windows event viewer with TGS3. Until such a point where bugs stop copping up I'd recommend Trace logging for the main log level.
• Historical Deployment Data: Every time code is compiled the following data is logged and stored.
  • The User that initiated it.
  • When it was started.
  • When it finished.
  • All revision information including local/remote SHAs, test merged pull requests and their SHAs.
  • The BYOND version used.
  • The DMAPI version in the binary
• Multiple chat bots per instance: Up to 65535 as a matter of fact (who knows why I chose that number?)!
• Automatic Chat Bot Reconnection Intervals: Set in minutes.
• Better Error State Handling: The Server and Watchdog aren't your momma's boys anymore. Every error state will be automatically resolved or reported with recommended actions.
• Watchdog Heartbeats: An interval in seconds can now be set at which TGS will send /world/Topic() packets to DreamDaemon. If four of these are missed, the server will be rebooted. No more endless @key Holder pings in discord (and I can finally unmute the /tg/ guild)! This feature can be disabled.
• Better DMAPI: No longer requires injecting a .NET runtime .dll into the DreamDaemon process. DD -> TGS communication is now handled securely via BYOND's native /world/Export() API ("But Cyberboss, BYOND only supports GET requests." Who said anything about respecting HTTP standards when dealing with BYOND?).
• Safe/Ultrasafe Security Support: Thanks to the new DMAPI, the ultrasafe and safe security levels may be used without running into BYOND's limitations. But no one really cares...
• Self Upgrading: To upgrade TGS3 you needed to download and run the installer. This was pretty seamless, but it's now even better in V4 as the command to upgrade can be given straight to the API. At that point the server will handle downloading the update, detaching running DreamDaemon instances, restarting with the new version, and reattaching to them. Easier than ever patch delivery.
• Gasp TESTING: TGS4 currently has over 60% code coverage in automated unit and full stack integration tests. I aim to have that number ever increasing to prevent trivial mistakes. Big improvement over V3 which had... literally none...

Along with these features, nearly every single V3 feature has been included and possibly improved in some fashion. This includes stuff like Windows accounts for logins, and using ACLs for static file handling. The following exceptions exist but are planned for future updates:

• Process memory/CPU diagnostic data is not generated: #611
• Process dumps may not be created: #612
• The DMAPI is required in DreamMaker code: #934
• The option to "Initialize Game Directories" is no longer present, but a variant is still performed every deployment for smoother error handling.
• Direct server announcements are no longer present but may be readded upon request.

Note that V3 instances cannot be imported into TGS4. This doesn't prevent the transfer of static game files however.

TGS4 is set to be the final major version. There will be no TGS5, not from me.