New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: Network protocol of AoC 1.0c #877

Merged
merged 24 commits into from Jan 21, 2018
Commits
Jump to file or symbol
Failed to load files and symbols.
+3,560 −244
Diff settings

Always

Just for now

@@ -12,6 +12,8 @@ This works for normal movement as well as for patrolling and "Attack Move", with
Although the flags on the ground start to disappear after 10 waypoints, there is no limit to how much waypoints can be added to the queue. This only applies to normal movement though. "Patrol" and "Attack"-Mode have a queue limit of 10 waypoints. If these modes are selected, the 10th `SHIFT + Right-Mouse-Button` automatically initiates movement.
Waypoints are not as precise as normal movement commands because they only use the coordinates of the tile they are placed on as a reference point, instead of a floating point value.
## Weird AoE2 Quirks
* Normal movement waypoints cannot be canceled, even if the unit is deselected.
@@ -0,0 +1,124 @@
# General Structure
This document defines the general structure of the protocol.
## Test Setup
The test environment consists of 4 hosts to simulate a 4-player multiplayer game. Each host was equipped with:
* Ubuntu 17.04
* Age of Empires 2: The Conquerors 1.0c
* Wine 1.8.7
* Wireshark 2.2.6
One game was established by using the DirectPlay feature, while all other play sessions were established over LAN. There is no detectable difference for the communication protocol and DirectPlay is only used for the purpose of connecting players.
## Transport Protocols
The game utilizes TCP to let players connect to a play session and UDP for all game configuration and ingame communications. This is not surprising, since UDP has the advantage of lightweight packets and therefore fast communication. This document will focus on the interesting part of the packets, i.e. the UDP data field.
## Packet types
There are 3 basic types of packets; sync packets, chat message packets and (player) action packets. Sync packets are used to synchronize communication turns, determine turn timers, calculate latency and validate the game state. Chat message packets transport everything the players type in the ingame or lobby chat over the network. The last type discussed in this documentation are the action packets which are utilized for commands from a player or an AI (e.g. movement, unit training).
Sync packets can be further categorized into periodic (sent in regular intervals) and non-periodic packets. Most of the network communication during a multiplayer game consists of synchronization data.
Packets are recognized by a one byte long "command byte" in the header. So far 12 different network commands have been identified.
Command | Purpose
--------|--------------
0x31 | Sync
0x32 | Sync
0x35 | Sync (Lobby)
0x3e | Player-issued
0x41 | Sync
0x43 | Chat Message
0x44 | Sync
0x4d | Sync
0x51 | De-Sync
0x52 | Readying (Lobby)
0x53 | Sync
0x5a | Lobby
All packets with command `0x3e` have a second "command byte" after the header that represents the command a player has given ingame. To avoid confusion, we will call all player-issued commands "actions" and reserve the term "commands" for the actual network commands seen above. Furthermore, the identifier for a player's action will be called "action byte". 34 of these can be found in network packets.
Action | Purpose
-------|------------
0x00 | Primary Action (Attacking, Resource gathering, Boarding Transport Ship)
0x01 | Stop
0x02 | Primary Action (AI)
0x03 | Move
0x0a | Move (AI)
0x0b | Resign
0x10 | Set waypoint
0x12 | Stance
0x13 | Guard
0x14 | Follow
0x15 | Patrol
0x17 | Formation
0x1b | Save & Exit
0x1f | Coordinated Move (AI)
0x64 | Train unit (AI)
0x65 | Research
0x66 | Build
0x67 | Diplomacy/Cheats/Change Speed
0x69 | Build wall
0x6a | Delete
0x6b | Attack ground
0x6c | Tribute
0x6e | Repair
0x6f | Unboard/Ungarrison
0x72 | Toggle gate
0x73 | Flare
0x75 | Garrison/Stop building unit
0x77 | Train unit (Human)
0x78 | Rally point
0x7a | Sell
0x7b | Buy
0x7e | Drop relic
0x7f | Toggle townbell
0x80 | Back to work
When the game is recorded, the UDP data stream of a `0x3e` packet (without the header) is written straight into the .mgx files with few changes. Viewing the recording will therefore simulate the exact actions that were done by the players. For more information on this, check the **Further Reading** section below.
Much of the actions where already figured out by Stefan Kolb as part of his [.mgx Specification](https://github.com/stefan-kolb/aoc-mgx-format). This analysis will use his document style ([Example](https://github.com/stefan-kolb/aoc-mgx-format/blob/master/spec/body/actions/03-move.md)) as a template.
## Values and data types
Values in the network protocol can have a length of one, two or four byte. Little endian notation is used. Therefore, values with a length of two and four bytes have to be read starting with the rightmost byte.
The data is described with few data types, which are shown in the table below.
Length | Data Types
---------|------------
1 byte | int8, (ASCII) char
2 byte | int16
4 byte | int32, float
other | (1-dimensional) array
Most of the fields present in the network protocol have a fixed length. The use cases for variable length fields are usually lists of `unit_id`s or waypoints and will be handled as arrays in this documentation.
## ID system
Age of Empires II uses an ID system to reference every object in the game by a unique numerical identifier with a length of 4 bytes. IDs are assigned by using a simple counter, that assigns every new object a the next unassigned number. New objects are not necessarily created by players. For example, cutting down a tree replaces a standing with a fallen tree, whereby the latter is handled as a new object. As a rule of thumb, one can assume that objects have been replaced with new ones when their sprite has changed.
The players are referenced with not less than three IDs; their *network_source_id*/*network_dest_id* (4 bytes), the *player_id* and the *player_number* (both 1 byte). *network_ids* are used in the header to determine from which person a packet comes from or is sent to. The reason why *player_id* and *player_number* are handled differently is due to an undocumented cooperative multiplayer mode. In this mode, which can be activated by assigning to players the same *player_number*, two or more players share control of units, buildings and resources. In consequence, *player_id* is unique for every player, *player_number* not necessarily.
## Coordinate system
In this document, we will assume that AoC uses a carthesian coordinate system with the left corner as an origin point. Keep in mind, that AoC uses a **Quadrant 4** representation for its coordinates, which means that the y-axis is represented by the edge in the bottom left and the x-axis by the top left edge.
## Further Reading
To get a better understanding of the networking design and the underlying principles, it might be beneficial to read these sources.
[1] Dave Pottering, *1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond*, 2001
https://www.gamasutra.com/view/feature/131503/1500_archers_on_a_288_network_.php?page=1
[2] Matthew Pritchard, *How to Hurt the Hackers: The Scoop on Internet Cheating and How You Can Combat It*, 2000
https://www.gamasutra.com/view/feature/3149/how_to_hurt_the_hackers_the_scoop_.php
[3] Stefan Kolb, *Age of Empires 2: The Conquerors — Savegame File Format Specification*,
https://github.com/stefan-kolb/aoc-mgx-format
[4] Renée Kooi, *RecAnalyst*, https://github.com/goto-bus-stop/recanalyst
@@ -0,0 +1,51 @@
# Protocol Header
The header has a fixed length of 20 bytes and is present in almost all of the packets. Exceptions are the sync packets with 16 byte length, who have a smaller header.
## Definition
```ruby
def Header
int32 :network_source_id
int32 :network_dest_id
int8 :command
int8 :option1
int8 :option2
int8 :option3
int32 :communication_turn
int32 :individual_counter
end
```
## Description
*:network_source_id*<br/>
The *:network_id* of the person who sent the packet. A *:network_id* is different for every game, but is not generated randomly for all players. When joining the lobby, every player gets assigned `last_network_id - 2` as his own *:network_id* where *last_network_id* is the ID of the person who joined before him.
*:network_dest_id*<br/>
The *:network_id* of the person who should receive the packet. Is only used for sync packets and remains unused for most commands.
*:command*<br/>
The command which tells the network engine how the data bytes should be interpreted.
*:option{1-3}*<br/>
These are options with unknown effects.
*:communication_turn*<br/>
The communication turn where the command is executed. Packets that contain a communication turn that has already passed are discarded.
*:individual_counter*<br/>
The individual counter of the player who receives the packet. It increments by 1 with every 16BC41 sync command. For individual players, this counter starts with values that are 1200 units (two players) or 2000 units (three players, `0x7d0`) apart from each other. In the latter example player 1 starts with the value `0x7d0`, player 2 with `0xfa0` and player 3 with `0x1770`. If the differences between these values is not a multiple of 2000, this indicates a de-sync.
## Examples
`82 b2 45 00 00 00 00 00 53 02 8f 0b 30 00 00 00 01 08 00 00`
>`82 b2 45 00` &mdash; network_source_id<br/>
>`00 00 00 00` &mdash; network_dest_id<br/>
>`53` &mdash; command<br/>
>`02` &mdash; option1<br/>
>`8f` &mdash; option2<br/>
>`0b` &mdash; option3<br/>
>`30 00 00 00` &mdash; communication_turn<br/>
>`01 08 00 00` &mdash; individual_counter
Oops, something went wrong.
ProTip! Use n and p to navigate between commits in a pull request.