This document aims to give an introduction to how module development is structured at Bastion of Peace. It introduces the systems and describes desired workflow. BoP is the basis for the document, but the framework contained in this repository has been made as generic as possible so that it may be used by anyone on any system.
If you either cannot, or struggle to, answer any of the following questions, then please read on.
- Why use these systems?
- What is git and why do we use it?
- What is module unpacking and why do we use it?
- How do we use git?
- How do I test locally?
- What is Docker and why do we use it?
- How do we use Docker?
Because the tools for developing module content were not created with collaboration in mind. If this is still unclear, consider the following scenario:
Now, Dropbox supports version control in that it keeps file history combined with the option of rolling back to previous versions, so Alice's dragon castle is not lost, but let's look at what happened here and explore our options.
The team realises that collaboration on a single .mod file poses the following challenges:
- Only one person can work on the .mod file at a time
- If a system in the module becomes broken, there's no history within the .mod file to help figure out what broke the system
- Sharing the module is cumbersome as the whole .mod file has to be uploaded and shared every time someone makes a change
Now, many may prefer a dragon castle over badgers in a flower field, but let's say we ignore coolness factor for a second and say we want both, we find ourselves in a position where there exists conflicting versions of the module (i.e. two .mod files) containing content that should be merged. This may be solved in two ways:
- Re-create the differing content of one module in the other
- Extract the differing content from one module and import it into the other
These two options work, but they require time and effort and are prone to errors. Content may be changed or even lost in the process, and while using Dropbox may work out under a strict development schedule to prevent overlap, this kind of conflict is bound to resurface over and over again.
Alternatively, we may choose a different strategy and not use Dropbox. Instead, we pass the module from developer to developer, ensuring in the process that only one person creates content at a time. While enforcing development on a single .mod file avoids the conflicting modules, it is a slow and disheartening process which may lead to updates taking months (if not years!), instead of days or weeks. And there's still no file history to look at when something gets broken.
Conflicting content from collaboration is nothing new and several tools already exist that handle this problem. They are described as systems of version control, examples of which are VCS, SVN, Mercurial, and git. In this project we're focusing on git. Now, you may ask why using a version control system was not mentioned in our options above. The answer is that for version control to make sense, we need to apply it on files in a readable format. The .mod file is a file archive and as such git will consider it as a binary file. This means git will be unable to merge the content of two versions, and as such is of no better use than Dropbox. What we want is text. Take a NWN script for instance. In the Toolset, scripts are readable text files, but when you hit save the Toolset packages all the module content into one file; the .mod archive. So what do we do?
We unpack the archive, and apply git to the unpacked content. Developers in the NWN community have created tools that can unpack .mod archives to source files, and repack source files in .mod archives. This enables us to utilize the power of git on module development, and provides complete file history along with merging capabilities.
Let's continuing with our story:
As already mentioned, git is a version control system. If you're unsure what this really means, please refer you to git basics for an introduction. git enables version control, but in order to reap the full benefits of version control, it has to be used in the right way. By everyone. So, we have to establish a set of ground rules. Please take the time to browse the following posts, use them as references, and most importantly - apply what you learn. Make sure to revisit these guidelines over and over again, and strive to use git for the benefit of the team as a whole:
- Beginner: GitCasts YouTube channel
- Beginner: Understanding Git Conceptually by Charles Duan
- Beginner: CLI introduction
- Beginner: Best practices
- Beginner: How to write a commit message
- Advanced: ProGit
- Advanced: Various tutorials
- Advanced: The fringes of git
Useful git commands
| Function | Command |
|---|---|
| Pull latest from repo | git pull |
| Push local changes to repo | git push |
| Current status of local repo | git status |
| Stage a file | git add <file> |
| Unstage a file | git reset HEAD <file> |
| Commit staged files | git commit |
| Discard changes in working directory | git checkout -- <file> |
The next challenge that must be solved is testing. The system structure of BoP development is depicted below
The clients are us, the developers, who push content to our private nwn-bop git repository that resides on github. The server is the computer that hosts the NWN module built from the sources in our git repo. The server hosts two game servers; the module built from the master branch, and the module built from the test branch. This setup implies the following workflow
See the uncertainty in this process? Because of it we push changes to the test branch, and deploy them on the test server to check out the effect. Now, if we had pushed quality changes every time, this would have been a sufficient setup, but alas, we do not. The commit log in the test branch has become littered with fix attempts, which is very poor git practice, and is becoming a considerable factor of useless noise to the rest of the team. So, we need to clean up, and the best way to do that is to make sure everyone can test their modifications locally before pushing a commit to the repository. Consider instead this workflow
See the difference? Not are there only fewer steps in the process, it's also faster because the developer does not rely on an external server to test his or her work. Great success!
So, how do we test? Simple, you load up the module as a local server and connect to it. Play through your content, and make modifications as you see fit. When the new content has been tried and tested, and only then, may the commit be pushed to the repo and deployed. Wait, hang on a minute.. Setting up a NWN server is not that straight forward! Quite right. Let's see how Bob, Alice and John copes with it.
So far, Alice has spent more time sysadmining than developing, and there seems to be no end to it. By setting up the same environments every time only for different OSes, she's doing the same thing over and over again. She needs a better solution.
Docker is that solution.
Unlike git which is used for version control of text files, Docker is kind of version control for computer environments. Specifically, Docker automates the deployment of applications inside software containers. What Docker allows us to do is set up a software environment with all the necessary tools for running a NWN server, wrap it up in a single file called a Docker image, and then distribute this image to the rest of the team. The image contains a version of Linux with all the right environment variables and software installed for running a NWN server. All anyone needs to do to use it, is install Docker and launch the image, regardless of the platform they're on. And the best thing? It's a one time process. Once created, docker image is ready for use by anyone.
We're not quite there yet, but I will add instructions here when ready. For now, wait in excitement!