Export Process
This page describes the actual process of exporting data from the game.
Class: FactorioItemBrowser\Export\Command\ProcessCommand
The Export is requesting the next export job from the Combination API with the status queued. If the API does not
provide a matching job, we are already done, and wait for the next cron trigger.
The "next" export job is currently the oldest job with the queued status, paying attention to the priority of the job,
so that e.g. jobs triggered by a user are preferred to those triggered by an automatic update.
Class: FactorioItemBrowser\Export\Command\ProcessStep\DownloadStep
The status of the export job is updated to downloading.
This step involves checking all mods already downloaded to the local disc whether they have their latest version, or are available at all. For this the Export accesses the API of the Factorio Mod Portal to check the latest versions of all mods from the export job (excluding the base mod of course). Any non-available or outdated mods get downloaded into the local mods directory.
The mods actually get extracted and are saved unzipped on the disk, and the version number is removed from the directory names. This is to easier access any files in the mods (to not always have to know the version number), especially when rendering the icons. This mods directory is not the actual mods directory of the Factorio instance.
Depending on the number of missing mods, this step is taking the longest of all.
Class: FactorioItemBrowser\Export\Command\ProcessStep\FactorioStep
The status of the export job is updated to processing.
Before the local Factorio can be launched, some preparations must be done:
- Created symlinks from the Factorio's mods directory to the actual mods, paying attention to insert the correct version numbers.
- Generate the
info.jsonfile of the Dump mod, and add the mod to the mods directory as well. Theinfo.jsonfile has all other mods as dependencies, which guarantees that our Dump mod is loaded last.
The Export hen launches the game in headless mode, letting a save create. Creating a save will have the game run for some ticks before it will cleanly shut down. We only need the first tick anyway.
Class: FactorioItemBrowser\Export\Command\ProcessStep\ParserStep
During the run the data gets gathered by the Dump mod and written to the log file as JSON. This log file will be read in, and the data gets parsed, normalizing it. In this step, the data part of the zip is created, which eventually gets uploaded to the web server.
Class: FactorioItemBrowser\Export\Command\ProcessStep\RenderIconsStep
Factorio allows the mod creators, to define icons of items and recipes etc. as separate layers, applying tint and shifts to them, and so on. While the game will render all layers on-the-fly whenever the icon is needed, this is not applicable to a website.
This is why in this step the Export renders all icons, creating from all layers the final icon as it appears ingame.
Because rendering icons, especially when applying a tint where each pixel must be calculated on its own, is a CPU intensive task and PHP is not the best language to handle this, the Export Icon Renderer was extracted into a separate executable, implemented in and compiled with Go.
All rendered icons get added to the already created zip archive as well.
Class: FactorioItemBrowser\Export\Command\ProcessStep\UploadStep
The status of the export job is updated to uploading.
An FTP connection to the web server is established, and the created zip archive is uploaded to the server.
Class: FactorioItemBrowser\Export\Command\ProcessStep\DoneStep
The status of the export job is updated to uploaded.
This status change is the signal for the importer to actually import the uploaded data into the database.
