compiling-unreal

A man's notes as he discovers how Unreal Engine compiles things

Google Slides

This is available in google slides form here, which might either contain more information or less information than this repo because honestly I forgot how I took these notes.

https://docs.google.com/presentation/d/1jRY2raIiX2AlBRhm5b8I98a93de6lihYSmm99asAMfo/edit?usp=sharing

Support

If you have gotten any use out of this, please consider becoming one of my GitHub Sponsors. Even a $1 is very appreciated.

No Warranty

I make no claims that any of this information is accurate, correct, or relevant in any way. It definitely won't ever be completed.

Licensing

MIT, please contribute knowledge if this helps you discover any. Please.

Build Pipeline Tools

Build Scripts
- Build.bat
DotNETUtilities
UnrealBuildTool
UnrealHeaderTool

Build Scripts — Build.bat

Wrapper for UnrealBuildTool.exe
Ensures that the current working directory is set to the Engine’s Source folder
Checks to see if UnrealBuildTool.exe exists and errors out if it doesn’t
Otherwise just passes all command line args to UnrealBuildTool.exe

DotNETUtilities

Provides general utilities for all DotNET tools for Unreal Engine
Contains things like:
- A JSON library
- A library for working with Perforce
- File Handling
- Logging
- Process management and queuing
- Command Line Arg Processing

Visual Studio Notes

Project files generated in Engine\Intermediate\ProjectFiles
This also appears to be the working directory for NMake/Build commands

How To Debug UnrealBuildTool in the Development Editor Configuration

Why would you want to debug while using the Development Editor configuration?
- I am hella lazy and I don’t want to keep toggling that Debug box
Do the following to DotNETUtilities and UnrealBuildTool:
- In Project Build Properties: Define DEBUG Constant, Disable Optimize Code
- Set Debugging Information in the Output Advanced window to Full
Use Entrian Attach to attach to Engine\Binaries\DotNET\UnrealBuildTool.exe
- Otherwise start up the UnrealBuildTool project using command line args like: UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

UnrealBuildTool

The following section is centered on the mechanisms specific to the UnrealBuildTool.

Building a Target

When you right-click a project in Visual Studio and click Build, this is ultimately what you're doing.

Visual Studio calls the Build.bat script with command-line arguments representing your desired build target. It does this in the form of:

Build.bat TargetName TargetPlatform Configuration OptionalAdditionalArgs

For example, when building the UnrealHeaderTool directly, you invoke:

Build.bat UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

Fun fact: -FromMsBuild means use MSBuild Message Format for warnings and errors, which is a format that Visual Studio understands allowing for navigating directly to the error. That format takes the form of the following DotNet Regular Expression:

(?'FILENAME'.+):(?'LINE'\d+):(?'COLUMN'\d+): (?'CATEGORY'error|warning): (?'TEXT'.*)

Important Objects To Know About

Feel free to skip this list and come back to it when you come across an object term or type you don't know. It'll hopefully be in here.

#TODO: Make a table of contents that links to the right shit

SingleInstanceMutexes

SingleInstanceMutexes

UnrealBuildTool uses SingleInstanceMutexes to ensure that no two UnrealBuildTool instances are going to stomp each other and compete for the same resources. This uses DotNet Mutexes that are named based on your Engine instance, so you can compile two different Engines at once, but you can't build two targets within the same Engine simultaneously.

SingleInstanceMutex(string MutexName, bool bWaitMutex)
- Creates Mutexes with formatted string: Global\\{0}_{1}
  - {0}: Mutex Name
  - {1}: DotNet String GetHashCode()
  - i.e. Global\UnrealBuildTool_Mutex_XmlConfig_-2108754181
Used Mutex Names:
- UnrealBuildTool_Mutex_XmlConfig, Assembly.GetExecutingAssembly().CodeBase
  - Used when reading XmlConfig files on UnrealBuildTool startup
- UnrealBuildTool_Mutex, Assembly.GetExecutingAssembly().CodeBase
  - Used when actually building
- UnrealBuildTool_WriteMetadata, UnrealBuildTool.RootDirectory.FullName
  - Used when UnrealBuildTool runs UnrealBuildTool in WriteMetaData mode as part of a build so that DebugGame and DevelopmentEditor .module files can be written in parallel

XMLConfigFiles

The UnrealBuildTool can be configured in many ways. One of them is through configuration files in .xml form. These are generally global configuration options that affect the entire build pipeline as opposed to settings you might want to be passed to tools via command-line. Like Unreal's .ini configuration system, there are multiple sources and layers to XMLConfigFiles.

XML Config Cache Validation

Skipped if XMLConfigCache= is passed
Non-Installed Engine, DefaultConfigFile is Engine\Saved\UnrealBuildTool\BuildConfiguration.xml
Installed Engine DefaultConfigFile is %localappdata%\UnrealEngine\XmlConfigCache-(EnginePath).bin
Global AppData override of UnrealBuildtool Configuration: %APPDATA%\Unreal Engine\UnrealBuildTool\BuildConfiguration.XML
Global Documents override of UnrealBuildtool Configuration?: My Documents\Unreal Engine\UnrealBuildTool\BuildConfiguration.XML
Reads XML Schema: Engine\Saved\UnrealBuildTool\BuildConfiguration.Schema.xsd
Checks to see if XMLConfigCache is up to date (UnrealBuildTool_Mutex_XmlConfig)
1. Checks to see if UnrealBuildTool’s executable is newer than cache, if so, rebuild cache
2. Checks to see if config files in cache’s timestamps are newer than cache timestamp as well as a schema validation check
3. If not, generates a Dictionary<string (CategoryName), Dictionary<string (FieldName), FieldInfo>> of all UnrealBuildTool C# types that have a field marked XMLConfigFileAttribute
  1. Non-installed engines will write out schema
  2. All XML files in the config cache listed above are read and validated against the schema
  3. All config values saved in the config cache file

UnrealBuildTool's Config Hierarchy / Config Layers

Practical Knowledge

{TYPE} refers to a Config Type Name that is passed in when merging config files. The “Engine” or “Game” or “Editor” or “Input” in DefaultEngine/DefaultGame/DefaultEditor/DefaultInput.ini
Platform becomes a config suitable name like “Windows” and is not the same as a Target Platform name which we’ll see later “Win64”
Note: All paths with Platform in them will first check to see if Engine\Platforms\PlatformName exists before looking into the normal Engine/Config/Platform folder. This allows for easier NDA and “add on” platform extensions, both config wise and code/content wise.
Projects with Platform can also be extended in this way with their extensions in Project\Platforms instead of Engine\Platforms

Non-User Specific Settings

Path	Notes
Engine/Base.ini	Empty, but required to exist
Engine/Base{TYPE}.ini
Engine/Platform/BasePlatform{TYPE}.ini	i.e. Engine/Windows/BaseWindows*.ini
Project/Default{TYPE}.ini	i.e. see image to the right
Engine/Platform/Platform{TYPE}.ini	i.e. Engine/Windows/Windows*.ini
Project/Platform/Platform{TYPE}.ini	i.e. Project/Windows/Windows*.ini

User Specific Settings

Path	Notes
UserSettings/.../User{TYPE}.ini	This no longer exists? i.e. AppData/Local/Unreal Engine/Engine/Config/UserEngine.ini
UserDir/.../User{TYPE}.ini	This no longer exists? i.e. Documents/Unreal Engine/Engine/Config/UserEngine.ini
Project/User{TYPE}.ini	i.e. Project/User*.ini

Technical Knowledge

This is how the Config paths are generated.

Tokens

{ENGINE} = Absolute path to the Engine
- D:\UE425\Engine
{PROJECT} = Absolute path to Project
{PLATFORM} = See slide on Platform
{TYPE} = Requested .ini type
- Engine, Game, Input, etc.
{EXTENGINE} = Extension folder
- Used if exists, otherwise Engine
{USERSETTINGS} = LocalAppData/Roaming
{USER} = User My Documents folder
- Expansion Tags (out of the box stock engine):
{ED} = Directory Prefix
- Note the /
{EF} = File Prefix

Any path with {ED} or {EF} gets thrown into an ConfigLayerExpansion loop. Within that loop, any path with {PLATFORM} gets thrown into a DataDerivedPlatformInfo loop.

Non-User Specific Settings

Path	Notes
{ENGINE}/Base.ini
{ENGINE}/{ED}{EF}Base{TYPE}.ini
{ENGINE}/{ED}{PLATFORM}/{EF}Base{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTENGINE}/{ED}{EF}Base{PLATFORM}{TYPE}.ini
{PROJECT}/{ED}{EF}Default{TYPE}.ini
{ENGINE}/{ED}{PLATFORM}/{EF}{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTENGINE}/{ED}{EF}{PLATFORM}{TYPE}.ini
{PROJECT}/{ED}{PLATFORM}/{EF}{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTPROJECT}/{ED}{EF}{PLATFORM}{TYPE}.ini

User Specific Settings

Path Notes

{USERSETTINGS}/Unreal Engine/Engine/Config/User{TYPE}.ini

{USER}/Unreal Engine/Engine/Config/User{TYPE}.ini

{PROJECT}/User{TYPE}.in

Expansion Loop

If config format has either {ED} or {EF} in it, it runs through ConfigLayerExpansions which allow for additional overrides

All config files with expansion tags run through entire array of the following ED (Directory), EF (File) pairs. i.e. {ENGINE}/Config/{ED}{EF}BaseEngine.ini

{ED}	{EF}	Example
		Engine/Config/BaseEngine.ini
	Shippable	Engine/Config/ShippableBaseEngine.ini
NotForLicensees/		Engine/Config/NotForLicensees/BaseEngine.ini
NotForLicensees/	Shippable	Engine/Config/NotForLicensees/ShippableBaseEngine.ini
NoRedist/		Engine/Config/NoRedist/BaseEngine.ini
NoRedist/	Shippable	Engine/Config/NoRedist/ShippableBaseEngine.ini

If config path has a {PLATFORM} tag within it, for every Expansion iteration we also perform Platform lookup.
1. Fetch DataDrivenPlatformInfo
  1. In non-extension platform folders, i.e. Engine/Config/Windows/, there exists a DataDrivenPlatformInfo.ini which is used to defining various platform configurations and targets. That platform name the .ini is loaded for is the file’s parent folder: Engine/Config/Windows/DataDrivenPlatformInfo.ini, platform = Windows
  2. In extension platform folders, same logic, platform name is two folders up: Engine/Platforms/Windows/Config/DataDrivenPlatformInfi.ini, platform = Windows
2. UnrealBuildTool only loads the following information per platform:
  1. bIsConfidential
  2. AdditionalRestrictedFolders (an array of strings)
  3. IniParentChain (an array of strings)
    1. If IniParentChain is not null, platform ini’s loaded from most generic to most specific, with most specific being our current active platform
    2. Unix then Linux i.e. Engine/Unix/BaseUnixEngine.ini, then Engine/BaseLinuxEngine.ini
    3. Unix then Arch64Linux
    4. Android then Lumin
    5. IOS then TVOS i.e. Engine/IOS/BaseIOSEngine.ini, then Engine/TVOS/BaseTVOSEngine.ini

Full List of Config Files

These are all the config files loaded for Linux for config type Engine. This process is repeated for Game, Input, etc...

Linux does not use the platform extension folder, which would change Engine config paths from Engine\Config\PlatformName to Engine\Platforms\PlatformName\Config.

Engine/Base.ini
Engine/BaseEngine.ini
Engine/ShippableBaseEngine.ini
Engine/NotForLicensees/BaseEngine.ini
Engine/NotForLicensees/ShippableBaseEngine.ini
Engine/NoRedist/BaseEngine.ini
Engine/NoRedist/ShippableBaseEngine.ini
Engine/Unix/BaseUnixEngine.ini
Engine/Unix/ShippableBaseUnixEngine.ini
Engine/Unix/NotForLicensees/BaseUnixEngine.ini
Engine/Unix/NotForLicensees/ShippableBaseUnixEngine.ini
Engine/Unix/NoRedist/BaseUnixEngine.ini
Engine/Unix/NoRedist/ShippableBaseUnixEngine.ini
Engine/Linux/BaseLinuxEngine.ini
Engine/Linux/ShippableBaseLinuxEngine.ini
Engine/Linux/NotForLicensees/BaseLinuxEngine.ini
Engine/Linux/NotForLicensees/ShippableBaseLinuxEngine.ini
Engine/Linux/NoRedist/BaseLinuxEngine.ini
Engine/Linux/NoRedist/ShippableBaseLinuxEngine.ini
Project/DefaultEngine.ini
Project/ShippableDefaultEngine.ini
Project/NotForLicensees/DefaultEngine.ini
Project/NotForLicensees/ShippableDefaultEngine.ini
Project/NoRedist/DefaultEngine.ini
Project/NoRedist/ShippableDefaultEngine.ini
Engine/Unix/UnixEngine.ini
Engine/Unix/ShippableUnixEngine.ini
Engine/NotForLicensees/Unix/UnixEngine.ini
Engine/NotForLicensees/Unix/ShippableUnixEngine.ini
Engine/NoRedist/Unix/UnixEngine.ini
Engine/NoRedist/Unix/ShippableUnixEngine.ini
Project/Unix/Unix/Engine.ini
Project/Unix/ShippableUnixEngine.ini
Project/NotForLicensees/Unix/UnixEngine.ini
Project/NotForLicensees/Unix/ShippableUnixEngine.ini
Project/NoRedist/Unix/UnixEngine.ini
Project/NoRedist/Unix/ShippableUnixEngine.ini
AppData/Local/Unreal Engine/Engine/Config/UserEngine.ini*
Documents/Unreal Engine/Engine/Config/UserEngine.ini*
Project/UserEngine.ini

I believe the two crossed out config files should have been updated to use a newer config method that Unreal Engine uses in which it places config files in versioned folders inside AppData. Maybe this is a bug in the UnrealBuildTool but no one has build settings in their User Data folders so it doesn't matter?

Global BuildConfiguration Settings

When building, the UnrealBuildTool creates a global BuildConfiguration object that configures all builds. Unless otherwise noted, default is false/null. It can be configured in various ways using both XMLConfigFiles and command-line options.

#TODO Write a script to parse this shit from .xml schema

XMLConfigFile Fields

Field	Description
bIgnoreOutdatedImportLibraries = `true`	Skips checking for outdated import libraries, speeds up iteration time
bUsePrecompiled	Use existing static libraries for all engine modules in this target.
bPrintDebugInfo	Whether debug info should be written to the console.
bLogDetailedActionStats	Whether to log detailed action stats. This forces local execution.
bAllowHybridExecutor	Whether the hybrid executor will be used (a remote executor and local executor).
bAllowXGE = `true`	Whether XGE may be used.
bAllowSNDBS = `true`	Whether SN-DBS may be used.
bUseUBTMakefiles = `true`	See slide about UBT Makefiles
bAllowDistcc	Whether DMUCS/Distcc may be used.
bAllowParallelExecutor = `true`	Whether to allow using parallel executor on Windows.
MaxParallelActions	Number of actions that can be executed in parallel. If 0, computes based on available cores
bForceHeaderGeneration	If `true`, force header regeneration. Intended for the build machine.
bDoNotBuildUHT	If `true`, do not build UHT, assume it is already built.
bFailIfGeneratedCodeChanges	If `true`, fail if any of the generated header files is out of date.
bAllowHotReloadFromIDE	`True` if hot-reload from IDE is allowed.
bForceDebugUnrealHeaderTool	If `true`, Debug version of UHT will be built and run instead of the Development version.
MaxRootPathLength = 50	Maximum recommended root path length.
MaxNestedPathLength = 200	Max length of a relative path. Windows limit to make paths portable between machines.

Command-line Options

Command-line Arg	Description
UsePrecompiled	Use existing static libraries for all engine modules in this target.
NoXGE	Disables use of XGE
NoUBTMakefiles	Disables use of UBT Makefiles
MaxParallelActions	Number of actions that can be executed in parallel. If 0, computes based on available cores
ForceHeaderGeneration	Forces header regeneration. Intended for the build machine.
NoBuildUHT	Skips building UnrealHeaderTool, assume that it is already built.
FailIfGeneratedCodeChanges	Fail if any of the generated header files is out of date.
NoHotReloadFromIDE	Disables hot reload from IDE
SkipRulesCompile	Skip compiling rules assemblies and just assume they are valid

Target Descriptors

A TargetDescriptor describes all of the information needed to initialize a UEBuildTarget object. It is made up of the following:

ProjectFile: (Can be Null) A file path to a .uproject file that describes project level information about are target.
Name: A string representing the name of the target
Platform: This target’s UnrealTargetPlatform i.e. Win32, Win64, LinuxAArch64, Mac, IOS, etc.
Configuration: This target’s UnrealTargetConfiguration, i.e. Unknown, Debug, DebugGame, Development, Shipping, Test
Architecture: A string representing this target’s architecture, usually just an empty string. Appears to be only used on Linux/HoloLens/iOS.

AdditionalArguments: Command line arguments to use when building this target; aggregate of initial UBT command line args as well as any additional args specified for a target when a TargetList is specified

Field Name	Command-line Arg	Description
ForeignPlugin	-ForeignPlugin	Foreign plugin to compile against this target
OnlyModuleNames	-Module=	List of modules to only build
SingleFileToCompile	-SingleFile=	File path to the only file that should be compiled
HotReloadMode	-NoHotReload, -ForceHotReload, -LiveCoding	Sets mode for Hot Reloading
WriteActionFiles	-WriteActions=	Export the actions for the target to a file
LiveCodingModules	-LiveCodingModules=	Path to a file containing a list of modules that may be modified for live coding
LiveCodingManifest	-LiveCodingManifest=	Path to the manifest for passing info about the output to live coding
bQuiet	-Quiet	Suppress messages about building this target

Example: Building UnrealHeaderTool Target Descriptor

Building the UHT, this is the initial UBT command line UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

Goal is to build a TargetDescriptor, which requires: ProjectFile (Can be Null), TargetName, Platform, Configuration, Architecture, Arguments

Skips Target List processing as -TargetList/-Target are not provided, go right into parsing this command line as a TargetDescriptor
ProjectFile set as null
Determines Platform is Win64
Determines Configuration is Development
Determines UnrealHeaderTool as the TargetName
Uses default Architecture for Win64, which is an empty string “”
Scan for any ProjectFiles that match this target, which there are none of, so ProjectFile remains null
-WaitMutex and -FromMSBuild added as additional commandline arguments for this target
TargetDescriptor fully built

Engine BuildVersion

A BuildVersion class describes all the versioning info for any given UE4 Engine.

Always located in Engine\Build\Build.version in JSON format.

Includes the following information:

Field Name	Description
MajorVersion	Is always 4 (the 4 in UE4)
MinorVersion	At time of writing, this is 25, as in 4.25
PatchVersion	At time of writing, this is .0, even though I’m using Preview 3
Changelist	GitHub Releases this should be 0
CompatibleChangelist	GitHub Releases this should be 0
IsLicenseeVersion	`True` if using your own versioning on top of UE4’s versioning?
IsPromotedBuild	`True` if build is a marked good build from a ‘clean sync’ of a specific changelist
BranchName	Name of the branch, i.e. ++UE4+Release-4.25, based on source GitHub/Perforce branch
BuildId	Current build id. Generated automatically when Engine binaries change if not set in BuildVersion
BuildVersion	Stored as BuildVersionString. Unique string identifying this target and engine, updated via AutomationTool

UEBuildTarget Object

A UEBuildTarget contains all the information needed to build a target. This is required to generate a Makefile.

Field Name	Description
Rules	The target rules
`RulesAssembly`	The rules assembly to use when searching for modules
MetadataCache	Cache of source file metadata for this target
ProjectFile	The project file for this target
ProjectDescriptor	The project descriptor for this target
TargetType	Type of target
AppName	Name of the application the target is part of. If bUseSharedBuildEnvironment = `true`, typically “UE4Editor” for editors
TargetName	The name of the target
bUseSharedBuildEnvironment	Whether target uses shared build environment. If false, all binaries should be written to the project directory.
Platform	Platform as defined by the VCProject and passed via the command line. Not the same as internal config names.
Configuration	Target as defined by the VCProject and passed via the command line. Not necessarily the same as internal name.
PlatformIntermediateFolder	Relative path for platform-specific intermediates (eg. Intermediate/Build/Win64)
ProjectDirectory	Root directory for the active project. Typically contains the .uproject file, or the engine root.
ProjectIntermediateDirectory	Default directory for intermediate files. Typically underneath ProjectDirectory.
EngineIntermediateDirectory	Directory for engine intermediates. Monolithic builds will be the project intermediate directory.
bHasProjectScriptPlugin	Identifies whether the project contains a script plugin. This will cause UHT to be rebuilt, even in installed builds.
BuildPlugins	All plugins which are built for this target
EnabledPlugins	All plugin dependencies for this target.
ForeignPlugin	Specifies the path to a specific plugin to compile.
Binaries	All application binaries; may include binaries not built by this target.
NonFilteredModules	Kept to determine the correct module parsing order when filtering modules.
bCompileMonolithic	`true` if target should be compiled in monolithic mode, false if not
TargetRulesFile	The name of the `.Target.cs` file, if the target was created with one
bDeployAfterCompile	Whether to deploy this target after compilation

ModuleRulesContext

Stores information about where a module rules object came from, and how it can be used.

Field Name	Description
RulesScope	The scope for this module. Used to validate references to other modules.
DefaultOutputBaseDir	The default directory for output files
Plugin	The plugin that this module belongs to
bCanHotReload	Whether this module should be included in the default hot reload set
bCanBuildDebugGame	Should module be compiled with optimization off in DebugGame configs (ie. whether it's a game module).
bCanUseForSharedPCH	Whether this module can be used for generating shared PCHs
bClassifyAsGameModuleForUHT	Whether to treat this module as a game module for UHT ordering
DefaultUHTModuleType	The default module type for UnrealHeaderTool. Do not use this for inferring other things about the module.

`RulesAssembly` Object

The most practical way to think of a RulesAssembly Object is to imagine a .dll that simply consists of classes defined by a group of Module .Build.cs files. DotNet C# allows dynamic compilation and loading of .dlls (Assemblies). The UnrealBuiildTool uses this mechanism so that Module authors can use C# to programmatically define the build rules for their Modules but not worry about building the UnrealBuildTool assemblies that are made up of all engine and project module build rules. The RulesAssembly Object is essentially a bunch of metadata about these .dlls as well as a reference to a .dll.

The Engine makes use of two RulesAssembly Objects.

Rules Assembly	Description
UE4Rules.dll	This is an Engine/Enterprise Assembly that contains all the compiled C# `.Build.cs` Module build rules in the `Engine`, excluding `Programs`
UE4ProgramRules.dll	This is an Engine/Enterprise Assembly that contains all the compiled build rules for all the Engine Program Targets including `UE4Editor.Target`. This assembly has `UE4Rules.dll` as a parent `RulesAssembly` as `Engine Programs` rely on `Engine Modules` and are scoped as such

A RulesAssembly stores information about a compiled DotNet C# .dll assembly that contains our assembly’s build rules and the types it contains. When a RulesAssembly Object is constructed, it attempts to compile and load a DotNet C# Assembly.

Field Name	Description
Scope	Outers scope for items created by this assembly. Used for chaining assemblies together.
CompiledAssembly	The compiled assembly
BaseDirs	The base directories for this assembly
Plugins	All the plugins included in this assembly
ModuleNameToModuleFile	Maps module names to their actual xxx.Module.cs file on disk
TargetNameToTargetFile	Maps target names to their actual xxx`.Target.cs` file on disk
ModuleFileToContext	Mapping from module file to its context.
ModuleHasSource	Cache for whether a module has source code
bContainsEngineModules	Whether this assembly contains engine modules. Used to set default values for bTreatAsEngineModule.
`DefaultBuildSettings`	Whether to use backwards compatible default settings for module and target rules. This is enabled by default for game projects to support a simpler migration path, but is disabled for engine modules.
bReadOnly	Whether the modules and targets in this assembly are read-only
Parent	The parent rules assembly that this assembly inherits. Game assemblies inherit the engine assembly, and the engine assembly inherits nothing.

Compiling and Loading a `RulesAssembly` Object

A RulesAssembly stores information about a compiled DotNet C# .dll assembly that contains our assembly’s build rules and the types it contains.

When constructed, consumes a Scope, list of base directories (Engine + EngineExt), list of PluginInfos, a dictionary mapping module files to ModuleRulesContext objects, list of TargetFiles to compile, an assembly name, whether this assembly contains engine modules, a BuildSettingsVersion (V1, V2, Latest), whether assemblies readonly, whether to skip compile, and our parent RulesAssembly
Builds list of Assembly SourceFileNames (union between all module and target files)
Grab all Preprocessor Definitions (this is the entire list)
- WITH_FORWARDED_MODULE_RULES_CTOR
- WITH_FORWARDED_TARGET_RULES_CTOR
- UE_4_{MINORVERSION}_OR_LATER, i.e. UE_4_25_OR_LATER; includes all defines MinorVersion 17 and upwards to current

DynamicCompilation module compiles an Assembly consuming OutputAssemblyPath, HashSet of SourceFileNames, a list of ReferencedAssemblies, list of preprocessor definitions, whether to not compile, and whether to treat warnings as errors

Load the Assembly manifest, same path as Assembly, just swap out .dll with Manifest.json
1. i.e. D:\UE425\Engine\Intermediate\Build\BuildRules\UE4RulesManifest.json
2. This file appears to be generated with Generating Project Files (GPF)
3. Contains a list of all Engine module files (*.Build.cs``) and the Engine version that it was generated with
If not skipping compiling, check to see whether we require compilation
1. Consumes SourceFileNames, the Assembly manifest, and the Assembly output path
2. Require compilation if Assembly output file does not exist
3. Require compilation if UnrealBuildTool is newer than Assembly
4. Require compilation if Assembly manifest does not exist (it is generally expected to always exist)
5. Require compilation if Engine version has changed from the manifest Engine version
6. Require compilation if any Module source files were added or removed or have a newer timestamp than Assembly
If we do not need to compile our assembly, we try to load it
1. DotNet C# has a built in way of loading Assemblies, simply Assembly.LoadFile(FilePath), if fails, mark for recompile

If we need to compile our assembly, use the DynamicCompilation class to CompileAssembly

Build DotNet CompileParameters

Parameter	Description
GenerateInMemory = false	Always compile the assembly to a file on disk
OutputAssembly = Output Path	This is the full path to the assembly file we're generating
GenerateExecutable = false	We always want to generate a class library, not an executable
Treat Warnings as Errors	Supplied as function arg to CompileAssembly
WarningLevel = 4	Set the warning level so that we will actually receive warnings
IncludeDebugInformation = `true`	Always generate debug information as it takes minimal time

Add compiler flag /optimize if we aren’t in DEBUG
If not supplied any ReferencedAssemblies, force referencing System.dll and System.Core.dll
Otherwise just use our supplied ReferencedAssemblies
Always add UnrealBuildTool and DotNETUtilities as ReferencedAssemblies
Add compiler args for preprocessor definitions /define:
Use DotNet C# CSharpCodeProvider with CompilerVersion 4.0 to compile our Assembly using our CompileParameters
Display any warnings or errors, throwing if any errors.
Clean up temporary files.

Save out new Assembly Manifest file containing all the source files we used to compile our Assembly
Load Assembly into Application memory
Map Module Names to Module Files and Target Names to Target Files
Print any Obsolete or Deprecated code in our RulesAssembly module build classes

Creating the Engine `RulesAssembly` Object

A RulesAssembly object representing the rules of assembling the Engine.

Consume Engine RulesScope, a list of all Engine and Engine Platform Extension directories, the Assembly Prefix (UE4), list of included Plugins, whether to use precompiled engine libraries (forced to if using an Engine build), whether to skip compiling for this assembly, and any RulesAssembly parent
Add to RulesScope: Engine -> Engine Plugins -> Engine Programs
Create a default ModuleRulesContext to associate with all modules in this assembly
- Scoped to Engine, uses Engine directory as DefaultOutputBaseDir, shared PCH enabled
Builds a dictionary of all shared engine module files excluding the Programs directory mapped to a ModuleRulesContext
1. Iterates all consumed root directories, should be the Engine directory as well as Engine Platform Extensions i.e. Engine\Platforms\XXX
  1. Adds all Engine modules within Runtime, Developer, Editor, and ThirdParty subdirectories within Source subdirectory
  2. All modules have a UHTModuleType associated with their Runtime, Developer, Editor, or ThirdParty location
  3. Grabs the .Build.cs files of these modules to use when mapping to a ModuleRulesContext
  4. 558 total out of the box with UE4.25
2. Iterates consumed list of Plugin directories
  1. Grabs all Plugin .Build.cs files and maps them to a new ModuleRulesContext the same as default but scoped to Engine Plugins
3. Determine OutputAssemblyPath, default to Engine folder i.e. D:\UE425\Engine
  1. If Engine is installed, use %LocalAppData%\UnrealEngine\{MajorVersion}.{MinorVersion} i.e. ...\UnrealEngine\4.25
4. Determine RulesAssembly and ProgramRulesAssembly`` file paths
  1. OutputAssemblyPath\Intermediate\Build\BuildRules\{AssemblyPrefix}{Program}Rules.{AssemblyFrameworkExtension}
  2. Example: D:\UE425\Engine\Intermediate\Build\BuildRules\UE4Rules.dll and ...\UE4ProgramRules.dll
5. Create RulesAssembly as well as a ProgramRulesAssembly``
6. Return ProgramsRulesAssembly`` as Engine assembly as it contains build rules for both `Engine` Programs and `Engine` Modules

`TargetRules Object`

Created from a RulesAssembly object. TargetRules is a data structure that contains the rules for defining a target (application/executable). RequiresUniqueBuildEnvironment attributes must be shared between all targets when building in a shared build environment.

#TODO: Write a script to generate this shit

All Bold Fields are RequiresUniqueBuildEnvironment attributes and must have identical values between all targets when using shared environments.

Internal Field Name	Config?	Command-line Arg?	Description
Name			The name of this target.
File			File containing this target
Platform			Platform that this target is being built for.
Configuration			The configuration being built.
Architecture			Architecture that the target is being built for (or an empty string for the default).
ProjectFile			Path to the project file for the project containing this target.
Version			The current build version
Type			The type of target (UnrealBuildTool.TargetType Game, Editor, Client, Server Program)
`DefaultBuildSettings`			Specifies the engine version to maintain backwards-compatible default build settings with (eg. DefaultSettingsVersion.Release_4_23, DefaultSettingsVersion.Release_4_24). Specify DefaultSettingsVersion.Latest to always use defaults for the current engine version, at the risk of introducing build errors while upgrading.
ConfigValueTracker			Tracks a list of config values read while constructing this target
bUsesSteam			Whether the target uses Steam.
bUsesCEF3			Whether the target uses CEF3.
bUsesSlate			Whether the project uses visual Slate UI (instead of the low level windowing/messaging, which is always available).
bUseStaticCRT			Forces linking against the static CRT. This is not fully supported across the engine due to the need for allocator implementations to be shared (for example), and TPS libraries to be consistent with each other, but can be used for utility programs.
bDebugBuildsActuallyUseDebugCRT			Enables the debug C++ runtime (CRT) for debug builds. By default we always use the release runtime, since the debug version isn't particularly useful when debugging Unreal Engine projects
bLegalToDistributeBinary			Whether the output from this target can be publicly distributed, even if it has dependencies on modules that are in folders with special restrictions (eg. CarefullyRedist, NotForLicensees, NoRedist).
UndecoratedConfiguration			Specifies the configuration whose binaries do not require a "-Platform-Configuration" suffix.
bBuildAllModules		-AllModules	Build all the modules that are valid for this target type. Used for CIS / making installed engine builds.
BuildPlugins		-BuildPlugin=	Additional plugins that are built for this target type but not enabled.
AdditionalPlugins			A list of additional plugins which need to be included in this target. This allows referencing non-optional plugin modules which cannot be disabled, and allows building against specific modules in program targets which do not fit the categories in ModuleHostType.
EnablePlugins		-EnablePlugin=	Additional plugins that should be included for this target.
DisablePlugins		-DisablePlugin=	List of disabled plugins. Project may reference them, so mark as optional to avoid runtime failing.
PakSigningKeysFile			Path to the set of pak signing keys to embed in the executable.
SolutionDirectory			Allows a Program Target to specify its own solution folder path.
bBuildInSolutionByDefault			Whether the target should be included in the default solution build configuration
bShouldCompileAsDLL		-CompileAsDll	Whether this target should be compiled as a DLL. Requires LinkType == TargetLinkType.Monolithic.
ExeBinariesSubFolder			Subfolder to place executables in, relative to the default location.
GeneratedCodeVersion			Allow target module to override UHT code generation version.
bEnableMeshEditor			Whether to enable the mesh editor.
bCompileChaos		-NoCompileChaos, -CompileChaos	Whether to compile the Chaos physics plugin.
bUseChaos		-NoUseChaos, -UseChaos	Whether to use the Chaos physics interface. Disables APEX and NvCloth
bUseChaosChecked			Whether to compile in checked chaos features for debugging
bUseChaosMemoryTracking			Whether to compile in chaos memory tracking features
bCustomSceneQueryStructure			Whether scene query acceleration is done by UE4. The physx scene query structure is still created, but we do not use it.
bCompilePhysX			Whether to include PhysX support.
bCompileAPEX	.ini		Whether to include PhysX APEX support.
bCompileNvCloth			Whether to include NvCloth.
bCompileICU	.ini		Whether to include ICU unicode/i18n support in Core.
bCompileCEF3	.ini		Whether to compile CEF3 support.
bCompileISPC			Whether to compile using ISPC.
bBuildRequiresCookedData			Compile code related to building assets? Consoles cannot build assets. Desktop platforms can. By default build requires cooked data if target is Game, Client, or Server. Can be overridden in any case.
bBuildWithEditorOnlyData			Compile WITH_EDITORONLY_DATA? `True` if type is Editor or Program, can be overridden in any case
bBuildDeveloperTools			Whether to compile the developer tools. `True` if compiling against Engine AND (we are either Type Editor, Type Program, or neither Test and Shipping configs)
bForceBuildTargetPlatforms			Whether to force compiling the target platform modules, even if they wouldn't normally be built.
bForceBuildShaderFormats			Whether to force compiling shader format modules, even if they wouldn't normally be built.
bCompileCustomSQLitePlatform	.ini		Compile SQLite using custom "Unreal" platform (`true`), or using native platform (false).
bUseCacheFreedOSAllocs	.ini		Whether to utilize cache freed OS allocs with MallocBinned
bCompileAgainstEngine			Enabled if including the engine. Disabled when building standalones only linking with Core.
bCompileAgainstCoreUObject			Enabled if including CoreUObject. Disabled when building standalones only linking with Core.
bCompileAgainstApplicationCore			Enabled for builds that need to initialize the ApplicationCore module.
bCompileRecast	.ini		Whether to compile Recast navmesh generation.
bCompileSpeedTree	.ini		Whether to compile SpeedTree support.
bForceEnableExceptions			Enable exceptions for all modules.
bUseInlining	.xml		Enable inlining for all modules.
bForceEnableRTTI		-rtti	Enable RTTI for all modules.
bWithServerCode			Compile server-only code. Default `true` except when Type is Client
bWithPushModel			When enabled, Push Model Networking will be used on the server. This can help reduce CPU overhead of networking, at the cost of more memory.
bCompileWithStatsWithoutEngine			Whether to include stats support even without the engine.
bCompileWithPluginSupport	.ini		Whether to include plugin support.
bIncludePluginsForTargetPlatforms			Whether to allow plugins which support all target platforms. Default `true` if Type is Editor
bCompileWithAccessibilitySupport			Whether to allow accessibility code in both Slate and the OS layer.
bWithPerfCounters	.ini		Whether to include PerfCounters support.
bWithLiveCoding			Whether to enable support for live coding
bUseDebugLiveCodingConsole	.xml		Whether to enable support for live coding
bWithDirectXMath			Whether to enable support for DirectX Math
bUseLoggingInShipping			Whether to turn on logging for test/shipping builds.
bLoggingToMemoryEnabled			Whether to turn on logging to memory for test/shipping builds.
bUseLauncherChecks			Whether to check that the process was launched through an external launcher.
bUseChecksInShipping			Whether to turn on checks (asserts) for test/shipping builds.
bCompileFreeType	.ini		`True` if we need FreeType support.
bCompileForSize	.ini		`True` if we want to favor optimizing size over speed.
bForceCompileDevelopmentAutomationTests			Whether to compile development automation tests.
bForceCompilePerformanceAutomationTests			Whether to compile performance automation tests.
bEventDrivenLoader			If `true`, event driven loader will be used in cooked builds. @todoio This needs to be replaced by a runtime solution after async loading refactor.
bUseXGEController	.xml		Whether the XGE controller worker and modules should be included in the engine build. These are required for distributed shader compilation using the XGE interception interface.
bIWYU		-IWYU	Enables "include what you use" by default for modules in this target. Changes the default PCH mode for any module in this project to PCHUsageMode.UseExplicitOrSharedPCHs.
bEnforceIWYU			Enforce "include what you use" rules; warns if monolithic headers (Engine.h, UnrealEd.h, etc...) are used, and checks that source files include their matching header first.
bHasExports			Whether the final executable should export symbols. Defaults to `true` if LinkType is Modular
bPrecompile		-Precompile	Make static libraries for all engine modules as intermediates for this target.
bEnableOSX109Support			Whether we should compile with support for OS X 10.9 Mavericks.
bIsBuildingConsoleApplication			`True` if this is a console application that's being built.
bBuildAdditionalConsoleApp			Creates an additional console app. Hack for Windows: not possible to conditionally inherit a parent's console Window based on how the app is invoked; have to link the same executable with a different subsystem setting. Default is `true` if Editor
bDisableSymbolCache			`True` if debug symbols that are cached for some platforms should not be created.
bUseUnityBuild	.xml	-DisableUnity	Whether to unify C++ code into larger files for faster compilation.
bForceUnityBuild	.xml	-ForceUnity	Whether to force unify C++ code into larger files for faster compilation.
bUseAdaptiveUnityBuild	.xml		If `true`, use SourceWorkingFileSets to exclude files being iterated on from unity builds
bAdaptiveUnityDisablesOptimizations	.ini		Disable optimization for files that are in the adaptive non-unity working set.
bAdaptiveUnityDisablesPCH	.ini		Disables force-included PCHs for files that are in the adaptive non-unity working set.
bAdaptiveUnityDisablesPCHForProject	.ini		Whether to disable force-included PCHs for project source files in the adaptive non-unity working set. Defaults to bAdaptiveUnityDisablesPCH;
bAdaptiveUnityCreatesDedicatedPCH	.ini		Creates a dedicated PCH for each source file in the working set, allowing faster iteration on cpp-only changes.
bAdaptiveUnityEnablesEditAndContinue	.ini
MinGameModuleSourceFilesForUnityBuild	.ini		The number of source files in a game module before unity build will be activated for that module. This allows small game modules to have faster iterative compile times for single files, at the expense of slower full rebuild times. This setting can be overridden by the bFasterWithoutUnity option in a module's Build.cs file.
ShadowVariableWarningLevel		-ShadowVariableErrors	Forces shadow variable warnings to be treated as errors on platforms that support it.
UnsafeTypeCastWarningLevel	.ini		Indicates what error level to treat unsafe type casts as on platforms that support it
bUndefinedIdentifierErrors	.ini		Forces use of undefined identifiers in conditional expressions to be treated as errors.
bUseFastMonoCalls	.xml	-NoFastMonoCalls, -FastMonoCalls	New Mono drivers have "fast calls" replacing D3d functions
bUseFastSemanticsRenderContexts	.xml		New Xbox driver supports a "fast semantics" context type.
NumIncludedBytesPerUnityCPP	.xml		Approximate number of bytes of C++ code to target for inclusion in a unified C++ file.
bStressTestUnity	.xml	-StressTestUnity	Whether to stress test the C++ unity build robustness by including all C++ files files in a project from a single unified file.
bForceDebugInfo		-ForceDebugInfo	Whether to force debug info to be generated.
bDisableDebugInfo	.xml	-NoDebugInfo	Globally disable debug generation; see DebugInfoHeuristics.cs for config and platform options.
bOmitPCDebugInfoInDevelopment	.xml		Whether to disable debug info on PC in development builds (for faster developer iteration, as link times are extremely fast with debug info disabled).
bUsePDBFiles	.xml	-NoPDB	Whether PDB files should be used for Visual C++ builds.
bUsePCHFiles	.xml	-NoPCH	Whether PCH files should be used.
bPreprocessOnly		-PreProcess	Whether to just preprocess source files for this target, and skip compilation
MinFilesUsingPrecompiledHeader	.xml		Minimum number of files that use a PCH before it will be created and used.
bForcePrecompiledHeaderForGameModules	.xml		When enabled, a precompiled header is always generated for game modules, even if there are only a few source files in the module. This greatly improves compile times for iterative changes on a few files in the project, at the expense of slower full rebuild times for small game projects. This can be overridden by setting MinFilesUsingPrecompiledHeader in a module's Build.cs file.
bUseIncrementalLinking	.xml	-NoIncrementalLinking, -IncrementalLinking	Whether to use incremental linking or not. Incremental linking can yield faster iteration times when making small changes. Currently disabled by default because it tends to behave a bit buggy on some computers (PDB-related compile errors).
bAllowLTCG	.xml	-LTCG	Whether to allow the use of link time code generation (LTCG).
bPGOProfile	.xml	-PGOProfile	Whether to enable Profile Guided Optimization (PGO) instrumentation in this build.
bPGOOptimize	.xml	-PGOOptimize	Whether to optimize this build with Profile Guided Optimization (PGO).
bAllowASLRInShipping	.xml		Use of ASLR (address space layout randomization) if supported. Applies to shipping.
bSupportEditAndContinue	.xml		Whether to support edit and continue. Only works on Microsoft compilers.
bOmitFramePointers	.xml		Whether to omit frame pointers or not. Disabling is useful for e.g. memory profiling on the PC.
bUseMallocProfiler	.xml		Enable memory profiling, sets USE_MALLOC_PROFILER=1 and forces bOmitFramePointers to false.
bUseSharedPCHs	.xml	-NoSharedPCH	Enables "Shared PCHs", a feature which significantly speeds up compile times by attempting to share certain PCH files between modules that UBT detects is including those PCH's header files.
bUseShippingPhysXLibraries	.xml		`True` if Development and Release builds should use the release configuration of PhysX/APEX.
bUseCheckedPhysXLibraries	.xml		`True` if Development and Release builds should use the checked configuration of PhysX/APEX.
bCheckLicenseViolations	.xml		Tells the UBT to check if module currently being built is violating EULA.
bBreakBuildOnLicenseViolation	.xml		Tells the UBT to break build if module currently being built is violating EULA.
bUseFastPDBLinking	.xml	-FastPDB	Whether to use the :FASTLINK option when building with /DEBUG to create local PDBs on Windows. Fast, but currently seems to have problems finding symbols in the debugger.
bCreateMapFile	.xml	-MapFile	Outputs a map file as part of the build.
bAllowRuntimeSymbolFiles	.xml		`True` if runtime symbols files should be generated as a post build step for some platforms. These files are used by the engine to resolve symbol names of callstack backtraces in logs.
BundleVersion		-BundleVersion	Bundle version for Mac apps.
bDeployAfterCompile		-SkipDeploy, -Deploy	Whether to deploy the executable after compilation on platforms that require deployment.
bAllowRemotelyCompiledPCHs	.xml		When enabled, allows XGE to compile pre-compiled header files on remote machines. Otherwise, PCHs are always generated locally.
bCheckSystemHeadersForModification	.xml		Whether headers in system paths are checked for modification when determining outdated actions.
bDisableLinking		-NoLink	Whether to disable linking for this target.
bFormalBuild		-Formal	Indicates that this is a formal build, intended for distribution. Largely useless.
bFlushBuildDirOnRemoteMac	.xml	-FlushMac	Whether to clean Builds directory on a remote Mac before building.
bPrintToolChainTimingInfo	.xml	-Timing	Whether to write detailed timing info from the compiler and linker.
bParseTimingInfoForTracing		-Tracing	Whether to parse timing data into a tracing file compatible with chrome://tracing.
bPublicSymbolsByDefault	.xml	-PublicSymbolsByDefault	Whether to expose all symbols as public by default on POSIX platforms
ToolChainName		-ToolChain	Overrides toolchain for this target. Must match name of a class in the UBT assembly.
bDisableUnverifiedCertificates			Whether to allow engine configuration to determine if we can load unverified certificates.
bAllowGeneratedIniWhenCooked			Whether to load generated ini files in cooked build, (GameUserSettings.ini loaded either way)
bAllowNonUFSIniWhenCooked			Whether to load non-ufs ini files in cooked build, (GameUserSettings.ini loaded either way)
bLegacyPublicIncludePaths			Add all the public folders as include paths for compiling. `True` if older than V2 BuildSettings
CppStandard		-CppStd	Which C++ standard to use for compiling this target. Can be Default, Cpp14, Cpp17, Latest
bNoManifestChanges		-NoManifestChanges	Do not allow manifest changes when building this target. Used to cause earlier errors when building multiple targets with a shared build environment.
BuildVersion		-BuildVersion	The build version string
LinkType		-Modular, -Monolithic	Specifies how to link modules in this target (monolithic or modular). Currently protected for backwards compatibility. Call GetLinkType() accessor until support for the deprecated ShouldCompileMonolithic() override has been removed.
GlobalDefinitions		-Define:	Macros to define globally across the whole target.
ProjectDefinitions			Macros to define across all macros in the project.
ExportPublicHeader			Specifies the path to write a header containing public definitions for this target. Useful when building a DLL to be consumed by external build processes.
ExtraModuleNames			List of additional modules to be compiled into the target.
ManifestFileNames		-Manifest	Path to a manifest to output for this target
DependencyListFileNames		-DependencyList	Path to a list of dependencies for this target, when precompiling
BuildEnvironment		-UniqueBuildEnvironment, -SharedBuildEnvironment	Specifies the build environment for this target.
bOverrideBuildEnvironment		-OverrideBuildEnvironment	Ignore violations to the SBE (eg. editor targets modifying definitions)
PreBuildSteps			Specifies a list of steps which should be executed before this target is built, in the context of the host platform's shell. Variables expanded: $(EngineDir), $(ProjectDir), $(TargetName), $(TargetPlatform), $(TargetConfiguration), $(TargetType), $(ProjectFile).
PostBuildSteps			Specifies a list of steps which should be executed after this target is built, in the context of the host platform's shell. Variables expanded: $(EngineDir), $(ProjectDir), $(TargetName), $(TargetPlatform), $(TargetConfiguration), $(TargetType), $(ProjectFile).
AdditionalBuildProducts			Specifies additional build products produced as part of this target.
AdditionalCompilerArguments		-CompilerArguments=	Additional arguments to pass to the compiler
AdditionalLinkerArguments		-LinkerArguments=	Additional arguments to pass to the linker
DisableUnityBuildForModules			List of modules to disable unity builds for
EnableOptimizeCodeForModules			List of modules to enable optimizations for
DisableOptimizeCodeForModules			List of modules to disable optimizations for
GeneratedProjectName			When GPF, is the name of the project file to use when there are multiple targets of the same type.
AndroidPlatform			Android-specific target settings.
IOSPlatform			IOS-specific target settings.
LuminPlatform			Lumin-specific target settings
LinuxPlatform			Linux-specific target settings.
MacPlatform			Mac-specific target settings.
PS4Platform			PS4-specific target settings.
SwitchPlatform			Switch-specific target settings.
WindowsPlatform			Windows-specific target settings. Requires 'this' parameter; initialized in constructor
XboxOnePlatform			Xbox One-specific target settings.
HoloLensPlatform			HoloLens-specific target settings.

Creating a `TargetRules Object`

RulesAssembly.CreateTargetRules consumes a TargetDescriptor
Find RulesAssembly that contains the TargetRules Object Type for our Target
1. If TargetNameToTargetFileMap doesn’t contain this Target, bubble up this call to the parent RulesAssembly (ProjectRules -> Rules)
2. If no RulesAssembly contains the target, we throw a build exception
Assemble TargetRules Type Name, which is just our Target’s name and the string “Target”, i.e. UnrealHeaderToolTarget
1. You should note here that this matches the enforced TargetRules objects we define in our .target.cs files, because that what those are
Grab DotNet C# Type with TargetRules Type Name through crazy built in C# DotNet Assembly magic, throws if this fails
Create an uninitialized instance of this Type, basically an instance of a TargetRules Object that has not ran its constructor
Set DefaultBuildSettings on our new TargetRules Object if DefaultBuildSettings has a value
Then run the TargetRules Object Constructor, this is what we’re defining when we create our Target’s .target.cs file
Sets the TargetFile (path to the .target.cs file this TargetRules represents) for the new TargetRules Object
Sets up default definitions based on Target Type, i.e. UE_GAME=1, UE_CLIENT=1, UE_EDITOR=1, UE_SERVER=1, USE_NULL_RHI=1
Make sure we have a LinkType set by this point, for Program targets this is manually set in the .target.cs constructor. Monolithic or Modular.
Make sure that if we are not using a shared build environment we are also not using an installed version of the Engine
Make sure bCompileAgainstCoreUObject is true if bCompileAgainstEngine is true
Make sure bBuildWithEditorOnlyData is true if is true
Make sure bDisableDebugInfo and bOmitPCDebugInfoInDevelopment are both false if bForceDebugInfo is true
Make sure bOmitFramePointers is false and USE_MALLOC_PROFILER=1 defined if bUseMallocProfiler is true
Make sure DISABLE_GENERATED_INI_WHEN_COOKED=1 is defined if bAllowGeneratedIniWhenCooked is true
Make sure DISABLE_NONUFS_INI_WHEN_COOKED=1 is defined if bAllowNonUFSIniWhenCooked is true
Make sure DISABLE_UNVERIFIED_CERTIFICATE_LOADING=1 is defined if bDisableUnverifiedCertificates is true
Platform then validates this TargetRules object
1. For example, Android forces bCompilePhysX = true and bCompileAPEX = false, presumably because APEX isn’t Android supported

UEBuildModule Object

A UEBuildModule represents a unit of code compilation and linking. Requires a RulesObject (.Target.cs) to construct. Represented by our .Build.cs files. Requires a Readonly TargetRules to be constructed. Our UEBuildTarget uses its RulesAssembly (represents our compiled .Build.cs and .Target.cs) to create our UEBuildModule

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore
A UEBuildModule is basically a wrapper around our ModuleRules, which is an instance what we define in our .Build.cs file
After wrapping this ModuleRules file, a UEBuildModule also adds all the default include folders, i.e. Classes, Public, Private

Constructing a UEBuildModule

Requires a RulesObject (.Target.cs). Creates a ModuleRules object which represents our .Build.cs files. Requires a Readonly TargetRules to be constructed. UEBuildTarget uses its RulesAssembly to make our UEBuildModule.

Consumes Module Name, a readonly instance of our TargetRules instantiated from our .Target.cs, and a file path to our .Target.cs
RulesAssembly uses ModuleName to grab the .Build.cs file path for this module using its cached list of .Build.cs paths to modules
1. If this fails, we ask the RulesAssembly’s parent RulesAssembly recursively for the .Build.cs file path, throwing if never found
2. Consumed .Target.cs file path is only used for logging in this step if the ModuleRules type can not be found
3. Does this so that we know that we have loaded a ModuleRules with typed with our Module Name somewhere
Determine the .DotNet C# Type of our UEBuildModule
1. Uses .DotNet shenanigans to get the Type of our ModuleRules using ModuleName or UnrealBuildTool.Rules.ModuleName namespace
  1. // Temporary hack to avoid System namespace collisions, // @todo projectfiles: Make rules assemblies require namespaces.
  2. This is the UnrealHeaderTool in public class UnrealHeaderTool : ModuleRules
2. Look for a ModuleRules Type that overrides this module for our platform
  1. First try our supplied Platform, i.e. Win64 (would be UnrealHeaderTool_Win64)
  2. Then try the names of the Platform Groups that contain our Platform (i.e. UnrealHeaderTool_Desktop, _Microsoft, etc.)
    1. There can only be one valid Platform Group module that works for us, otherwise we throw an error
    2. If a Platform ModuleRules class exists, it basically stomps the base ModuleRules transparently
Create an uninitialized instance of our ModuleRules class with our determined Type
Disables Precompiling if Precompiling is set and we aren’t an Engine module or UE4Game
Sets bUsePrecompiled if this RulesAssembly has marked this module or target read-only
If our ModuleRules type is overridden (i.e. a Platform override _Win64), iterate our parent ModuleRules
1. // go up the type hierarchy (if there is a hierarchy), looking for any extra directories for the module
Finally grab and invoke our ModuleRules Constructor passing in our ReadOnlyTargetRules object
1. The constructor here actually is very trivial and just stores the passed in ReadOnlyTargetRules in a field named TargetRules
Then set a bunch of default values for our ModuleRules
1. If we are a C++ module that isn’t named Launch, mark that we require an IMPLEMENT_MODULE macro (bRequiresImplementModule)
2. If ProjectFile supplied, add all Module names listed in the ProjectFile to PrivateDependencyModuleNames
3. Remove trailing slash from Include and Library paths, as the ending slash can escape enclosing quotes for command line ops
4. Validate RulesObject
  1. Ensure that no Privately/Publicly depended Modules are also DynamicallyLoadedModules
  2. Ensure a PrivatePCHHeaderFile is set if not using shared PCHs.
    1. If no PCH is set, we will grab the first C++ source file we come across and use it’s first include as our PCH
5. // Clear bUsePrecompiled flag if compiling a foreign plugin; as it's treated like an engine module, it will default to true in installed builds.
6. // Get the base directory for paths referenced by the module. If the module's under the UProject source directory use that, otherwise leave it relative to the Engine source directory.
7. // Get the generated code directory. Plugins always write to their own intermediate directory so they can be copied between projects, shared engine intermediates go in the engine intermediate folder, and anything else goes in the project folder.
  1. For example, D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\UnrealHeaderTool
8. Allow all UEBuildPlatforms to modify our rules via UEBuildPlatform.ModifyModuleRulesForOtherPlatform
  1. This allows things like undisclosed platforms and being able to compile Android shaders on Windows for mobile previewing
Finally call InstantiateModule passing our RulesObject and GeneratedCodeDirectory to create, cache, and return our UEBuildModule

UEBuildBinary

Represents a binary built by UBT.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

CPPCompileEnvironment

Encapsulates the environment that a C++ file is compiled in.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

LinkEnvironment

Encapsulates the environment that is used to link object files.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

UEToolChain

#TODO: This is really important to cover.

Encapsulates the environment that a C++ file is compiled in.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

On Windows, VCToolChain (child of ISPCToolChain) is used unless you are using the PVS Studio Static Code Analyzer
UEToolChain defines things like building the command-line args to pass to C++ compilers and linkers and such

UHTModuleType

Type of module. Mirrored in UHT as EBuildModuleType. This should be sorted by the order in which we expect modules to be built.

Program
EngineRuntime
EngineUncooked
EngineDeveloper
EngineEditor
EngineThirdParty
GameRuntime
GameUncooked
GameDeveloper
GameEditor
GameThirdParty

UHTManifest

Information to be sent to the UnrealHeaderTool.

Field Name	Description
bool IsGameTarget	// True if the current target is a game target
string RootLocalPath	// The engine path on the local machine
string TargetName	// Name of the target currently being compiled
string ExternalDependenciesFile	// File to contain additional dependencies that the generated code depends on
List Modules

UHTManifest.Module

Information about a Module stored in a UHTManifest.

Field Name	Description
string Name
string ModuleType
string BaseDirectory
string IncludeBase	// The include path which all UHT-generated includes should be relative to
string OutputDirectory
List ClassesHeaders
List PublicHeaders
List PrivateHeaders
string GeneratedCPPFilenameBase
bool SaveExportedHeaders
EGeneratedCodeVersion UHTGeneratedCodeVersion

Actually Building A Target

1. Startup and Determining Build Mode

Sets current working directory to the Engine Source directory
Determine GlobalOptions using Command Line Args
- By default, “Mode” is a null string
- Different modes can be invoked using the -Mode=[Name] argument on the command line, where [Name] is determined by the ToolModeAttribute on a ToolMode derived class. The log system will be initialized before calling the mode, but little else.
- All valid ToolMode classes are tagged with a ToolModeAttribute, which defines their “Mode” lookup string
  - ToolModeAttribute can define options that configure UnrealBuildTool, defined as an enum with BitField/BitMask values
    1. None
    2. StartPrefetchingEngine
    3. XMLConfig
    4. BuildPlatforms
    5. BuildPlatformsHostOnly
    6. BuildPlatformsForValidation
    7. SingleInstance
    8. ShowExecutionTime
- If “Mode” remains null after processing args, default mode is BuildMode
- BuildMode has the following options set:
  1. XmlConfig
  2. BuildPlatforms
  3. SingleInstance
  4. StartPrefetchingEngine
  5. ShowExecutionTime

2. Startup and Configuration Loading

Engine content prefetching
1. Stores metadata about every file in the form of a map that stores the UnrealBuildTool DirectoryItem and System.IO.FileItem info of a file
2. System.IO.FileItem knows file directory, path, length, existence
Fetch XML Configuration Data (UnrealBuildTool_Mutex_XmlConfig)
1. XML Config files can be specified by -XmlConfigCache=
  1. Force use of the cached XML config without checking if it's valid (useful for remote builds)
2. If no config cache specified, default is Engine\Intermediate\Build\XMLConfigCache.bin, validate cache (see slide on Config Validation)
Locks SingleInstance Mutex (UnrealBuildTool_Mutex)
Register All Build Platforms, not including not installed platforms, not limited to host only platform
1. Initializes Installed Platform (Invokes a Static Constructor)
  1. BuildPlatformInfo.Current is a Singleton initialized to the current build platform child class via GetRuntimePlatform() determined by DotNet PlatformID Environment.OSVersion.Platform
  2. Linux Vs. Mac determined by File.Exists("/System/Library/CoreServices/SystemVersion.plist");
2. Builds Config Hierarchy for InstalledPlatform with TYPE set to “Engine” (Puts the Engine in DefaultEngine.ini / BaseEngine.ini)
  1. Creates a list of ConfigFiles which are just a list of ConfigFileSections, which are just a name and a list of ConfigLines:
    1. ConfigLines are a Name, Value, and an Action
    2. Available Config Actions are Set, Add, RemoveKey, RemoveKeyValue
3. Looks for Config Command Line Arg “-ini:Engine:”
  1. If found, this simply just adds the given .ini file as the final config on the Config Hierarchy
Also attempts to parse InstalledPlatforms if configuration value InstalledPlatforms.HasInstalledPlatformInfo exists and has a meaningful value
1. I don’t know what this is used for, appears to only affect Installed Builds which I currently am not interested in diving into

3. BuildPlatformFactory Discovery

Register all tool chains and build platforms that are present. Iterate through all C# classes filtering for non-abstract UEBuildPlatformFactorys

Creates a temp instance of iterated UEBuildPlatformFactory
If bHostPlatformOnly == true, filter for only platforms that match host platform. This appears to be usually false
InstalledPlatform determines whether iterated BuildPlatform is valid, always valid if including all non-installed platforms
1. Does a Platform config check
2. Does a Project config check, either desired build project type is Any, BuildPlatform is marked as Any, or types match
3. If valid, calls Factory’s RegisterBuildPlatforms()
  1. Factories generally initialize and validate their SDKs in this step, either setting up a AutoSDK or ManualSDK
    1. For example, LinuxPlatformFactory tries to verify that it has Clang, GCC, or other toolchain
Example: WindowsPlatformFactory
1. Windows simply always returns true that it has a valid manual sdk installed
2. Registers Win32 and Win64 build targets
3. Adds both targets to Windows, Microsoft, and Desktop platform groups
BuildPlatforms loaded “out of the box” with stock Unreal Engine 4
1. XXXPlatformFactory
2. MacPlatformFactory
3. TVOSPlatformFactory
4. AndroidPlatformFactory
5. HoloLensPlatformFactory
6. IOSPlatformFactory
7. LinuxPlatformFactory
8. LuminPlatformFactory
9. WindowsPlatformFactory

4. Executing The BuildTool ToolMode

Now that our UnrealBuildTool has loaded all configuration and platform info, it will now run the BuildMode ToolMode’s Execute function which appears to be the real meat and potatoes of building a target in the Unreal Engine ecosystem.

BuildMode applies any Command Line Arg options to itself. These are the available options, all default to false

Args	Description
-IgnoreJunk	Whether to skip checking for files identified by the junk manifest.
-SkipBuild	Skip building; just do setup and terminate.
-XGEExport	Whether we should just export the XGE XML and pretend it succeeded
-NoEngineChanges	Do not allow any engine files to be output (used by compile on startup functionality)
-WriteOutdatedActions=	Whether we should just export the outdated actions list (requires a file path)

Initializes Trace Logging
Sets up UnrealBuildTool.InitialEnvironment, grabbing ALL system Environment variables
Applies XML Config data to itself. Out of the box, there appears to be zero config values on an initial UnrealHeaderTool build.
If not configured otherwise, initializes a logger to Engine\Programs\UnrealBuildTool\Log.txt
1. Can be disabled with -NoLog
2. Can have suffixes applied to path by using -LogSuffixes= which consumes an array of strings.
  1. -LogSuffixes="Ass, Foo" would output to Engine\Programs\UnrealBuildTool\Log_Ass_Foo.txt
Creates BuildConfiguration object, a long list of settings that applies to all builds, see slides about BuildConfiguration configuration
Check to see if root path isn’t too long, but only on Windows, because Windows is an inferior platform
Deletes Junk, Loads JunkManifest.txt file and removes all junk files/folders defined in it. Currently don’t know what generates a JunkManifest?
Parse all Target Descriptors
Codepath separates if needing to handle remote building if remote building for Mac
Processes and builds all Target Descriptors

5. Determining Target Descriptors

A TargetDescriptor describes all of the information needed to initialize a UEBuildTarget object. Uses CommandLineArguments struct in DotNetUtilities.

List of TargetDescriptors generated by parsing the command line
1. If Target List Args are present (-TargetList, -Target):
  1. Looks for a -TargetList= arg which accepts a list of file paths that represent a list of targets
    1. Each line in a target list represents a build target and is processed through the same command line parser
    2. When re-ran through the command line parser, it adds the target list line args to the args initially passed into UnrealBuildTool but with the -TargetList and -Target args stripped
  2. Looks for a -Target= arg which accepts a list of build targets and command line args, basically an inlined TargetList
  3. This parses the command line recursively until it has a command line for building that includes only a single target
    1. All ParseCommandLine calls filter into ParseSingleCommandLine which does the actual TargetDescriptor parsing
2. Otherwise treat the entire initial command line argument string as build target info and go right into ParseSingleCommandLine
ParseSingleCommandLine (Build a TargetDescriptor based off of command line arg info or target string from target list:
1. First tries to get Project file path by checking for -Project=, can be null, can only specify a single project file per target, if specified must exist
2. Looks for arguments without ‘-’ prefix
  1. Also looks for a .uproject file reference as an unnamed arg value by seeing if arg ends with text “.uproject”
  2. If not a project file, tries to parse args as Platform references
    1. Arg must match a platform that the UnrealBuildTool is aware of during startup configuration
    2. Multiple platforms can be specified by separating with the “+” character
  3. If not a Platform reference, tries to parse args as an UnrealTargetConfiguration
    1. Valid UnrealTargetConfigurations are Unknown, Debug, DebugGame, Development, Shipping, Test
    2. Multiple configurations also specifiable with the “+” character
  4. If not an UnrealTargetConfiguration, assume arg without a ‘-’ prefix is a Target name
3. Must have at least one Platform and one Configuration to be able to move forward
4. Iterate and Validate all parsed Platforms, but only if using an installed build of the Engine, otherwise all Platforms assumed valid
  1. Determine Architecture for the platform (An Architecture is really just a Platform modifier? Usually an Empty String)
    1. If no Architecture is specified with -Architecture=, grabs the default Architecture from configured Platform info
      1. Multiple Architectures supported if using the ‘+’ character, similar to Platforms and Configurations
      2. If you manually specify an Architecture, you are required to also specify a -TargetType list?
  2. Iterate all parsed Configurations, goal is to ultimately build a list of TargetNames which will then allow for making TargetDescriptors
    1. Build list of Target Names
      1. If ProjectFile is specified, iterate all -TargetType args
        
        A valid TargetType is one of Game, Editor, Client, Server, Program
        
        Determines TargetName from ProjectFile and TargetType and adds this to our list of targets we want to build
        
        Does some crazy shit with C#/DotNet and actually compiles a new DotNet assembly with our Target .cs classes
        
        Loads the newly compiled RulesAssembly DotNet assembly and uses RTTI stuff to grab the right target name
    2. If ProjectFile is null, we should already have a TargetName, likely from our initial UBT command line args, i.e. UnrealHeaderTool
      1. Remember, args not ‘-’ prefixed in any target command line iteration that aren’t a Platform/Configuration are treated as TargetNames
    3. Build list of TargetDescriptors using list of TargetNames
      1. Attempt to find a ProjectFile for all TargetNames if ProjectFile still null, iterates all TargetNames:
        
        Determine BaseDirectories by enumerating all .uprojectdirs files in Engine Root folder (i.e. D:\UE425), restricted to subdirs
        
        Enumerates through all .uproject files found in all BaseDirectories recursively
        
        Looks for ProjectFile with matching Target file located in ProjectFile’s Source and Intermediate\Source paths
      2. Now have all required information for a TargetDescriptor to be added to our list of targets to build!
        
        ProjectFile (Can be Null), TargetName, Platform, Configuration, Architecture, Arguments
5. All TargetDescriptors built for all targets for all desired target types for all desired configurations for all desired architectures for all desired platforms

6. Building our SourceFileWorkingSet

Iterate every target and prefetch metadata about all Source and Plugin\Source folders defined by any target ProjectFiles
Create a SourceFileWorkingSet from the Root Engine folder as well as a list of all project paths found in step 1
1. Behavior dependent on current source file provider, which could be one of the following: None, Default, Perforce, or Git.
  1. If provider is None, or we’re set to generate project files, we return an empty SourceFileWorkingSet
  2. If provider is Git, we try to create a GitSourceFileWorkingSet, consuming root engine path and all relevant project paths
    1. Checks for a .git file in the root directory, if found, initializes a GitSourceFileWorkingSet from this .git file
      1. Runs git status --porcelain to get all modified source files to exclude from a Unity build
    2. If a root .git isn’t found, iterate .git files in all relevant project folders, creates GitSourceFileWorkingSets
      1. GitSourceFileWorkingSets can have sub-GitSourceFileWorkingSets that represent Git submodules
  3. If provider is Perforce, we try to create a PerforceSourceFileWorkingSet
    1. Filter out all source files that end in .gen.cpp, as well as filter out all files that are marked read-only
      1. This creates a SourceFileWorkingSet that only has writable source files without actually contacting Perforce
  4. If provider is Default, we attempt to create a GitSourceFileWorkingSet, and if that fails, a PerforceSourceFileWorkingSet
Finally invokes the main Build function defined in our BuildMode ToolMode
1. Consumes all of our TargetDescriptors, our global BuildConfiguration object, our SourceFileWorkingSet, any additional build command line arg configurations, a optional file path to write outdated actions to

7. Loading a Makefile if Present

Finally invoking the main Build function defined in our BuildMode ToolMode. This function attempts to load or create a Makefile per target.

If we’re using UBT Makefiles and we’re not compiling a single file, generate the location to a potentially pre-existing Makefile
1. Base directory is our relevant Project directory, but if passed in a null ProjectFile, our base directory is the Engine folder
2. BaseDirectory\Intermediate\Build{TARGETPLATFORM}{TARGETNAME}{TARGETCONFIGURATION}\Makefile.bin
  1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\Makefile.bin
Attempt to load a pre-existing Target Makefile at the determined Makefile file location
1. Returns load failure if Makefile does not exist
2. Returns load failure if Makefile timestamp older than Engine BuildVersion timestamp
3. Returns load failure in non-installed Engines if any Engine ProjectFile (i.e. UnrealHeaderTool) is newer than Target Makefile
4. If Target ProjectFile is not null:
  1. Returns load failure if Makefile is older than Project’s .uproject file
  2. Returns load failure if Makefile is older than Project’s generated C++ ProjectFile in Project\Intermediate\ProjectFiles
5. Returns load failure if Makefile is older than timestamp of the UnrealBuildTool executable
6. Returns load failure if any of the BuildConfiguration XMLConfigFile files are newer than our Makefile
7. Returns load failure if Makefile TargetMakefile version does not match the UnrealBuildTool TargetMakefile version
8. Returns load failure if Makefile arguments do not match command line arguments passed in from the UnrealBuildTool
9. Returns load failure if Makefile’s referenced config files are outdated or changed in any way as Project .inis can affect building
10. Returns load failure if build platform metadata has changed
  1. Allows platforms to define data that may or may not invalidate makefiles
  2. Currently only used on a out of the box Engine on Windows platforms for discovering changes to a Project’s .exe Icon
If Makefile is loaded, run pre-build scripts and additional checks to really fully ensure that our Makefile is still valid and up-to-date
1. If any pre-build scripts defined, run them, and immediately invalidate our Makefile because we don’t know what may have changed
2. Check to see if all source files are still valid for this Makefile
  1. Check to see if any source files or directories containing source files for our target have been added, removed, or modified
  2. Check to see if any External or Internal Dependencies have changed @TODO What is a Makefile dependency?
  3. Check to see that no new plugins have been added
  4. Check to see that no source header files with reflection markup have changed
    1. Regular Expression for Reflection Markup: ^\s*U(CLASS|STRUCT|ENUM|INTERFACE|DELEGATE)\b
  5. If adaptive unity build enabled, check to see if any files have been added, removed, or modified from our SourceFileWorkingSet
3. If we have finally determined that we were able to load a valid pre-existing Makefile:
  1. Set all environment variables desired by the Makefile
  2. Determine and run UnrealHeaderTool if required @TODO figure out
  3. Return our pre-existing Makefile
4. If we have no valid Makefile or our Makefile was invalidated for any reason, we will create a new Makefile now
5. In order to build a Makefile, we first need to create a UEBuildTarget object...

8. Creating a UEBuildTarget

A UEBuildTarget contains all the information needed to build a target. This is required to generate a Makefile.

Consumes a TargetDescriptor, whether to skip rules compiling, and whether to use precompiled static libraries for editor modules
Throws if Target Platform is unavailable or unbuildable
Create a RulesAssembly
1. If ProjectFile is supplied, we try to create a Project RulesAssembly @TODO: Document
2. Otherwise, create Engine RulesAssembly
  1. Return cached Engine RulesAssembly if one was already made prior in this instance of UnrealBuildTool
    1. Build a list of all Engine PluginInfos (304 of them out of the box on 4.25)
    2. Create an Engine scoped RulesScope with no scope parent
      1. A RulesScope is a LinkedList representing a flat hierarchy i.e. Project -> Engine -> GameFramework
      2. All children of a RulesScope are “in scope” or contained within the parent scope
      3. Dependencies of modules can only go deeper in scope, never upward into parent scopes
    3. Grabs the Engine and Engine Platform Extension directories
    4. Create the actual Engine RulesAssembly object (see slides on Creating an Engine RulesAssembly)
  2. If creating an Engine RulesAssembly failed because Target is an Enterprise Target, make an Enterprise RulesAssembly
3. If ForeignPlugin is supplied, create a Plugin RulesAssembly
Create a TargetRules object using our RulesAssembly (see the next large amount of slides)
If we can’t generate project files, check to see if TargetRules allows building for desired platform.
1. Supported platforms defined by SupportedPlatformsAttribute in Module .build.cs or based off of TargetType and PlatformClass
Make sure our desired BuildConfiguration is supported, either defined by SupportedConfigurationsAttribute or based off TargetType
1. Does a bunch of platform checking based on installed Engine builds, i.e. “throw Download support for building targets from the launcher.”
If using a shared environment, validate all RequiresUniqueBuildEnvironment attributes are identical
1. Target fields checked against the vanilla UE4Game, UE4Editor, UE4Client, UE4Server target rules based on target type.
If we’re precompiling, generate a list of all the files we depend on
1. Stored as Engine or Project\Intermediate\DependencyLists\TargetName\Configuration\Platform\DependencyList.txt
2. Saved as DependencyList-AllModules.txt if building all modules
If TargetDescriptor SingleFileToCompile is supplied, disable all Unity build options and disable linking
If TargetDescriptor ForeignPlugin is supplied, disable linking of this output in monolithic target builds and disable shared PCH’s
Disable linking if we’re just preprocessing only
If building a Project with nativization, build the Plugin RulesAssembly object for the plugin with all the nativized code
Finally call the UEBuildTarget constructor using our TargetDescriptor, a ReadOnly version of our RulesObject, and the RulesAssembly
1. Creates a SourceFileMetadataCache, Engine or Project\Intermediate\Build\SourceFileCache.bin
2. Does general boring initialization of build info and build path determination
3. Generates Receipt file path Engine or Project\Binaries\Platform\Configuration\TargetName-Platform-Configuration.target
  1. i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.target
  2. Configuration folder is skipped if Development, -Platform-Configuration suffix also skipped
4. Build a ProjectDescriptor if Project file is given
Run UEBuiltTarget’s PreBuildSetup, Setup target before build. This method finds dependencies, sets up global environment etc.
1. Most of this work is based around building UEBuildModules, ModuleRules and Module objects from the .Target.cs files, similar to .build.cs Rules

9. UEBuildTarget - PreBuildSetup

PreBuildSetup is a function that gets called on a UEBuildTarget immediately after a UEBuildTarget is constructed.

SetupBinaries
1. Find or create the Launch UEBuildModule
2. // Get the intermediate directory for the launch module directory. This can differ from the standard engine intermediate directory because it is always configuration-specific.
3. OutputDirectory: // Construct the output paths for this target's executable
  1. Determine if this Target is a Platform Extension Target
  2. Determine whether to output to Project directory
  3. Use Platform Extension or Project directories as output if our .Target.cs file resides in either of these directories
4. Determine if this binary should compile as a .dll (TargetRules.bShouldCompileAsDLL && bCompileMonolithic)
5. Construct list of Binary OutputPaths (usually just a single Binary output i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.exe)
  1. Get Binary Directory (OutputDirectory\Binaries\Platform i.e. D:\UE425\Engine\Binaries\Win64\
  2. Allows for a TargetRules defined ExeBinariesSubFolder (i.e. Datasmith uses this to output in 3dsMax\2017, 3dsMax\2018, etc.)
  3. Create Binary file name
    1. Prefix “lib” if on Linux and compiling a dynamic or static library
    2. If Configuration == TargetRules.UndecoratedConfiguration, just use Module Name i.e. UnrealHeaderTool
    3. Otherwise use the form ModuleName-{Platform}-{Configuration} i.e. UnrealHeaderTool-Win64-Debug
    4. Determine binary extension for platform (i.e. Windows executables = .exe, Windows libraries = .dll)
    5. // Allow the platform to customize the output path (and output several executables at once if necessary)
      1. Default just creates a single item array from input, a single output binary
      2. I.e. Mac, overrides binary path to UnrealHeaderTool/.app/Contents/MacOS/UnrealHeaderTool
6. Create a UEBuildBinary that represents the Binary that UBT will build
7. Set our Launch Module’s Binary property to our new Binary
SetupPlugins
1. Does a bunch of stuff to get Plugin module info for building into our Target
2. #TODO Write this if I ever care about it, a lot of stuff goes on here
3. If compiling monolithic, mark all these Plugin Modules to output to our first Binary we made in SetupBinaries
4. If compiling modularly, mark all these Plugin Modules to output to their own Dynamic Library (.dll)
AddExtraModules, Creates UEBuildModules for all modules listed in our Rules.ExtraModuleNames list
1. If compiling monolithic, mark all these Extra Modules to output to our first Binary we made in SetupBinaries
2. If compiling modularly, mark all these Extra Modules to output to their own Dynamic Library (.dll)
Create all of the UEBuildModules that our Binary depends on recursively
1. #TODO: This is actually kind of neat but all you need to know is that this basically walks the tree of public/private module dependencies
Set the Binary for all dependent UEBuildModules we just created, either .exe or .dll in the same fashion as Plugin/Extra modules
If TargetRules.bBuildAllModules is true (which appears to only be true with the UE4Editor target)
1. /// Adds all the precompiled modules into the target. Precompiled modules are compiled alongside the target, but not linked into it unless directly referenced.
2. #TODO
// Add the external and non-C++ referenced modules to the binaries that reference them.
1. This basically iterates through all of the modules we depend on and if they have any dependent modules that don’t have a Binary set, set them to their parent’s Binary (leave no module behind!)
If not compiling monolithically, determine whether binaries create their import libraries separately
1. On Windows, mark all Binaries
  1. // On Windows create import libraries for all binaries ahead of time, since linking binaries often causes bottlenecks
2. Otherwise only mark circularly dependent modules
3. @TODO: Learn what it means for a binary to create their import library separately
Our UEBuildTarget is finally created.

10. Making the Makefile

Generating and executing the UBT Makefile is really the crux of the UnrealBuildTool. Now that we have a UEBuildTarget, we can continue making it.

After we make our UEBuildTarget, we create our PreBuildStep script files
1. Simply a list of strings that represent shell commands
2. /// Specifies a list of steps which should be executed before this target is built, in the context of the host platform's shell.
3. /// The following variables will be expanded before execution:
4. $(EngineDir), $(ProjectDir), $(TargetName), $(TargetPlatform), $(TargetConfiguration), $(TargetType), $(ProjectFile), $(PluginDir)
5. PreBuildSteps in ProjectDescriptors (.uproject) or in Plugin definitions (.uplugin) gathered first
6. PreBuildSteps in .Target.cs gathered second
7. PreBuildSteps in .uplugin files are gathered last
8. Writes a series of build scripts (either .bat or .sh based on platform)
  1. Outputs scripts to Intermediate\Build\Platform\TargetName\Configuration\PreBuild-1.bat, incrementing name with each script
  2. For example: D:\UE425\Engine\Intermediate\Build\Win64\DatasmithFacadeCSharp\Development\PreBuild-1.bat
  3. Inserts “@echo off” on Windows platforms as first line of each batch file
  4. Does a string replace to expand all the variables mentioned above in each script line
  5. Writes all lines to output script file
  6. Adds reference to outputted script file to UEBuildTarget’s ScriptFiles array
Execute our PreBuildStep scripts
1. On Windows call %COMSPEC% /C “ScriptFile” i.e. C:\Windows\System32\cmd.exe /C “PreBuild-1.bat”
2. Throws build error if any script returns with a non-zero error code
We then kick off building our Target, as our UBT Makefile is actually formed after building of the target

11. UEBuildTarget.Build()

In order to make the UBT Makefile to save time for future builds, we must first do a build.

Create CppCompileEnvironment and LinkEnvironment objects
Create our UEToolChain for our desired platform
Set our Environment Variables, this is set by the desired rules for our platform in our TargetRules object
1. Remember that the TargetRules object also consists of WindowsTargetRules, LinuxTargetRules, etc?
2. #TODO: Go make slide about WindowsTargetRules when WindowsTargetRules is first mentioned
Fetch External Metadata?
// Get diagnostic info to be printed before each build
1. Using Visual Studio 2019 14.16.27031 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10).
2. @TODO: Find example of TargetRules Diagnostic Info
// Setup the hot reload module list
// If we're compiling monolithic, make sure the executable knows about all referenced modules
// Add global definitions for project-specific binaries. HACK: Also defining for monolithic builds in binary releases. Might be better to set this via command line instead?
1. // Defines UE_ENGINE_DIRECTORY=EnginePath
Setup Toolchain bundle dependencies, // On Mac and Linux we have actions that should be executed after all the binaries are created
ExternalExecution.SetupUObjectModules, // Generate headers
1. Map Modules to their UHTModuleTypes
2. // Sort modules by type, then by dependency, StableTopologicalSort() #TODO Write about this because it is neat
3. // Create the info for each module in parallel
  1. Build UHTModuleInfo objects for each Module
  2. Queue each module through SetupUObjectModule
    1. Finds all headers for all modules and classifies them as Classes, Public, or Private headers.
    2. Headers must have a match to the RegEx: "^\s*U(CLASS|STRUCT|ENUM|INTERFACE|DELEGATE)\b"
    3. All headers cached in SourceFileMetadataCache, stores Header’s last write time and whether it contains markup
4. Iterate through all Modules and if they have any reflection markup…
  1. Determine GeneratedCPPFilenameBase, Path.Combine(Module.GeneratedCodeDirectory.FullName, Info.ModuleName) + ".gen"
    1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\CoreUObject\CoreUObject.gen
  2. If not using precompiled modules, determine Module GeneratedCodeWildcard
    1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\CoreUObject*.gen.cpp
5. Output list of UObject dependent modules (contains reflection) and a list of UHTModuleHeaderInfos
  1. A UHTModuleHeaderInfo is simply a list of all a module’s source and header files, and whether this module uses precompiled modules
ExternalExecution.ExecuteHeaderToolIfNecessary, Runs UnrealHeaderTool for a module if needed
1. Determine our UHT Manifest file name i.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\UnrealHeaderTool.uhtmanifest
2. Determine UnrealHeaderTool Receipt file name i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.target
3. Check to see if the UnrealHeaderTool (not target specific, always check the UHT) Receipt exists and is up to date
  1. If UHT Receipt (which is a .target in Binaries path) does not exist, UHT must not be built
  2. If UHT is installed, always up to date
  3. If UHT build products do not exist or are newer than UHT Receipt, UHT is out of date
  4. If UHT Receipt is newer than all build products, UHT should be be up to date
4. Determine if we have the UHT available. We must not be building the UnrealHeaderTool target and an up to date Receipt must exist
5. Determine whether we should run UnrealHeaderTool
  1. If UHT is not available, we must run it (lol)
  2. If our BuildConfiguration is forcing header generation, we must run it
  3. If AreGeneratedCodeFilesOutOfDate, we must run it
    1. // Get CoreUObject.init.gen.cpp timestamp. If source files older than CoreUObject generated code, we must run it
    2. If any module is missing its generated code directory, we must run it
    3. Checks to see if special Timestamp file is good #TODO what?
    4. If any source files are older than UnrealHeaderTool.exe, we must run it
    5. If any rules files are older than our UHT timestamp, we must run it
    6. If any header was changed, added, or removed, we must run it
  4. If manifest corresponding .uhtpath doesn’t exist, run it. If .uhtpath doesn’t contain path to UHT Receipt, run it.
  5. If manifest corresponding .deps doesn’t exist, run it, If AreExternalDependenciesOutOfDate, run it #TODO
6. Generate a list of UHTManifest.Modules for all of our Modules that contain UObjects
7. Create the UHTManifest
8. If we need to run UnrealHeaderTool and we’re not building UnrealHeaderTool….
  1. #TODO
9. Update directory timestamps for all UObject Modules, ExternalExecution.UpdateDirectoryTimestamps()
  1. Don’t actually update directory timestamps as that’ll fuck everything up, instead use a special Timestamp file
  2. Timestamp file serves as directory timestamp but it also contains all timestamps of all headers with Reflection Markup (UObjects)

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
img		img
LICENSE		LICENSE
README.md		README.md

Path	Notes
{USERSETTINGS}/Unreal Engine/Engine/Config/User{TYPE}.ini
{USER}/Unreal Engine/Engine/Config/User{TYPE}.ini
{PROJECT}/User{TYPE}.in

License

Allar/compiling-unreal

Folders and files

Latest commit

History

Repository files navigation

compiling-unreal

Google Slides

Support

No Warranty

Licensing

Build Pipeline Tools

Build Scripts — Build.bat

DotNETUtilities

Visual Studio Notes

How To Debug UnrealBuildTool in the Development Editor Configuration

UnrealBuildTool

Building a Target

Important Objects To Know About

SingleInstanceMutexes

XMLConfigFiles

XML Config Cache Validation

UnrealBuildTool's Config Hierarchy / Config Layers

Practical Knowledge

Technical Knowledge

Tokens

Expansion Loop

Full List of Config Files

Global BuildConfiguration Settings

XMLConfigFile Fields

Command-line Options

Target Descriptors

Example: Building UnrealHeaderTool Target Descriptor

Engine BuildVersion

UEBuildTarget Object

ModuleRulesContext

RulesAssembly Object

Compiling and Loading a RulesAssembly Object

Creating the Engine RulesAssembly Object

TargetRules Object

Creating a TargetRules Object

UEBuildModule Object

Constructing a UEBuildModule

UEBuildBinary

CPPCompileEnvironment

LinkEnvironment

UEToolChain

UHTModuleType

UHTManifest

UHTManifest.Module

Actually Building A Target

1. Startup and Determining Build Mode

2. Startup and Configuration Loading

3. BuildPlatformFactory Discovery

4. Executing The BuildTool ToolMode

5. Determining Target Descriptors

6. Building our SourceFileWorkingSet

7. Loading a Makefile if Present

8. Creating a UEBuildTarget

9. UEBuildTarget - PreBuildSetup

10. Making the Makefile

11. UEBuildTarget.Build()

About

Resources

License

Stars

Watchers

Forks

`RulesAssembly` Object

Compiling and Loading a `RulesAssembly` Object

Creating the Engine `RulesAssembly` Object

`TargetRules Object`

Creating a `TargetRules Object`