Build process
Rubberduck uses a customized build process, which is designed to target both the debug and release builds and it must maintain compatibility with AppVeyor when building the release builds. To make those customization possible, a whole project is dedicated to it, the Rubberduck.Deployment and Rubberduck.Deployment.Build projects. In addition to the DLLs generated from other projects, the Rubberduck.Deployment also uses custom MSBuild tasks run in both Pre and Post build events which processes the registration and other configuration. The build tasks are defined in the meta-project Rubberduck.Deployment.Build.
When using COM interop, a common approach to make COM components visible on the machine is to execute the regasm.exe. In fact, Rubberduck's original installer basically does this at the install time. However, this has a host of problems and actually goes against the recommended approach when developing COM components. The traditional recommendation is to write registry entries directly, which has the advantage that the install becomes more atomic and is easier to transact, without requiring arbitrary code execution. However, the traditional recommendation can be difficult to implement due to large amount of registry keys required to be created and the maintenance of them.
The whole purpose of the Rubberduck.Deployment project is to help manage the maintenance of the registry keys for COM registration plus few other installer-related aids such as maintaining the copyright dates.
The workflow will run for every build the developer or AppVeyor performs, as it is part of the build events of the project. It will run custom MSBuild tasks which acts as the coordinator. For COM registration, it will:
If the C++ build tools are available then:
- Convert the
AssemblyintoTypeLibusing .NET'sTypeLibConverterclass - Pass to olewoo DLL with a custom listener (
IDListenerclass) to generate a IDL file, with custom attributes injected by the listener - Compile the
idlfile into atlbfile using the MIDL compiler
Else:
- Execute
tlbexp.exetool to generate both 32-bit and 64-bit TLB for a given DLL
Subsequently:
- Execute WiX's
heat.exetool to generate XML fils for the DLL and its 32-bit TLB. - Invoke the builder to parse the XML files generated in the previous step
- The builder will create a list of all registry entries based on the XML data and provide to the build task
- The build task will then invoke the writer to generate file content as a string with the registry commands
- The build task will then write the returned string as a file to the specified location
Though .NET allows us to use attributes to decorate a COM-visible class, there are numbers of things that are not possible using .NET's Interop attributes alone. For example, we cannot mark an interface as restricted, nor can we define a enumeration name. To circumvent this limitation, we make use of OleWoo library with a custom listener which enable us to generate a IDL file during the build time. The listener then will inject desired changes to the IDL file, adding or removing attributes and changing the names/signatures.
The IDL file is then provided to the MIDL compiler, which comes as a part of C++ build toolchain or various Windows SDK. It is then compiled into a TLB file with all the customization applied, granting us with more control over how we will expose the COM API.
For contributors who chose to not have C++ build tools installed as part of their Visual Studio installation, the project is set up to fall back on tlbexp.exe which means that on those builds, the customizations will not be applied leading to some differences between what is observed on a local debug build vs. what may be released in a release build. For those not dealing with COM interop, it should not be a problem. For those who needs full fidelity or want to work on COM interop, it is recommended that the C++ build tools be used. When the project cannot locate the MIDL compiler, it will issue a warning at the build.
The project contains the WiX toolset binaries within the folder WixToolset. This is where the heat.exe` is located and will be used in the build task. Should an update to WiX toolset binaries be needed, simply download the binaries from WiX's github and replace the contents of the folder.
Because, maintenance. We already had an Inno Setup script and we have complex scripting logic which would be difficult to replicate in a WiX project. Furthermore, the WiX documentation assumes too much from the developer, which can make it hard to use and troubleshoot. Because Rubberduck is an OSS project, it is in its best interest that we use tools that are easy to approach and usable by a larger audience. Despite using Pascal script, Inno Setup fits this requirement better than WiX at the time of writing.
Originally, the Rubberduck's main project had Register for COM Interop checked, which was the equivalent of performing a regasm.exe at the build time. With the Rubberduck.Deployment, there is no need to use Reigster for COM Interop checkbox anywhere. In fact, it is NOT recommended that it be checked as this can lead to "my machine only" bugs. The other objective for the Rubberduck.Deployment is to ensure that the developer's debug build will have the same COM registration data in the machine's registry as the user who used an installer would have.
Furthermore, when modifying the COM-visible component, perhaps adding new or removing old components, there is the potential to create orphaned records in the registry with the checkbox because it only generates a new registry without necessarily cleaning up the previous entries. In fact, a previous build might have entries that no longer exist in the current build and thus won't be handled by the regasm.exe tool. On the other hand, the Rubberduck.Deployment project writes out a registry script with all keys enumerated for each build so that for the next build, the registry script is executed and all old keys are removed prior to writing the new keys.
Yes, it does and it is woefully insufficient. It contains only the data for the DLL itself but none of the type library as well as the additional keys that needs to be added when a type library is present. Furthermore, regasm.exe disallow the use of the /regfile switch with the /tlb switch. Thus it is basically useless to us.
The project should reference any other assemblies within the Rubberduck's solution where there is COM registration needing to be done. Referencing ensures that those assemblies' output are then copied to the Rubberduck.Deployment's output directory, which in turn simplify the macros used to execute the build task. We'd rather not have to use path that reach across the projects as that makes for a fragile build process since renaming of project could then break the build. Thus, the references are used to avoid the problem. However, because referencing can import much more than just one assembly from another project, it is necessary to manually specify which assembly needs to be processed for COM registration, as described in the next section.
Within Rubberduck.Deployment.csproj, we indicate we want to use the build tasks from Rubberduck.Deployment.Build project with the following lines:
<UsingTask TaskName="RubberduckPreBuildTask" AssemblyFile="..\Rubberduck.Deployment.Build\bin\Rubberduck.Deployment.Build.dll" />
<UsingTask TaskName="RubberduckPostBuildTask" AssemblyFile="..\Rubberduck.Deployment.Build\bin\Rubberduck.Deployment.Build.dll" />
We can then use the actual build task further down the same .csproj file:
<Target Name="PreBuildTask" BeforeTargets="PreBuildEvent">
<RubberduckPreBuildTask WorkingDir="$(ProjectDir)" OutputDir="$(TargetDir)" />
<Message Text="Ran Rubberduck prebuild task" Importance="normal" />
</Target>
Which references the class of the same name and pass parameters in. Likewise for post build:
<Target Name="PostBuildTask" AfterTargets="PostBuildEvent">
<GetFrameworkSdkPath>
<Output TaskParameter="Path" PropertyName="SdkPath" />
</GetFrameworkSdkPath>
<CreateProperty Value="$(ProjectDir)$(OutputPath)$(TargetFileName)">
<Output TaskParameter="Value" PropertyName="TargetAssembly" />
</CreateProperty>
<RubberduckPostBuildTask
Config="$(ConfigurationName)"
NetToolsDir="$(SdkPath)bin\NETFX 4.6.1 Tools\"
WixToolsDir="$(ProjectDir)WixToolset\"
SourceDir="$(TargetDir)"
TargetDir="$(TargetDir)"
ProjectDir="$(ProjectDir)"
IncludeDir="$(ProjectDir)InnoSetup\Includes\"
FilesToExtract="Rubberduck.dll" />
<Message Text="Ran Rubberduck postbuild task" Importance="normal" />
</Target>
Which also links to this build class.
The main purpose of the parameters passed is to allow the build task to run off macros that is available only to the Visual Studio so we can at least avoid hard-coding the absolute path to various things, including the tools tlbexp.exe and heat.exe which are located outside of the Rubberduck.Deployment's project directory.
The NetToolsDir parameter should refer to the directory where the tlbexp.exe is located since it is not in the PATH environment variable.
The WixToolsDir parameter should refer to the directory where the WiX toolset is which is pulled by a nuget package and thus can be located within the package directory at the solution level.
The SourceDir and TargetDir parameters represents input and output directory to be used by the build task for processing. They can be same if we don't need to write to a different directory. Because we need other assemblies that aren't part of the COM registration, it simplifies thing to use the same directory. The TLB files generated as the result will be placed into the TargetDir.
The ProjectDir parameter refers to the root directory of the Rubberduck.Deployment to allow the build task to locate subfolders that are not a part of the build. For example, we have folders used to hold temporary registration files for debug builds as will be explained later.
The IncludeDir parameter refers to the Includes directory which is used by Inno Setup to pull in autogenerated .iss files so therefore is where the output of the InnoSetupRegistryWriter gets placed.
The FilesToExtract parameter accepts a |-delimited lists of assembly to be extracted, so multiple assemblies can be registered for COM registration, though at the time of writing, only one is.
The build task will then process each DLL through either the MIDL compiler or the tlbexp.exe tool located in the NetToolsDir parameter twice, one for 32-bit and again for 64-bit. The preference is to use MIDL compiler if it's available. But on environments without C++ build tool, we will fall back to tlbexp.exe. It is similar to the regasm.exe except that it does not actually register the type library as the regasm.exe would have.
The next step the build task will perform is to generate XML files, one from the DLL and other from the 32-bit TLB file through WiX's heat.exe tool. We do not need to do this for the 64-bit TLB file because the output will be same as the 32-bit TLB anyway. The XML files contains all the information that we need to build the registry entries. The build task will in turn invoke the builder to transform the data in the XML files into a list of RegistryEntry structs, so that we have a good abstraction of the registry entry we must create. The builder will return the list back to the build task.
The build task will then invoke a writer, providing the list from the build task. The writer will then generate a file that contains the registry entries in a format that is appropriate for its use. For example, InnoSetupRegistryWriter class will generate content that is a suitable for a .iss file containing registry commands for the Inno Setup installer to consume. The writer returns the appropriately formatted text to the build task.
The build task will finally write the writer's resulting string as a file to the filesystem at the specified path. In the case of the output from the InnoSetupRegistryWriter, it would write the new .iss file to the IncludesDir parameter so that it is available to be picked up by the Inno Setup when it compiles the install script.
The class is built within the Rubberduck.Deployment.Build assembly which is then used by the build task, with this command:
private IOrderedEnumerable<RegistryEntry> BuildRegistryEntriesFromMetadata(DllFileParameters parameters)
{
var builder = new RegistryEntryBuilder();
return builder.Parse(parameters.TlbXml, parameters.DllXml);
}
Basically it passes in the XML files generated by the WiX's heat.exe tool to the builder's Parse method. Within the parse method, the class will load the XML files then generate various mapping for different type of COM registrations. The XML already contains all the necessary data, so builder mainly needs to transform them into a useful shape, which is the RegistryEntry struct. The builder will generate all RegistryEntry structs it needs to describe every single registry entry that must be created, for all sub-branches of the Software\Class branch. It will also collect information on whether 32-bit and 64-bit counterpart are needed as there are differences on what must be written to the 32-bit and 64-bit. For example, the TypeLib registry branch will have win32 and win64 subkey that should be generated with no respect to the registry virtualization.
Its implementation requires a good understanding of how COM registration works. For details, refer to COM Registration for all the details on registry entries needed to register COM components.
There are a number of keys that requires specific data, notably a full path to a directory, the DLL or the TLB file. Those cannot be known until the install time. For that reason, the builder will locate any keys that contains the parameterization and insert a placeholder (see PlaceHolder static class) in where it is needed. At the time of writing, they are only inserted into the Value member of the RegistryEntry struct.
It is then the writer's responsibility to convert those placeholders into an appropriate format so that it may either be appropriately expanded at the install time using the installer's convention of expansion or at the build time for the local builds.
Once the build task has gotten the list of RegistryEntry, it will invoke a writer which must implement the interface. At the time of writing, there are two implementations:
InnoSetupRegistryWriterLocalDebugRegistryWriter
The only method implemented is Write method, which takes the list of RegistryEntry and transform them into a string. It is up to the writer to generate a string that's appropriate for whatever will consume it. The build task will then create an actual physical file at a specified location using the output as shown:
private void CreateInnoSetupRegistryFile(IOrderedEnumerable<RegistryEntry> entries, DllFileParameters parameters)
{
var writer = new InnoSetupRegistryWriter();
var content = writer.Write(entries, parameters.DllFile, parameters.Tlb32File, parameters.Tlb64File);
var regFile = Path.Combine(IncludeDir, parameters.DllFile.Replace(".dll", ".reg.iss"));
// To use unicode with InnoSetup, encoding must be UTF8 BOM
File.WriteAllText(regFile, content, new UTF8Encoding(true));
}
In the case of Inno Setup, a registry entry will typically look like this in a .iss script:
Generic form:
Root: "<Hive to write to>"; Subkey: "<subkey path, without the hive>"; ValueType: <data type>; ValueName: "<name, if any>"; ValueData: "<value, if any>"; Flags: <Inno Setup specific flags> Check: <Inno Setup specific check functions>
Example:
Root: "HKCU64"; Subkey: "Software\Classes\CLSID\{{40F71F29-D63F-4481-8A7D-E04A4B054501}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.PermissiveAssertClass"; Flags: uninsdeletekey; Check: IsWin64 and not InstallAllUsers
Therefore, the InnoSetupRegistryWriter will generate a line that conforms to the Inno Setup's registry command, taking in the appropriate parameterization. It also uses the Bitness information from the RegistryEntry to help it decide how it should write for both 64-bit and 32-bit hives and to include appropriate check to prevent writing keys where it's not needed. For example, it will include IsWin64 to ensure that the registry entry will be only generated only when the installing machine is itself 64-bit when writing an entry to HKCU64.
There need not be a one-to-one correspondence between the list of RegistryEntry to the output from the InnoSetupRegistryWriter. For some registry entries, there may be multiple lines written, mainly to handle all 64-bit, 32-bit and neutral variations. As an illustration, here is a possible output from a single registry entry. Note the Check parameter at the end of the line.
Root: "HKCU64"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: IsWin64 and not InstallAllUsers
Root: "HKCU32"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: IsWin64 and not InstallAllUsers
Root: "HKCU"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: not IsWin64 and not InstallAllUsers
Root: "HKLM64"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: IsWin64 and InstallAllUsers
Root: "HKLM32"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: IsWin64 and InstallAllUsers
Root: "HKLM"; Subkey: "Software\Classes\CLSID\{{69E194DA-43F0-3B33-B105-9B8188A6F040}"; ValueType: string; ValueName: ""; ValueData: "Rubberduck.UnitTesting.AssertClass"; Flags: uninsdeletekey; Check: not IsWin64 and InstallAllUsers
The writer is only used for debug and is what developers will consume. This primarily exists so that developers will be using the same data generated by the RegistryEntryBuilder, making their debug build much more like the installed builds without actually installing via the Inno Setup installer.
However, it will have a side-effect of actually writing to the developer's HKCU hive as it processes the RegistryEntry entries. It will then write out a command suitable in a .reg format to delete the same key it just wrote, similar to the following:
Windows Registry Editor Version 5.00
[-HKEY_CURRENT_USER\Software\Classes\CLSID\{40F71F29-D63F-4481-8A7D-E04A4B054501}]
[-HKEY_CURRENT_USER\Software\Classes\CLSID\{40F71F29-D63F-4481-8A7D-E04A4B054501}\Implemented Categories\{62C8FE65-4EBB-45e7-B440-6E39B2CDBF29}]
...
and that registry script is returned to the build task. The file will be then stored in a different location, in Rubberduck.Deployment\LocalRegistryEntries.
Thus, when the next time the developer builds the solution, the build task will check whether there is a previous registry script already saved into the folder, and if so, execute it. This has the effect of deleting all the keys from the previous build. That ensures that the developer is not left with stale registry keys as the developer makes changes to the COM visible components which may no longer exist in the next build.
As the name implies, the writer is only executed only for a Debug build, which is why the build task takes the $(Configuration) macro as one of its parameters.
Note that at the time of writing, whenever a build runs and the build task has executed the deletion registry script, it will rename the registry script with a imported_yyyyMMddhhmmss suffix with UTC timestamp. That provides the developer with a history of what was deleted from the registry. Currently, those files will not be deleted and must be manually deleted.
The Rubberduck.Deployment project is also used to help perform maintenance. In this case, we have a license which contains a copyright. Every year, it'd "expire" and someone has to update it manually since Inno Setup does not allow parameterization of a license file. To allay that, the Rubberduck.Deployment will run the build task which will use the template license. At the time of writing the license.rtf only has one parameter, $(YEAR$), which the build task will replace with the current year at the build time. It then copies the file into the appropriate location for the Inno Setup installer script to pick up, thus ensuring that the new builds will reflect the current year they were made.
rubberduckvba.com
© 2014-2021 Rubberduck project contributors