Skip to content

Commit

Permalink
ver 0.1
Browse files Browse the repository at this point in the history
  • Loading branch information
old-account-392754 committed Sep 6, 2022
1 parent 13b60dd commit d7f8ab2
Show file tree
Hide file tree
Showing 10 changed files with 130 additions and 120 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@
<ExecutablePath>$(ExecutablePath)</ExecutablePath>
<IncludePath>$(SolutionDir)\CommonIncludes\;$(IncludePath)</IncludePath>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<IncludePath>$(SolutionDir)\CommonIncludes\;$(IncludePath)</IncludePath>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<WarningLevel>Level4</WarningLevel>
Expand Down Expand Up @@ -120,6 +123,7 @@
<IntrinsicFunctions>true</IntrinsicFunctions>
<SDLCheck>true</SDLCheck>
<ConformanceMode>true</ConformanceMode>
<LanguageStandard>stdcpp17</LanguageStandard>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
Expand All @@ -131,9 +135,6 @@
<ClCompile Include="Catch_core_datacontainer_tests.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\CommonIncludes\CommonIncludes.vcxproj">
<Project>{6ab3fa0b-e833-4828-a494-0cae6f640242}</Project>
</ProjectReference>
<ProjectReference Include="..\DataContainerGenerator\DataContainerGenerator.vcxproj">
<Project>{ea045c4d-fcab-40dc-afb8-2a64f2b56354}</Project>
</ProjectReference>
Expand Down
6 changes: 1 addition & 5 deletions Catch_ve_tests/Catch_ve_tests.vcxproj
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@
<SDLCheck>true</SDLCheck>
<ConformanceMode>true</ConformanceMode>
<LanguageStandard>stdcpp17</LanguageStandard>
<EnableEnhancedInstructionSet>AdvancedVectorExtensions2</EnableEnhancedInstructionSet>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
Expand All @@ -130,11 +131,6 @@
<ItemGroup>
<ClCompile Include="Catch_ve_tests.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\CommonIncludes\CommonIncludes.vcxproj">
<Project>{6ab3fa0b-e833-4828-a494-0cae6f640242}</Project>
</ProjectReference>
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
Expand Down
22 changes: 13 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@

## What is it?

The DataContainer project contains a code generation tool that turns text specifications describing objects and their relationships into a C++ header files defining a `data_container` class that manages the described objects and relationships.
The DataContainer project contains a code generation tool (DataContainerGenerator) that turns text specifications describing objects and their relationships into a C++ header files defining a `data_container` class that manages the described objects and relationships.

## Why would anyone want to use this?
## Why does this exist?

Since that sounds a lot like what `struct`s and `class`es in a C++ header file already are, why would anyone want to go through the extra trouble required to use a code generator? The generated data container is intended to solve the following problems:
Since that sounds a lot like what structs and classes in a C++ header file already are, why would anyone want to go through the extra trouble required to use a code generator? The generated data container is intended to solve the following problems:

- Values stored in standard C++ objects end up arranged in memory in a way that is relatively unfriendly to SIMD operations. The standard C++ solution is to use a "struct of arrays" rather than an "array of structs", but actually doing this can be quite cumbersome. The generated data container, however, automatically stores values for the same property in contiguous memory and exposes a user-friendly SIMD interface to them.
- Managing relationships between objects involves both repetitive boilerplate and is a common source of bugs. The code required to manage the relationships is also often hard to make changes to without significant effort. The generated data container helps solve this problem by automatically generating the code required to keep relationships up to date as objects are created and deleted, and automatically maintains any indexes you require for quickly finding an object based on a given relationship, or vice versa.
- Serializing and deserializing C++ objects is a pain in the neck. It is easy enough to dump their contents out to a file, but anything non trivial requires more than that. You may want the ability to selectively save some of an object's properties and not others when it is possible to recalculate that information without saving it directly. And you almost certainly want the ability to load serialized data created by old versions of your software, even after member variables have been added, removed, or changed type. Because the generator is provided a description of your objects and relationships that it can understand as a whole, it can provide you with robust serialization and deserialization routines.
- And yes, there are existing libraries for doing all of the above things (often quite good ones). However, the problems discussed above are all fundamentally about storing and managing the data that is structured into objects and relationships. Thus, using a library to solve one of these problems may very well make it harder to use a library to solve another of them. And so the approach of this project is to try to solve them all in one go, and thus to be able to hide the sometimes ugly details from the end user completely.
- OK, but why a code generator? Some early iterations of these ideas were implemented with dark template magics, but doing so made compilation significantly slower. So while it is harder to initially integrate a code generator into your project, I think that ultimately it is the more user friendly solution.
- Values stored in standard C++ objects end up scattered in memory in a way that is relatively unfriendly to SIMD operations. The standard C++ solution is, on paper, to use a "struct of arrays" rather than an "array of structs", but actually doing this can be quite cumbersome. The generated data container, however, automatically stores values for the same property in contiguous memory and exposes a user-friendly SIMD interface to them.
- Managing relationships between objects involves both repetitive boilerplate and is a common source of bugs. The code required to manage the relationships is also often hard to make changes to without significant effort.[^1] The generated data container helps solve this problem by automatically generating the code required to keep relationships up to date as objects are created and deleted, and automatically maintains any indexes you require for quickly finding an object based on a given relationship, or vice versa.
- Serializing and deserializing C++ objects is a pain in the neck. It is easy enough to dump their contents out to a file, but anything non trivial requires more than that. You may want the ability to selectively save some of an object's properties and not others when it is possible to recalculate that information without saving it directly. And you almost certainly want the ability to load serialized data created by old versions of your software, even after member variables have been added, removed, or changed type. Because the generator has a description of your objects and relationships that it can understand as a whole, it can provide you with robust serialization and deserialization routines without any additional effort on your part.
- And yes, there are existing libraries for doing all of the above things (often quite good ones). However, the problems discussed above are all fundamentally about storing and managing the data that is structured into objects and relationships. Thus, using a library to solve one of these problems may very well make it harder to use a library to solve another of them.[^2] And so the approach of this project is to try to solve them all in one go, and thus to be able to hide the sometimes ugly details from the end user completely.
- OK, but why a code generator? Some early iterations of these ideas were implemented with dark template magics, but doing so made compilation significantly slower. So while it is harder to initially integrate a code generator into your project, I think that ultimately it is the more user-friendly solution.

## Disclaimer

Expand All @@ -33,4 +33,8 @@ The following documentation explains all you need to know about how to generate

## Feedback

Feel free to leave comments and suggestions as you see fit. If you are actively using, or intend to use, a data container generated by this project in one of your own projects, I will try to prioritize any fixes or features that you need. Otherwise, additions will be mostly driven by my own needs.
Feel free to leave comments and suggestions as you see fit. If you are actively using, or intend to use, a data container generated by this project, I will try to prioritize any fixes or features that you need. Otherwise, additions will be mostly driven by my own needs.

[^1]: This is the primary reason that this version the idea exists (as an improvement to a previous version that generated containers for each logical object individually); managing the code required to keep the relationships working correctly, and to make sure that they serialized and deserialized correctly, was tedious and error prone in a way that made me unwilling to make changes at all.

[^2]: For example, a robust serialization library is typically build out of techniques that emulate compile-time reflection on C++ classes and structs. This tends to be hard to integrate with the techniques you might use to generate SIMD-friendly memory layouts.
Loading

0 comments on commit d7f8ab2

Please sign in to comment.