Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added resource module with packer target, implemented resource packing across examples #66

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

jonjondev
Copy link
Member

@jonjondev jonjondev commented Mar 17, 2024

Description

This PR introduces a new resource packing system to the engine, extracting many dependencies such as tinyobjectloader and stb_image from the engine, and ensuring source asset files do not get packaged up alongside the executable.

This functionality is achieved via a meta-program, the "packer", which is a newly added target, compiled before the runtime engine targets. It is called from the project Makefiles with the paths of assets for package, and will output a a .pck file that can be mounted and queried at runtime using the original file paths.

The structure of the .pck file is as follows:

  • Header
  • Body Entries
  • Magic Number
  • Table of Contents Entries

Under this structure, the header will contain an offset to the Table of Contents Entries, which each know their own size, and can be indexed via file path once parsed, and each entry contains an offset (from the start of the Body Entries) to its related entry along with it's length.

The resource packing system currently supports the following resources:

  • StaticMeshData from .obj files
  • Texture2DData from .png, .jpg, and .jpeg files (more types are technically supported but not searched for)
  • ShaderData from .spv files
  • FontData from .ttf files
  • SceneData and SceneEntityData from .scene, and .entity files respectively (entities are now being packed directly into their related scenes to reduce indirection)

The resource packing system currently does not support the following resources:

  • Scene data from .scene files

Updates must be made to the scene system to support read-only mode so that it does not try to write back to the pack file or some random location on your disk. Right now, I have added a stopgap that will allow tests to continue to function as intended.

In addition, it is important to note that the packing method disables padding (using #pragma pack(push, 1)) for simplicity, so it's important to keep in mind that it may result in unaligned reads.

There are currently no tests setup for the resource system as very little can be unit tested. A scheme will need to be devised to support integration-level testing. This has now been setup. It's not quite unit tested, but it's being tested just about as well as we can hope for right now.

The PR has been...

  • provided a reasonable name that is not just the branch name (e.g "Added Vulkan render delegate")
  • linked to its related issue
  • assigned a reviewer from the team
  • labelled appropriately

The code has been...

  • made mergable and free of conflicts in relation to master (according to GitHub)
  • tested in a packaged state using the package targets
  • pulled to the reviewer's machine and reasonably tested

@jonjondev jonjondev added core Relating to the engine's core build Relating to the engine's build system labels Mar 17, 2024
@jonjondev jonjondev requested a review from Raelr March 17, 2024 10:24
@jonjondev jonjondev force-pushed the feature/resource-packing branch 2 times, most recently from 65c493e to c1e053a Compare March 17, 2024 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Relating to the engine's build system core Relating to the engine's core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant