New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Archive Reader implementation #16129
Adding Archive Reader implementation #16129
Conversation
ef4d1c0
to
89948c5
Compare
Gems/Archive/Code/Include/Archive/Clients/ArchiveInterfaceStructs.h
Outdated
Show resolved
Hide resolved
Gems/Archive/Code/Include/Archive/Clients/ArchiveInterfaceStructs.inl
Outdated
Show resolved
Hide resolved
Gems/Archive/Code/Include/Archive/Clients/ArchiveInterfaceStructs.inl
Outdated
Show resolved
Hide resolved
Gems/Archive/Code/Include/Archive/Clients/ArchiveInterfaceStructs.inl
Outdated
Show resolved
Hide resolved
…terfaces Implementors of `ICompressionInteraface` or `IDecompressionInterface` can now implement the `GetCompressionAlgorithmName` function to provide a human readable name associated with the registered compression/decompression interface. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
This pairs with the implementation of the ArchiveWriter API and therefore can be used to write and O3AR archive files. The Archive Reader supports the following operations 1. `ListFileInArchive` - This function can query a file within an archive by file path or file token which was queried previously from a ListFileInArchive call 1. `ExtractFileFromArchive` - This function can extract file from the archive using either the relative file path of the contained file or the file token queried from a call to `ListFileInArchive` The file is extracted to a user provided allocated buffer via an `AZStd::span<AZStd::byte>` paramter 1. `EnumerateFilesInArchive` - This functions enumerates all non-deleted files within the archive and invokes a user provided callback with the ArchiveListFileResult structure for each file. The `ArchiveListFileResult` structure is the same type returned by `ListFileInArchive` Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
The issue was an `<Item>` element was trying to be created to show the success value, but there was data associated with it. The proper way to visualize such an element is to use teh `<Synthetic>` element. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
89948c5
to
370bada
Compare
Gems/Archive/Code/Include/Archive/Clients/ArchiveInterfaceStructs.inl
Outdated
Show resolved
Hide resolved
return blockCount <= MaxBlocksNoJumpEntry | ||
? MaxBlocksNoJumpEntry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The blockCount <= MaxBlocksNoJumpEntry
is checked before using this function in the lines below, so is this check necessary here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No it is not.
I'll remove in this PR if there is a need for a non-trivial update.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also make sure which logic is right. The one below returns zero, this one returns MaxBlocksNoJumpEntry
if blockCount <= MaxBlocksNoJumpEntry
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well the inner lambda condition is always returning the "else" part of the Ternary since the condition is checked before calling it.
The entire purpose of the two function is to return the minimum block index for the final set of block lines that don't have a jump entry (confusing and hard to explain, I know).
But pretty much every compressed file in the Archive that is >16 MiB starts with its first block line (index 0) with a jump entry.
And then every three block lines afterwards(index 3, 6, 9, ...) would have a Jump entry if there more than 18 MiB (9 blocks) remaining in the file from the start of that block line.
It is explained better in the Streaming Archive document at here, but the implementation is complicated.
Fixed cmake configure when the attempting to the add the Archive Game and Editor API targets as dependencies of the Asset Bundler Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
longer mention that it takes ownership of the raw TOC buffer. Added a `GetBlockLineSpanForFile` doxygen @param parameter for the Archive TOC View. Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly notes on comments/docs. nice work
Signed-off-by: lumberyard-employee-dm <56135373+lumberyard-employee-dm@users.noreply.github.com>
Archive Gem changes
Adding complete implementation of the new Archive ArchiveReader API
This pairs with the implementation of the ArchiveWriter API and
therefore can be used to write and O3AR archive files.
The Archive Reader supports the following operations
ListFileInArchive
- This function can query a file within anarchive by file path or file token which was queried previously from
a ListFileInArchive call
ExtractFileFromArchive
- This function can extract file from thearchive using either the relative file path of the contained file or
the file token queried from a call to
ListFileInArchive
The file is extracted to a user provided allocated buffer via an
AZStd::span<AZStd::byte>
paramterEnumerateFilesInArchive
- This functions enumerates all non-deleted files within the archive and invokes a user provided callback with the ArchiveListFileResult structure for each file. TheArchiveListFileResult
structure is the same type returned byListFileInArchive
Archive Factory addition
Added an Archive Reader and Writer Factory implementation
The Archive Reader Factory allows IArchiveReader instances to be created
by external Client gem modules by depending on the Archive.Clients.API target
The Archive Writer Factory allows IArchiveWriter instances to be created
by external Tool gem modules by depending on the Archive.Tools.API
target.
This other Gems such as the AssetValidation or some other library such as the AssetBundler to create Archive class instances needed to interact with the API
Compression Gem changes
Adding a new interface function of
GetCompressionAlgorithmName
to the ICompressionInterface/IDecompressionInterface interfaces.Overriding this function can used to provide a human readable name associated with the Compression algorithm
The plan for this change is to make it easier for a future CLI to allow uses to specify the compression algorithm to use when adding files to an archive.
How was this PR tested?
Added UnitTest for the ArchiveReader functionality to the Archive.Editor.Tests module
Validated the existing ArchiveWriter test still passes.