Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve loading performance for loader/compression classes. #20940

Merged
merged 1 commit into from
Jul 7, 2023

Conversation

RoosterDragon
Copy link
Member

@RoosterDragon RoosterDragon commented Jun 28, 2023

  • In LCWCompression, RLEZerosCompression use dedicated Array.Fill, Array.Copy and Array.Clear methods instead of open-coded loops. Resolve TODO in VqaVideo.
  • In VoxelLoader.GenerateSlicePlanes.Get use TryGetValue to avoid repeated array and dictionary lookups.
  • In TmpTSLoader.UnpackTileData use ReadBytes to populate array with less overhead compared to repeated one byte reads.

The LCWCompression changes improve the load time in CNC for this method from 0.4% to 0.3%.
The other changes combined improve the load time in TS for those methods from 3.8% to 2.1%.

Copy link
Member

@abcdefg30 abcdefg30 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This crashes for me with

OpenRA engine version {DEV_VERSION}
OpenRA Language: en
Red Alert mod version {DEV_VERSION}
Date: 2023-06-30 17:25:36Z
Operating System: Windows (X64, Microsoft Windows NT 10.0.19045.0)
Runtime Version: .NET CLR 6.0.14
Installed Language: de (Installed) de (Current) de (Current UI)
Exception of type `System.IndexOutOfRangeException`: Index was outside the bounds of the array.
   at System.Array.Clear(Array array, Int32 index, Int32 length)
   at OpenRA.Mods.Common.FileFormats.RLEZerosCompression.DecodeInto(Byte[] src, Byte[] dest, Int32 destIndex) in \OpenRA\OpenRA.Mods.Common\FileFormats\RLEZerosCompression.cs:line 29
   at OpenRA.Mods.Cnc.SpriteLoaders.ShpD2Loader.ShpD2Frame..ctor(Stream s) in \OpenRA\OpenRA.Mods.Cnc\SpriteLoaders\ShpD2Loader.cs:line 81
   at OpenRA.Mods.Cnc.SpriteLoaders.ShpD2Loader.ParseFrames(Stream s) in \OpenRA\OpenRA.Mods.Cnc\SpriteLoaders\ShpD2Loader.cs:line 151
   at OpenRA.Mods.Cnc.SpriteLoaders.ShpD2Loader.TryParseSprite(Stream s, String filename, ISpriteFrame[]& frames, TypeDictionary& metadata) in \OpenRA\OpenRA.Mods.Cnc\SpriteLoaders\ShpD2Loader.cs:line 167
   at OpenRA.Graphics.FrameLoader.GetFrames(Stream stream, ISpriteLoader[] loaders, String filename, TypeDictionary& metadata) in \OpenRA\OpenRA.Game\Graphics\SpriteLoader.cs:line 98
   at OpenRA.Graphics.FrameLoader.GetFrames(IReadOnlyFileSystem fileSystem, String filename, ISpriteLoader[] loaders, TypeDictionary& metadata) in \OpenRA\OpenRA.Game\Graphics\SpriteLoader.cs:line 85
   at OpenRA.Graphics.FrameCache.<>c__DisplayClass1_0.<.ctor>b__0(String filename) in \OpenRA\OpenRA.Game\Graphics\SpriteLoader.cs:line 73
   at OpenRA.Exts.GetOrAdd[K,V](Dictionary`2 d, K k, Func`2 createFn) in \OpenRA\OpenRA.Game\Exts.cs:line 120
   at OpenRA.Primitives.Cache`2.get_Item(T key) in \OpenRA\OpenRA.Game\Primitives\Cache.cs:line 34
   at OpenRA.Graphics.FrameCache.get_Item(String filename) in \OpenRA\OpenRA.Game\Graphics\SpriteLoader.cs:line 76
   at OpenRA.Graphics.CursorSequence..ctor(FrameCache cache, String name, String cursorSrc, String palette, MiniYaml info) in \OpenRA\OpenRA.Game\Graphics\CursorSequence.cs:line 34
   at OpenRA.Graphics.CursorProvider..ctor(ModData modData) in \OpenRA\OpenRA.Game\Graphics\CursorProvider.cs:line 47
   at OpenRA.ModData.InitializeLoaders(IReadOnlyFileSystem fileSystem) in \OpenRA\OpenRA.Game\ModData.cs:line 141
   at OpenRA.Game.InitializeMod(String mod, Arguments args) in \OpenRA\OpenRA.Game\Game.cs:line 476
   at OpenRA.Game.Initialize(Arguments args) in \OpenRA\OpenRA.Game\Game.cs:line 432
   at OpenRA.Game.InitializeAndRun(String[] args) in \OpenRA\OpenRA.Game\Game.cs:line 301
   at OpenRA.Launcher.Program.Main(String[] args) in \OpenRA\OpenRA.Launcher\Program.cs:line 32

when loading RA.

@RoosterDragon
Copy link
Member Author

Reverted the LCWCompression changes since they had minimal impact and were responsbile for the crash. All mods now load.

Mailaender
Mailaender previously approved these changes Jul 1, 2023
Copy link
Member

@Mailaender Mailaender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't spot regressions in sprite nor video loading.

@PunkPun
Copy link
Member

PunkPun commented Jul 1, 2023

Context: the duplicated decoder should have been removed in #18773

- In RLEZerosCompression use dedicated Array.Clear method instead of open-coded loop.
- In VoxelLoader.GenerateSlicePlanes.Get use TryGetValue to avoid repeated array and dictionary lookups.
- In TmpTSLoader.UnpackTileData use ReadBytes to populate array with less overhead compared to repeated one byte reads.
- Resolve TODO in VqaVideo.
@PunkPun PunkPun merged commit 1c0885c into OpenRA:bleed Jul 7, 2023
3 checks passed
@PunkPun
Copy link
Member

PunkPun commented Jul 7, 2023

Changelog

@RoosterDragon RoosterDragon deleted the perf-load branch July 7, 2023 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants