v2.6.3

Release Highlights

This release is all about filesystem efficiencies.

Feature: retry failed writes

File write failures (exceptions, test failures) are unexpected and can be caused by transient issues. Writes will now be tried up to three times before giving up with an error. This number can be controlled with the --write-retry <count> advanced option.

Fix: add write permissions to copied files

@alienth found a bug where read-only files that were copied kept their read-only status, resulting in later errors. Copied files will now have write permissions added when needed.

Fix: reduced the default reader and writer threads

The default reader threads have been reduced 10->8, and the default writer threads have been reduced 10->4. The goal here is to increase speed by decreasing thrash on drives, especially platter-based ones.

Fix: generate candidates simultaneously

This matters when raw-writing files (not extracting or zipping) while also testing. In those cases, checksums need to be calculated for archive files (as opposed to the checksums of files within the archive that were calculated during ROM scanning), and this can be sped up by calculating multiple at once.

All changes

12 April 2024 - 62a34f7 - v2.6.3 (#1089)
12 April 2024 - ef19665 - CI: allow manual runs of the Node.js test CI workflow (#1097)
12 April 2024 - eb667f1 - Refactor: explicitly flush written files to disk (#1094)
12 April 2024 - 8cf535a - Fix: update copied file permissions with 0o222 (#1093)
12 April 2024 - 65e632f - Fix: revert FsPoly.copyDir() changes (#1098)
12 April 2024 - 9e14aad - Refactor: filesystem walk with file types (#1096)
12 April 2024 - 94f8da2 - CI: FilePoly unit tests (#1091)
12 April 2024 - 362f98d - Chore: update softprops/action-gh-release action to v2 (#1054)
12 April 2024 - 515c793 - Refactor: remove most usages of util.promisify() (#1087)
12 April 2024 - 89c3aa9 - Feature: retry failed writes (#1088)
12 April 2024 - 30a5f26 - Fix: chmod +w files after copying them (#1086)
12 April 2024 - 81cb28c - Chore: update Node.js to v20.12.2 (#1081)
12 April 2024 - 113f4c4 - Fix: use node.exe for the caxa command on Windows (#1085)
10 April 2024 - 09295d9 - CI: Renovate: don't update Node.js immediately after release (#1082)
7 April 2024 - 7fabb9a - Refactor: generate multiple candidates simultaneously (#1077)
7 April 2024 - e3d0d2e - Fix: reduce the default reader and writer thread count (#1076)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.6.2

Release Highlights

Fix: parent/clone inference with game "alternates"

@MrNoOneRed found that the GoodTools-style [a] and [a#] tags were not getting grouped into parent/clones with TOSEC DATs. This has been fixed.

Checksum cache updates

Feature: checksums of non-archives are now cached.

This is most helpful when copying or moving large files such as ISOs, or when calculating MD5/SHA1/SHA256 hashes.

Fix: test for cache write permissions.

Previously, igir would write its cache file to ./igir.cache. But it can't be assumed that this file path is writable, as reported by @TheBrainScrambler on NixOS. So now igir will fall back to writing the cache file to ~/igir.cache.

##$ Token updates

Feature: new {romm} token.

@maxexcloo has added support for RomM, a web UI to help visualize ROM collections. As of writing, RomM does not have any mass ROM sorting functionality, so igir is a great choice to help organize RomM collections.

Feature: added "cracked" game type.

A new "cracked" game type has been added that will affect the {gameType} output token. Games considered "cracked" have had some copy protection removed, and by definition are not retail, so they will also be excluded by the --only-retail option. Thank you @MrNoOneRed for the feature suggestion!

Fix: "Famicom" and "Super Famicom" DATs.

"Nintendo Entertainment System" and "Super Nintendo Entertainment System" have been supported for a long time, but @kerobaros pointed out that "Famicom" and "Super Famicom" were not. This has been fixed.

All changes

7 April 2024 - 45596f8 - v2.6.2 (#1075)
7 April 2024 - 470cad6 - Feature: Added RomM Export (#1045)
7 April 2024 - dbf5566 - Fix: make sure the cache file is writable (#1070)
7 April 2024 - 3b6579b - Feature: detect "cracked" games, treat as non-retail (#1074)
7 April 2024 - ec43210 - Fix: treat GoodTools alternates as clones (#1073)
7 April 2024 - 8e26ebf - Fix: cache checksums of raw files (#1063)
6 April 2024 - f386ccd - Chore: Bump pillow from 10.2.0 to 10.3.0 in /docs (#1062)
6 April 2024 - d8b56d9 - Chore: update docs dependencies (#1068)
6 April 2024 - 2b626e2 - Chore: update Node.js to v20.12.1 (#1067)
6 April 2024 - 33e322f - Chore: update Node.js to v20.12.0 (#1046)
6 April 2024 - 391c12f - Chore: bump typescript-eslint packages to v7.5.0 (#1066)
6 April 2024 - 993d4a6 - CI: Dependabot settings (#1064)
29 March 2024 - d6ec0c6 - Chore: update lockfile (#1059)
28 March 2024 - c834f7a - Chore: update lockfile (#1058)
28 March 2024 - d95f140 - Chore: update lockfile (#1057)
28 March 2024 - 65a7925 - Chore: update dependency term-size to v4 (#1053)
28 March 2024 - cc2f2ac - Chore: update lockfile (#1056)
27 March 2024 - 7f4c8ff - Chore: update dependency archiver to v7 (#1051)
27 March 2024 - b7dcb7f - Chore: update dependencies (#1049)
27 March 2024 - 3b59fec - Chore: update dependency eslint-plugin-jsdoc to v48.2.1 (#1048)
27 March 2024 - 9b63add - Chore: update dependency @types/node to v20.11.30 (#1047)
27 March 2024 - 6d38295 - Chore: update dependency typescript to v5.4.3 (#1055)
27 March 2024 - 89b1f4f - Chore: update docs dependencies (#1050)
25 March 2024 - 6ab8d34 - Fix: Famicom and Super Famicom output token matching (#1044)
24 March 2024 - a721494 - Refactor: extract the same input files at the same time (#1042)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.6.1

Release Highlights

Feature: SHA256 support

v2.6.0 added the ability to calculate MD5 and SHA1 checksums, and now SHA256 has been added as well. SHA256 is non-standard in that it's not part of the original Logiqx XML DTD, but some groups include it such as No-Intro's "standard" (non-parent/clone) DATs, and Hardware Target Game Database.

Feature: {onion} output token updates

@maxexcloo added support for the following consoles:

• Amiga CD32
• Commodore VIC20
• NEC PC-FX
• Nintendo DS
• Sinclair ZX-81

Thank you for your code contributions and support!

All changes

23 March 2024 - 268d2fd - v2.6.1 (#1040)
23 March 2024 - 49d1e7e - Feature: SHA256 support (#1032)
23 March 2024 - 4aebb8d - Fix: Added Newly Supported Onion Game Systems (NDS, etc) (#1039)
23 March 2024 - adb5c06 - CI: add an "ongoing" GitHub label (#1038)
23 March 2024 - 06d9b35 - CI: update inactive discussion lock language (#1037)
22 March 2024 - d02725a - CI: automatically lock old discussions (#1033)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.6.0

Release Highlights

Feature: the ability to calculate and match MD5 & SHA1 checksums

A long missing feature of igir has been the ability to match ROMs to DATs based on something other than CRC32 + filesize. This addition brings igir ever closer to feature parity with other major ROM managers.

igir will now automatically determine all file checksums it needs to calculate to match input files to ROMs in every DAT provided. This is important because not every DAT provides every checksum (e.g. MAME sometimes only provides SHA1, Hardware Target Game Database doesn't always provide filesize).

You can control the minimum checksum algorithm used with the --input-min-checksum <algorithm> option.

Feature: archive checksum cache

To support MD5 & SHA1 checksums, and as prep work for CHD support (#937), igir will now cache the checksums of any archived files that it had to compute itself.

Some archive formats such as .zip and .7z include pre-calculated CRC32 information so igir will skip calculating it by default. No supported archive format includes pre-calculated MD5 or SHA1 checksums, so any need to calculate them can greatly slow down ROM scanning. Having a cache file with igir-calculated checksums can greatly speed up subsequent scans.

Fix: directories in ROM names

Up until now, igir tried to intelligently guess if a DAT used / or \ path separators in ROM names intentionally. This led to the issue that was fixed in v2.5.3 for some No-Intro DATs. No-Intro doesn't allow these characters in game names, and Redump also follows No-Intro's naming conventions.

From now on, igir won't try to be smart about game or ROM names, and will instead trust the DAT groups that they know what they're doing.

All changes

21 March 2024 - 950f352 - v2.6.0 (#1028)
21 March 2024 - 0f4b900 - Fix: treat the provided checksum algorithm as the minimum (#1026)
21 March 2024 - 9b92ebf - Refactor: remove the "unique" language from progress bar messages (#1025)
21 March 2024 - ef0abb0 - Refactor: only log the logger prefix at trace log level (#1023)
21 March 2024 - 6596c13 - Feature: disable cache option (#1024)
21 March 2024 - af610f7 - Feature: cache archive entry checksums (#1004)
21 March 2024 - 307770c - Feature: ability to match ROMs via MD5 & SHA1 (#945)
21 March 2024 - f016c87 - Refactor: prefer calculated CRC32 of archive entry contents (#1021)
20 March 2024 - caa51ae - Refactor: centralize the code to hash a raw file (#1022)
20 March 2024 - 5a58ecb - Feature: render days in progress bar ETAs (#1020)
20 March 2024 - 2063004 - Refactor: don't sanitize path separators in game & ROM names (#1019)
20 March 2024 - facb20e - Fix: treat duplicate game names as clones (#1018)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.5.3

Release Highlights

Fix: directory paths not preserved for No-Intro DATs

@tnvu reported a bug with directory paths in Xbox 360 digital ROM names not being respected. This has been fixed.

Fix: long exit time

An issue was fixed with the CLI progress bar rendering that could cause igir to needlessly wait up to 10 seconds after processing before finally exiting.

Refactors: preparation for MD5 & SHA1 matching

Most of this release involves code refactors that were necessary pre-work for MD5 and SHA1 matching (#818). Please report and new bugs found with this.

All changes

20 March 2024 - e82dc6f - v2.5.3 (#1017)
19 March 2024 - d56a8e2 - Fix: minimum for number of games with path characters (#1016)
19 March 2024 - b548560 - Fix: revert: warn the user of any unused patches (#1015)
19 March 2024 - 90affeb - Docs: use the word "headerless" everywhere (#1014)
19 March 2024 - 24b2b1a - Feature: test zip archives with the highest checksum available (#1013)
19 March 2024 - 9f2c274 - Feature: test raw files with the highest checksum available (#1012)
19 March 2024 - 67431bd - Refactor: allow files to not have CRC32s (#1011)
18 March 2024 - 63ce80b - CI: increase test coverage (#1010)
18 March 2024 - 77601c7 - Fix: check if an existing file is a symlink during symlink testing (#1009)
18 March 2024 - 02ca43e - Refactor: throw when extracting invalid entry path from archive (#1008)
18 March 2024 - 35b5bdb - Fix: retain "ROM names contain directories" attribute when combining DATs (#1007)
17 March 2024 - 97bfeeb - Refactor: centrally manage Node.js timeouts (#1005)
15 March 2024 - 82eb81d - Refactor: use FileProps and ArchiveEntryProps for instantiation (#1003)
9 March 2024 - e992d98 - Feature: file-backed cache (#1002)
9 March 2024 - a1acfd3 - Refactor: IndexedFiles class (#992)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.5.2

Release Highlights

Fix: headered ROMs being written when headerless was desired

While continuing to work on ROM matching via MD5 and SHA1, it was discovered that sometimes igir was writing headered files when a DAT specified a headerless ROM. This should be fixed.

Because igir can automatically remove headers when needed, it can match headered files in the input directory when processing a headerless DAT. This primarily affected the No-Intro NES headerless DAT, as most DAT groups don't release headerless DATs.

Fix: reading bad zip files

@tnvu reported a TorrentZipped .zip file that was causing ungracefully handled errors. There is some issue with the underlying zip library, archiver, but igir will at least catch these errors gracefully now.

All changes

8 March 2024 - 8cbd9fc - v2.5.2 (#999)
8 March 2024 - 978e473 - Fix: re-enable GitHub Actions workflow linter (#928)
8 March 2024 - 6d86a9f - Fix: add more try/catch around zip files (#994)
8 March 2024 - 176d7ef - Fix: writing headered files instead of unheadered (#998)
8 March 2024 - 1f4ac62 - Fix: games can't be both a parent and clone (#997)
8 March 2024 - 16bb8ed - Refactor: ROMProps extends ChecksumProps (#996)
8 March 2024 - 5859b9b - Refactor: don't prefer input archives unless necessary (#993)
8 March 2024 - d572d6e - Refactor: log input file size during writing (#995)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.5.1

Release Highlights

Fix: filtering parents broke parent/clone relationships

@halsafar found a fairly critical bug with 1G1R filtering. Previously, if a --filter-* option would filter out the parent in a set of parent/clone games, all of the clones would be orphaned and therefore all written to the output. In other words, --single wasn't being respected in all cases. This should be fixed now, thank you for the report!

Feature: --no-program and --only-program options

The existing --no-test-roms and --only-test-roms options have been renamed to --no-program and --only-program. These options now filter ROMs that are "programs" (applications). The old options still exist, but will be removed in the future.

Thank you @lucaorio for the suggestion!

Feature: demo, prototype, and sample filters updated

The code determining whether a game is a demo, prototype, or sample has been updated to include some less often-used tags/language. This should bring the three options parity with Retool.

Fix: --clean-dry-run log level updated

@PhasecoreX updated the log level that indicates what files were skipped from cleaning because of the --clean-dry-run option to INFO, which hopefully makes it more usable. Thank you!

Sponsors

Thank you @maxexcloo for sponsoring igir on GitHub! Your recent feedback and code contributions have been greatly helpful.

All changes

7 March 2024 - 5509c78 - v2.5.1 (#990)
7 March 2024 - 23e24cc - Fix: don't orphan children when filtering out parents (#989)
7 March 2024 - 768bfc3 - Feature: warn about low log levels with --clean-dry-run (#988)
7 March 2024 - 6248ccd - Fix: Update clean dry run to use info logging instead of debug (#978)
4 March 2024 - ddaa41c - Refactor: logging updates (#986)
3 March 2024 - 3ce2b6b - Refactor: don't allow mutually exclusive commands (#985)
3 March 2024 - d8af7ea - Fix: written ROMs can't be considered "moved" (#984)
29 February 2024 - 2a64c62 - Chore: update lockfile (#980)
29 February 2024 - 7527476 - Chore: update lockfile (#979)
28 February 2024 - fa98d5a - Chore: update lockfile (#977)
28 February 2024 - 5885816 - Chore: update lockfile (#976)
28 February 2024 - 86dab08 - Chore: update lockfile (#975)
28 February 2024 - d2f2881 - Chore: update lockfile (#974)
28 February 2024 - 7eb68e4 - Chore: update release-drafter/release-drafter action to v6 (#970)
28 February 2024 - 2dfcdd7 - Chore: update peter-evans/create-pull-request action to v6 (#969)
28 February 2024 - 4cea081 - Feature: only-program, no-program filter options (#972)
28 February 2024 - c81f866 - Feature: debug, demo, prototype, sample filter updates (#973)
27 February 2024 - 00116dd - Chore: update dependency eslint-plugin-simple-import-sort to v12 (#967)
27 February 2024 - 20dd19b - Chore: update dependency @types/semver to v7.5.8 (#971)
27 February 2024 - 293afce - Chore: update dependency eslint-plugin-unicorn to v51 (#968)
27 February 2024 - f46ef1b - Chore: update rinchsan/renovate-config-validator action to v0.2.0 (#965)
27 February 2024 - 7a20f21 - Chore: update ESLint (#964)
27 February 2024 - 95c581d - Chore: update docs dependencies (#963)
27 February 2024 - 46668a4 - Chore: update dev dependencies (#962)
27 February 2024 - abc4088 - Chore: update dependency @types/jest to v29.5.12 (#961)
27 February 2024 - 60d0034 - Chore: update dependencies (#960)
27 February 2024 - 5fbe5a1 - Chore: update adRise/update-pr-branch action to v0.7.2 (#959)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.5.0

Release Highlights

Feature: hard link support

First, the existing symlink command has been renamed to link. symlink will continue to work the same as link --symlink for now, but the command will be removed in the future.

@maxexcloo had a great suggestion to support hard links in addition to symbolic links. Hard links can be helpful when you want to move files around more freely, as opposed to symbolic links that can break without warning.

A new --symlink option has been added to create symlinks, and it is required for the existing --symlink-relative option.

Fixes: reading & writing file links

The deep dive into adding hard links revealed a few issues:

• @PhasecoreX pointed out that using input directories that had symlinks was not working
• Relative symlinks were constructed incorrectly in some cases, mainly when dealing with macOS' /private directory

These should be fixed, and quite a few unit tests around file links have been added.

Fix: reading files that no longer exist

Trying to read .tar and .tar.gz files that didn't exist would cause igir to hang forever. That was the primary issue fixed, but several unit tests have been added for all supported file types to ensure they don't hang or crash.

Tokens: {es} and {retrodeck} added

@maxexcloo added two new output path tokens:

• {es} for EmulationStation, which is great timing now that it's available on Android!
• {retrodeck} for RetroDECK

Thank you for these contributions!

All changes

24 February 2024 - e81214d - v2.5.0 (#958)
24 February 2024 - 679aa5c - CI: auto-merge disabler for outside contributions (#957)
24 February 2024 - 260d87a - Docs: add warning comments to README (#954)
24 February 2024 - 186711c - Feature: Add New Outputs (EmulationStation + RetroDECK) (#956)
24 February 2024 - 6d077c3 - Docs: add file linking to alternatives table (#953)
24 February 2024 - 43ce6c8 - CI: set Renovate 'automerge' not 'platformAutomerge' in packageRules (#955)
24 February 2024 - d88a8fc - Fix: test reading nonexistent files (#952)
24 February 2024 - 310051a - Feature: hard link support (#951)
24 February 2024 - cdbce2e - Fix: ability to scan directory symbolic links (#949)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.4.2

Release Highlights

Feature: --clean-dry-run option added

@PhasecoreX had a suggestion to add an option that would log what output files would be cleaned rather than cleaning them. The option has been added, and documentation has been updated.

Refactor: log level overhaul

Thanks to a suggestion from @PhasecoreX, log levels have been overhauled to be more usable. They fall into three buckets now:

• -v logs about actions that are happening (file creation, modification, and deletion)
• -vv logs about actions that are happening, and actions that are skipped (not overwriting files, etc.)
• -vvv logs about actions that are happening, actions that are skipped, and debug information for bug reports

See the new logging documentation page for more information.

Refactor: slimmer executable packages

A bit of effort was put in to slim down the executables that get compiled for GitHub, trimming a lot of unnecessary junk that developers publish to npm. Please open bug reports for any issues you experience.

All changes

22 February 2024 - 1a65db3 - v2.4.2 (#950)
22 February 2024 - fa3a186 - Feature: clean-dry-run option (#948)
22 February 2024 - c70c637 - Docs: logging & troubleshooting pages (#947)
22 February 2024 - 4d15f69 - Refactor: update log levels (#946)
22 February 2024 - 80811e7 - Chore: update Node.js to v20.11.1 (#936)
22 February 2024 - 5362365 - CI: custom script for executable packaging (#944)
13 February 2024 - e8e5cdb - Feature: ability to calculate headerless MD5 & SHA1 (#934)
13 February 2024 - a909c75 - Fix: always get the size of symlink targets (#935)
11 February 2024 - 5b3a2cf - Fix: increase README help message width for GitHub (#933)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian

v2.4.1

Release Highlights

Feature: macOS ARM binaries for Apple Silicon

Every new release, including this one, will have 64-bit ARM binaries built for macOS.

As always, running igir via npm is the most reliable method, and it should be preferred over the binaries published to GitHub.

Fix: reducing memory usage

@TheBrainScrambler has been helping debug fatal out-of-memory crashes that can affect large DAT packs.

Some minor changes were made to reduce the memory footprint of DATs. But a major change with this release is replacing the old XML parsing library with a different one, resulting in a ~60% memory reduction during DAT scanning on my M2 MacBook Air. This change carries some risk that the two libraries parse DATs differently, so please open GitHub issues for any problems you encounter.

All changes

11 February 2024 - 905aacf - v2.4.1 (#932)
11 February 2024 - b7c4147 - Refactor: filter out DATs earlier (#930)
11 February 2024 - 64b027f - Chore: update ESLint (#890)
11 February 2024 - 02acebb - Docs: dat-ignore-parent-clone option (#931)
11 February 2024 - a5deb8e - Chore: pin dependencies (#929)
11 February 2024 - 546ab4a - Refactor: replace xml2js with fast-xml-parser (#926)
10 February 2024 - ffc4913 - Refactor: remove some unused game properties (#927)
10 February 2024 - da27e06 - Refactor: eliminate unnecessary game name storage (#924)
10 February 2024 - 77b7c3b - Refactor: use a set for progress bar waiting messages (#925)
10 February 2024 - f1b90c8 - Fix: greatly reduce the number of allowed filesystem threads (#923)
9 February 2024 - 740975b - Feature: compile igir for Apple Silicon (#922)
9 February 2024 - a523656 - Fix: reduce README help message width for GitHub (#921)

igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.

❤️ Christian