PS: Cleanup upgrade/downgrade chains #225
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… current dbscheme, and add a single downgrade script that downgrades the current dbscheme to the old dbscheme. IMPORTANT: the .gitattributes ensure that we keep CLRF (i.e., windows) line-endings on the old dbscheme scripts so that they match what is produced by the extractor.
…xtractor and injecting it into the CLI.
dilanbhalla
reviewed
May 13, 2025
dilanbhalla
approved these changes
May 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An upgrade script is identifies by a hash, and that hash is computed by
git hash-object path-to-dbscheme(git hash-objectjust prefixes some constant string onto the stream before computing it's sha1 digest).A subtle issue that I failed to realize in all my previous upgrade/downgrade scripts is that care must be taken related to windows vs. unix line-endings when computing this sha. For example
gitwill to "normalize" line-endings depending on your OS/git settings, and if that's done to your dbscheme file after you have computed the hash you may end up with a sha that's incorrect. So when the CodeQL cli tries to match up the hash in the upgrade scripts folder with the hash of the dbscheme that was used to build an old DB the hashes won't match!To fix this I've deleted all the old dbscheme upgrade/downgrades and remade a single upgrade/downgrade script while taking extra care to match the line endings. So now we have a single upgrade script which:
Similarly, we have a single downgrade script which:
So now the upgrade/downgrade scripts can just be Unix-style line endings going forward like the other CodeQL supported languages uses.
We can delete those old dbscheme upgrade/downgrade scripts because the PowerShell extractor was never published after I made those incorrect upgrade/downgrade scripts 😂.
Additionally, I've also added an entry in
.gitattributesto ensure that git doesn't normalize the line endings in a way that messes things up the first time git touches the files.While here, I also noticed that we were missing an operation in the PowerShell build script that copied over the downgrade scripts when building the PowerShell extractor. I've fixed this in 2bf076d.
I've tested that this works on a random old DB I had from a previous failing PowerShell campaign run. And for that one it now works perfectly 🥳