Git tags after v5.6.3 are lightweight instead of annotated resulting in an incorrect submodule version

[alex-xage]

Crypto++ Issue Report

System Info

OS: MacOS
Crypto++ version: 8.6.0
Git version: 2.31.1

Actual Behavior

When crypto++ is embedded as a submodule in another repository, the command git submodule status shows the incorrect tag and version for the submodule:

$> git submodule status | grep cryptopp
 69bf6b53052b59ccb57ce068ce741988ae087317 cryptopp (CRYPTOPP_5_6_3-5333-g69bf6b53)

As seen above the latest version tag is listed as v5.6.3, but the commit 69bf6b5 refers to the commit for the v8.6.0 release.

Expected Behavior

When crypto++ is embedded as a submodule in another repository, the command git submodule status should show the latest version. If the commit is the same as a tagged release version it should only show the tag name.

$> git submodule status | grep cryptopp
 69bf6b53052b59ccb57ce068ce741988ae087317 cryptopp (CRYPTOPP_8_6_0)

Root Cause

The command git submodule status uses git describe to determine the submodule status:

Show the status of the submodules. This will print the SHA-1 of the currently checked out commit for each submodule, along with the submodule path and the output of git describe for the SHA-1

The git describe command only shows annotated tags by default:

By default (without --all or --tags) git describe only shows annotated tags.

After release v5.6.3 the crypto++ project switched from annotated tags to lightweight tags:

((HEAD detached at 69bf6b53)) $> git for-each-ref refs/tags
1fc174a620bf9cfdf2daef156a7a44e8ab9ff1c9 tag	refs/tags/CRYPTOPP_5_0
60aecd44e11fd59518f4291ce980e5c4073d70a6 tag	refs/tags/CRYPTOPP_5_1
5abaac539aa986e97d2fc4a5e771becc455f79f2 tag	refs/tags/CRYPTOPP_5_2
d4571853293b0644a089103005f8b09789a2efc7 tag	refs/tags/CRYPTOPP_5_2_1
84a30d7bcf244f52a2b2824697387a99bd3c7e47 tag	refs/tags/CRYPTOPP_5_2_3
828de611d7d9212bceda1e3cee93bb5bb5ead6db tag	refs/tags/CRYPTOPP_5_3_0
c8274f05d45ae0dbcd94f53bdfd8d26f62e511f7 tag	refs/tags/CRYPTOPP_5_4
d1422afa8cd48a352e1bebd2a91d6a44593f120d tag	refs/tags/CRYPTOPP_5_5
cf5be720ac9de0b561cb379fe1e6118c13939cdf tag	refs/tags/CRYPTOPP_5_5_1
41bdb6c917549f35704f88afed6a77ca7ce6bd4f tag	refs/tags/CRYPTOPP_5_5_2
11ea22253158e710be2d2f248aca54c0f0eb7fcc tag	refs/tags/CRYPTOPP_5_6_0
ddf10160f2cb16df342c1ea068b1cb82fb8c9346 tag	refs/tags/CRYPTOPP_5_6_1
96f9107f22a5cca5ca491ef33ed39ea117efc019 tag	refs/tags/CRYPTOPP_5_6_2
804519f4290ee328be18b388924e791e80251422 tag	refs/tags/CRYPTOPP_5_6_3
4132d85888ac6c30729f58bb442a4a26a5b16cfe commit	refs/tags/CRYPTOPP_5_6_4
c621ce053298fafc1e59191079c33acd76045c26 commit	refs/tags/CRYPTOPP_5_6_5
b6c6684451f925ab2e0b99acbaf66fd487ebfbd5 commit	refs/tags/CRYPTOPP_6_0_0
5be140bcea453a00f7f2fec09fb9e37849d65d98 commit	refs/tags/CRYPTOPP_6_1_0
c8d8caf70074655a2562ae1ea45cb30e28fee2b4 commit	refs/tags/CRYPTOPP_7_0_0
5e5fb6c855e0f0232065079ab96414ce1a78c411 commit	refs/tags/CRYPTOPP_8_0_0
1c34979592f7d5e5d73e265a9c8650cc006620d2 commit	refs/tags/CRYPTOPP_8_1_0
9dcc26c58213abb8351fbb1b2a7a1d2c667366e4 commit	refs/tags/CRYPTOPP_8_2_0
d71bc515eafc0a6418372f63ba0f058a55d25951 commit	refs/tags/CRYPTOPP_8_3_0
434e3189db61ff4ced13b47fe450a42b3c8cb676 commit	refs/tags/CRYPTOPP_8_4_0
f2102243e6fdd48c0b2a393a0993cca228f20573 commit	refs/tags/CRYPTOPP_8_5_0
69bf6b53052b59ccb57ce068ce741988ae087317 commit	refs/tags/CRYPTOPP_8_6_0

Thus when calling git submodule status the last annotated tag is returned which is for v5.6.3. See [1] for full debugging log.

Solution

The lightweight tags in this repository should be recreated as annotated tags.

Appendix

[1] Full debugging log

[noloader]

Thanks Alex.

Another Git usability fuckup. Who would have thought someone can screw up tags. The Git folks make it impossible to do simple things.

If you can provide a script to fix things I will run it for you.

[alex-xage]

A collaborator could run the following commands for each tag:

# this is essentially: `git tag [-a ] [-f] <tagname> [<object>]` because a tag is an object
git tag -a -f <tagname> <tagname>
# Force is needed b/c: https://github.com/git/git/blob/master/Documentation/RelNotes/1.8.2.txt#L7-L12
git push --force origin <tagname>

HOWEVER After looking into this further it may not be possible to update these old tags without deleting the associated releases, see this StackOverflow comment:

Warning: If you do this on a GitHub repo, and the lightweight tag you are replacing was associated with a release, deleting it will silently delete the entire release (and release notes) associated with the lightweight tag. As noted here, there's no way to restore an accidentally-deleted release.

Tags are also uniquely identified by names so there cannot be an annotated tag and lightweight tag with the same name:

Unless -f is given, the named tag must not yet exist.

So to update the tag a collaborator would need to:

1. save the release info
2. run above command
3. recreate release with new tag and saved info

I believe that even if you recreate a release with the same information the timestamp / creation of the release will not match the original release date which is also not desired behavior. Due to this it may not be possible to fix this for existing releases. I see some other options moving forward:

1. only create annotated tags for future releases
2. create annotated tags with a separate format e.g. vX.Y.Z instead of CRYPTOPP_X_Y_Z for past and future releases

[noloader]

@alex-xage,

I think the past is under the bridge. Its another problem created by that cursed Git tool. What a fucking dumpster fire...

Moving forward we will use annotated tags. That means we create the annotated one on the command line, and not use GitHub for tagging.

I updated our procedures to specify annotated tags from the command line. See https://www.cryptopp.com/wiki/Release_Versioning#Tagging and https://www.cryptopp.com/wiki/Release_Process#Announce_the_release .

---

After release v5.6.3 the crypto++ project switched from annotated tags to lightweight tags

While not readily apparent, here's what changed. After Crypto++ 5.6.2 we migrated from Sourceforge to GitHub. So all the release tags from Crypto++ 2.0 to Crypto++ 5.6.2 got created by hand. At Crypto++ 5.6.3, we were on GitHub and used GitHub to create the tags.