test_update_tips_callback_error incompatible with libgit2 >= 1.4.0

[dvzrv]

Hi! When trying to rebuild ruby-rugged against libgit2 1.4.1 on Arch Linux I also ran tests.
However, test_update_tips_callback_error fails on the following:

  1) Skipped:
RemoteNetworkTest#test_remote_check_connection_push_credentials [/build/ruby-rugged/src/rugged-1.3.1/test/remote_test.rb:36]:
Skipped, no message given

  2) Failure:
RemoteTransportTest#test_update_tips_callback_error [/build/ruby-rugged/src/rugged-1.3.1/test/remote_test.rb:368]:
Expected #<Rugged::Branch:400 {name: "origin/packed", target: #<Rugged::Commit:420 {message: "packed commit two\n", tree: #<Rugged::Tree:440 {oid: f82a8eb4cb20e88d1030fd10d89286215a715396}>
  <"another.txt" 7c3f1a8504912d590d12048d32cd31d2d75d69ac>
  <"second.txt" bb61d8117a8cae026fe4061e15c29a96aea3496e>
, parents: ["5001298e0c09ad9c34e4249bc5801c75e9754fa5"]}>}> to not be truthy.

544 runs, 2333 assertions, 1 failures, 0 errors, 1 skips
rake aborted!
Command failed with status (1): [ruby -w -I"lib:lib:test" /usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/rake_test_loader.rb "test/blame_test.rb" "test/blob_test.rb" "test/branch_test.rb" "test/cherrypick_test.rb" "test/commit_test.rb" "test/config_test.rb" "test/diff_test.rb" "test/dotgit_test.rb" "test/errors_test.rb" "test/index_test.rb" "test/lib_test.rb" "test/merge_test.rb" "test/note_test.rb" "test/object_test.rb" "test/patch_test.rb" "test/rebase_test.rb" "test/reference_test.rb" "test/remote_test.rb" "test/repo_apply_test.rb" "test/repo_ignore_test.rb" "test/repo_pack_test.rb" "test/repo_reset_test.rb" "test/repo_test.rb" "test/revert_test.rb" "test/settings_test.rb" "test/signature_test.rb" "test/status_test.rb" "test/submodule_test.rb" "test/tag_test.rb" "test/tree_test.rb" "test/walker_test.rb" --verbose]
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:130:in `block (3 levels) in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils.rb:57:in `sh'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils.rb:104:in `ruby'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:117:in `block (2 levels) in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/file_utils_ext.rb:58:in `verbose'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/testtask.rb:111:in `block in define'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/usr/lib/ruby/gems/3.0.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/usr/bin/rake:23:in `load'
/usr/bin/rake:23:in `<main>'
Tasks: TOP => test

Full build log: ruby-rugged-1.3.1-build.log

[carlosmn]

I am seeing this when trying to update to that version. It looks like a bug in the library, but we'd have to investigate. In general rebuilding against a different version of the library does involve potentially updating the bindings either way so it's not always a bug that things don't work if you link against an unexpected version.

[dvzrv]

@carlosmn I guess a CI pipeline that checks against the current latest version of libgit2 (built from its default branch) could help detect these issues earlier and allow integration much faster.

The ruby bindings exist under the same organization as libgit2 and they should be released in tandem, otherwise downstreams are blocked from upgrading or run into unknown issues.

[carlosmn]

Such a CI job would have to allow for failing and it may show some issues, but someone would have to actively check. Like anything else, the bindings are maintained on a best effort basis and sometimes it doesn't work out to release quickly.

[dvzrv]

Such a CI job would have to allow for failing and it may show some issues, but someone would have to actively check.

I guess, but it would allow everyone to see that an issue will come up in the future. Work on resolving it can start before the release of a new libgit2 version, which is objectively less stressful :)

Like anything else, the bindings are maintained on a best effort basis and sometimes it doesn't work out to release quickly.

I understand. However, I still believe that a CI pipeline building against the current latest libgit2 commit would already allow more (even outside) people to get involved (maybe they won't, but the added visibility surely makes it much easier for them) :)

[carlosmn]

Maybe, but it seems higher effort to go check that CI and to try things out for oneself.

But remember this isn't something that is an incompatibility that needs to be worked out in the bindings. This was a bug that wasn't caught in the library and the fix for this is to fix it in the library and patch that testing gap, which has now been done.