Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrieve full name matches over hazy short hash matches #1293

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1aea465
Commit up so I can work later on work computer
RoahNo Jul 16, 2022
6ef602d
test needs some fine-tuning but should be at a good place
Jul 18, 2022
d3a8e5e
tests are showing our problem now, need to ensure we are doing all we…
Jul 18, 2022
9c11966
tests breaking, but looking into what issue could be, think issue has…
Jul 19, 2022
a392480
yup, head wasn't correct, acting appropriately now, tests are passing…
Jul 19, 2022
bb238bf
JENKINS-62592 cleaning up test which was impacted by change, fixed now!
Jul 19, 2022
77d3abb
JENKINS-62592 reformat, renamed some bad test variable names
Jul 19, 2022
eb77163
JENKINS-62592 adding a more "to-the-point" test regarding the issue w…
Jul 19, 2022
b224776
JENKINS-62592 adding a test to show branch will resolve first over ha…
Jul 21, 2022
858c917
Commit up so I can work later on work computer
RoahNo Jul 16, 2022
726e440
test needs some fine-tuning but should be at a good place
Jul 18, 2022
3f796fa
tests are showing our problem now, need to ensure we are doing all we…
Jul 18, 2022
a25cdc3
tests breaking, but looking into what issue could be, think issue has…
Jul 19, 2022
a78a503
yup, head wasn't correct, acting appropriately now, tests are passing…
Jul 19, 2022
817cfaa
JENKINS-62592 cleaning up test which was impacted by change, fixed now!
Jul 19, 2022
a54838e
JENKINS-62592 reformat, renamed some bad test variable names
Jul 19, 2022
bbd1558
JENKINS-62592 adding a more "to-the-point" test regarding the issue w…
Jul 19, 2022
94500f8
JENKINS-62592 adding a test to show branch will resolve first over ha…
Jul 21, 2022
057fa3f
Merge remote-tracking branch 'origin/retrieve-full-name-first' into r…
Aug 9, 2022
d1bcc80
JENKINS-62592 adding a test to show branch will resolve first over ha…
Aug 16, 2022
909c6e7
Merge remote-tracking branch 'origin/retrieve-full-name-first' into r…
Aug 16, 2022
fd37877
Merge branch 'jenkinsci:master' into retrieve-full-name-first
RoahNo Aug 16, 2022
8466080
JENKINS-62592 address wildcard import and unnecessary whitespace changes
Sep 26, 2022
1f046f7
Merge remote-tracking branch 'origin/retrieve-full-name-first' into r…
Sep 26, 2022
fa93af1
JENKINS-62592 more whitespace changes
Sep 26, 2022
22a0a09
Merge branch 'master' into retrieve-full-name-first
MarkEWaite Jun 19, 2023
5673ebc
Remove unnecessary change from pom.xml
MarkEWaite Jun 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 40 additions & 33 deletions src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java
Original file line number Diff line number Diff line change
Expand Up @@ -870,6 +870,7 @@
Set<String> fullHashMatches = new TreeSet<>();
String fullHashMatch = null;
GitRefSCMRevision candidateOtherRef = null;
Boolean shortRevisionAmbiguity = false;
for (Map.Entry<String,ObjectId> entry: remoteReferences.entrySet()) {
String name = entry.getKey();
String rev = entry.getValue().name();
Expand Down Expand Up @@ -923,18 +924,25 @@
break;
}
}
if (rev.toLowerCase(Locale.ENGLISH).startsWith(revision.toLowerCase(Locale.ENGLISH))) {
if (!revision.isEmpty() && rev.toLowerCase(Locale.ENGLISH).startsWith(revision.toLowerCase(Locale.ENGLISH))) {
shortNameMatches.add(name);
listener.getLogger().printf("Candidate partial match: %s revision %s%n", name, rev);
if (shortHashMatch == null) {

Check warning on line 930 in src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line

Line 930 is only partially covered, one branch is missing
listener.getLogger().printf("Candidate partial match: %s revision %s%n", name, rev);
shortHashMatch = rev;
} else {
listener.getLogger().printf("Candidate partial match: %s revision %s%n", name, rev);
listener.getLogger().printf("Cannot resolve ambiguous short revision %s%n", revision);
return null;
if (fullTagMatches.isEmpty() && fullHashMatches.isEmpty() && fullHashMatch == null && candidateOtherRef == null) {

Check warning on line 934 in src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line

Line 934 is only partially covered, 8 branches are missing
// We haven't found any matches, and we have ambiguous matches, cannot determine
shortRevisionAmbiguity = true;

Check warning on line 936 in src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered line

Line 936 is not covered by tests
}
}
}
}
if (fullHashMatch != null) {
// JENKINS-62592, for predictability, if an exact match is found, that should be returned
//since this would have been skipped if this was a head or a tag we can just return whatever
return new GitRefSCMRevision(new GitRefSCMHead(fullHashMatch, fullHashMatches.iterator().next()), fullHashMatch);
}
if (!fullTagMatches.isEmpty()) {
// we just want a tag so we can do a minimal fetch
String name = StringUtils.removeStart(fullTagMatches.iterator().next(), Constants.R_TAGS);
Expand All @@ -944,34 +952,6 @@
context.wantTags(true);
context.withoutRefSpecs();
}
if (fullHashMatch != null) {
//since this would have been skipped if this was a head or a tag we can just return whatever
return new GitRefSCMRevision(new GitRefSCMHead(fullHashMatch, fullHashMatches.iterator().next()), fullHashMatch);
}
if (shortHashMatch != null) {
// woot this seems unambiguous
for (String name: shortNameMatches) {
if (name.startsWith(Constants.R_HEADS)) {
listener.getLogger().printf("Selected match: %s revision %s%n", name, shortHashMatch);
// WIN it's also a branch
return new GitBranchSCMRevision(new GitBranchSCMHead(StringUtils.removeStart(name, Constants.R_HEADS)),
shortHashMatch);
} else if (name.startsWith(Constants.R_TAGS)) {
tagName = StringUtils.removeStart(name, Constants.R_TAGS);
context.wantBranches(false);
context.wantTags(true);
context.withoutRefSpecs();
}
}
if (tagName != null) {
listener.getLogger().printf("Selected match: %s revision %s%n", tagName, shortHashMatch);
} else {
return new GitRefSCMRevision(new GitRefSCMHead(shortHashMatch, shortNameMatches.iterator().next()), shortHashMatch);
}
}
if (candidateOtherRef != null) {
return candidateOtherRef;
}
//if PruneStaleBranches it should take affect on the following retrievals
boolean pruneRefs = context.pruneRefs();
if (tagName != null) {
Expand All @@ -998,6 +978,34 @@
context,
listener, pruneRefs, retrieveContext);
}
if (shortHashMatch != null) {
// woot this seems unambiguous
for (String name: shortNameMatches) {
if (name.startsWith(Constants.R_HEADS)) {
listener.getLogger().printf("Selected match: %s revision %s%n", name, shortHashMatch);
// WIN it's also a branch
return new GitBranchSCMRevision(new GitBranchSCMHead(StringUtils.removeStart(name, Constants.R_HEADS)),
shortHashMatch);
} else if (name.startsWith(Constants.R_TAGS)) {
tagName = StringUtils.removeStart(name, Constants.R_TAGS);
context.wantBranches(false);
context.wantTags(true);
context.withoutRefSpecs();
}
}
if (tagName != null) {
listener.getLogger().printf("Selected match: %s revision %s%n", tagName, shortHashMatch);
} else {
return new GitRefSCMRevision(new GitRefSCMHead(shortHashMatch, shortNameMatches.iterator().next()), shortHashMatch);
}
}
if (candidateOtherRef != null) {
return candidateOtherRef;
}
if (shortRevisionAmbiguity) {

Check warning on line 1005 in src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Partially covered line

Line 1005 is only partially covered, one branch is missing
// Check if we have any other matches first, if not, return null as we cannot determine a match
return null;

Check warning on line 1007 in src/main/java/jenkins/plugins/git/AbstractGitSCMSource.java

View check run for this annotation

ci.jenkins.io / Code Coverage

Not covered line

Line 1007 is not covered by tests
}
// Pokémon!... Got to catch them all
listener.getLogger().printf("Could not find %s in remote references. "
+ "Pulling heads to local for deep search...%n", revision);
Expand All @@ -1015,7 +1023,6 @@
//just to be safe
listener.error("Could not resolve %s", revision);
return null;

}
hash = objectId.name();
String candidatePrefix = Constants.R_REMOTES.substring(Constants.R_REFS.length())
Expand Down
119 changes: 115 additions & 4 deletions src/test/java/jenkins/plugins/git/AbstractGitSCMSourceTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import com.cloudbees.plugins.credentials.common.StandardCredentials;
import com.cloudbees.plugins.credentials.domains.Domain;
import com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl;
import edu.umd.cs.findbugs.annotations.NonNull;
import hudson.EnvVars;
import hudson.FilePath;
import hudson.Launcher;
Expand All @@ -25,6 +26,7 @@
import hudson.plugins.git.extensions.impl.LocalBranch;
import hudson.util.StreamTaskListener;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
Expand All @@ -38,7 +40,6 @@
import jenkins.plugins.git.traits.IgnoreOnPushNotificationTrait;
import jenkins.plugins.git.traits.PruneStaleBranchTrait;
import jenkins.plugins.git.traits.TagDiscoveryTrait;

import jenkins.scm.api.SCMHead;
import jenkins.scm.api.SCMHeadObserver;
import jenkins.scm.api.SCMRevision;
Expand All @@ -53,6 +54,7 @@
import jenkins.scm.api.SCMSourceCriteria;
import jenkins.scm.api.SCMSourceOwner;
import jenkins.scm.api.metadata.PrimaryInstanceMetadataAction;
import jenkins.scm.api.trait.SCMSourceTrait;
import org.eclipse.jgit.errors.MissingObjectException;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.transport.RefSpec;
Expand Down Expand Up @@ -425,7 +427,7 @@ public void retrieveByName() throws Exception {
listener.getLogger().printf("%n=== fetch('%s') ===%n%n", devHash);
rev = source.fetch(devHash, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl) rev).getHash(), is(devHash));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(devHash));
assertThat(rev.getHead().getName(), is("dev"));

listener.getLogger().printf("%n=== fetch('%s') ===%n%n", devHash.substring(0, 10));
Expand Down Expand Up @@ -536,7 +538,7 @@ private void retrievePrimaryHead(boolean duplicatePrimary) throws Exception {
}

PrimaryInstanceMetadataAction primary = null;
for (Action a: actions) {
for (Action a : actions) {
if (a instanceof PrimaryInstanceMetadataAction) {
primary = (PrimaryInstanceMetadataAction) a;
break;
Expand Down Expand Up @@ -835,7 +837,7 @@ public void fetchOtherRef() throws Exception {
StreamTaskListener listener = StreamTaskListener.fromStderr();

final SCMHeadObserver.Collector collector =
source.fetch((SCMSourceCriteria) (probe, listener1) -> true, new SCMHeadObserver.Collector(), listener);
source.fetch((SCMSourceCriteria) (probe, listener1) -> true, new SCMHeadObserver.Collector(), listener);

final Map<SCMHead, SCMRevision> result = collector.result();
assertThat(result.entrySet(), hasSize(4));
Expand Down Expand Up @@ -1157,6 +1159,115 @@ public void when_commits_added_during_discovery_we_do_not_crash() throws Excepti
sharedSampleRepo = null;
}
}

@Issue("JENKINS-62592")
@Test
public void exactBranchMatchShouldSupersedePartialBranchMatch() throws Exception {
sampleRepo.init();
String masterHash = sampleRepo.head();
sampleRepo.git("checkout", "-b", "dev");
sampleRepo.write("file", "modified");
sampleRepo.git("commit", "--all", "--message=dev");
sampleRepo.git("tag", "v1");
String v1Hash = sampleRepo.head();
sampleRepo.write("file", "modified2");
sampleRepo.git("commit", "--all", "--message=dev2");
sampleRepo.git("tag", "-a", "v2", "-m", "annotated");
String v2Hash = sampleRepo.head();
sampleRepo.write("file", "modified3");
sampleRepo.git("commit", "--all", "--message=dev3");
// Grab the devHash, but lets try and generate a hash that we know will cause an issue in our test
ArrayList<String> hashFirstLetter = new ArrayList<>(
Arrays.asList(masterHash.substring(0, 1), v1Hash.substring(0, 1), v2Hash.substring(0, 1))
);
sampleRepo.git("tag", "devTag");
String devTagHash = sampleRepo.head();
int devTagIteration = 4; // In order to name new files and create new commits
String previousNewHash = null;
String newHash = devTagHash;
while (!hashFirstLetter.contains(devTagHash.substring(0, 1))) {
// Generate a new commit and try again
sampleRepo.git("tag", "devTag" + devTagIteration);
sampleRepo.write("file", "modified" + devTagIteration);
sampleRepo.git("commit", "--all", "--message=dev" + (devTagIteration++));
previousNewHash = newHash;
newHash = sampleRepo.head();
hashFirstLetter.add(newHash.substring(0, 1));
}
GitSCMSource source = new GitSCMSource(sampleRepo.toString());
source.setTraits(new ArrayList<>());

TaskListener listener = StreamTaskListener.fromStderr();
listener.getLogger().printf("ArrayList of first hash chars: %s", hashFirstLetter);

// Test existing functionality with additional revisions
listener.getLogger().println("\n=== fetch('master') ===\n");
SCMRevision rev = source.fetch("master", listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(masterHash));
listener.getLogger().println("\n=== fetch('v1') ===\n");
rev = source.fetch("v1", listener, null);
assertThat(rev, instanceOf(GitTagSCMRevision.class));
assertThat(((GitTagSCMRevision)rev).getHash(), is(v1Hash));
listener.getLogger().println("\n=== fetch('v2') ===\n");
rev = source.fetch("v2", listener, null);
assertThat(rev, instanceOf(GitTagSCMRevision.class));
assertThat(((GitTagSCMRevision)rev).getHash(), is(v2Hash));

listener.getLogger().printf("%n=== fetch('%s') ===%n%n", masterHash);
rev = source.fetch(masterHash, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(masterHash));
assertThat(rev.getHead().getName(), is("master"));

// Test new functionality and verify that we are able to grab what we expect (full match versus hazy short hash)
if (!devTagHash.equals(newHash)) {
listener.getLogger().printf("%n=== fetch('%s') ===%n%n", newHash);
rev = source.fetch(newHash, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(newHash));
assertThat(rev.getHead().getName(), is("dev"));

listener.getLogger().printf("%n=== fetch('%s') short hash ===%n%n", newHash.substring(0, 6));
rev = source.fetch(newHash.substring(0, 6), listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(newHash));
assertThat(rev.getHead().getName(), is("dev"));


int lastDevTag = devTagIteration == 4 ? 4 : devTagIteration - 1;
String headHash = devTagIteration == 4 ? devTagHash : previousNewHash;
listener.getLogger().printf("%n=== fetch(devTag%d) ===%n%n", lastDevTag);
rev = source.fetch("devTag" + lastDevTag, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(rev.getHead().getName(), is("devTag" + lastDevTag));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(headHash));

String ambiguousTag = hashFirstLetter.get(hashFirstLetter.size() - 1);
String ambiguousHash = sampleRepo.head();
sampleRepo.git("tag", ambiguousTag);
sampleRepo.write("file", "modified and ambiguous");
sampleRepo.git("commit", "--all", "--message=ambiguousTagCommit");
listener.getLogger().printf("%n=== fetch(%s) ===%n%n", ambiguousTag);
rev = source.fetch(ambiguousTag, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(rev.getHead().getName(), is(ambiguousTag));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(ambiguousHash));

// Remove ambiguous tag from the repo and thne create a branch with the ambiguous search
sampleRepo.git("tag", "-d", ambiguousTag);
sampleRepo.git("checkout", "-b", ambiguousTag);
sampleRepo.write("file", "modified and ambiguous but a branch this time!");
sampleRepo.git("commit", "--all", "--message=ambiguousBranchCommit");
listener.getLogger().printf("%n=== fetch(%s) branch ===%n%n", ambiguousTag);
String ambiguousBranchHash = sampleRepo.head();
rev = source.fetch(ambiguousTag, listener, null);
assertThat(rev, instanceOf(AbstractGitSCMSource.SCMRevisionImpl.class));
assertThat(rev.getHead().getName(), is(ambiguousTag));
assertThat(((AbstractGitSCMSource.SCMRevisionImpl)rev).getHash(), is(ambiguousBranchHash));
}
}

//Ugly but MockGitClient needs to be static and no good way to pass it on
static GitSampleRepoRule sharedSampleRepo;

Expand Down