TS-38561 Rework

CHANGELOG.md

@@ -5,7 +5,7 @@ We use [semantic versioning](http://semver.org/):
 - PATCH version when you make backwards compatible bug fixes.
 
 # Next Release
-- [feature] _agent_: Support overwriting the git commit inside `git.properties` files with `teamscale.timestamp` (see [docs](agent/README.md#overwriting-the-git-commit-inside-gitproperties-files-with-teamscaletimestamp))
+- [feature] _agent_: Support overwriting the git commit inside `git.properties` files with `teamscale.commit.branch` and `teamscale.commit.time` (see [docs](agent/README.md#overwriting-the-git-commit-inside-gitproperties-files-with-teamscalecommitbranch-and-teamscalecommittime))
 - [fix] _agent_: New default date format of the [git-commit-id-maven-plugin](https://github.com/git-commit-id/git-commit-id-maven-plugin) could not be parsed (see [GitHub Issue](https://github.com/cqse/teamscale-jacoco-agent/issues/464))
 
 # 33.2.0

agent/README.md

@@ -158,8 +158,9 @@ directories, you can get the commit info via
   a `git.properties` file generated with [the corresponding Maven or Gradle plugin][git-properties-spring] and stored in a jar/war/ear/...
   If nothing is configured, the agent automatically searches all loaded Jar/War/Ear/... files for a `git.properties` file.
   This file must contain either 
+  - the one of the properties `git.commit.id` or `git.commit.id.full` with the git SHA1
   - the properties `git.branch` and `git.commit.time` (in the format `yyyy-MM-dd'T'HH:mm:ssZ` or `yyyy-MM-dd'T'HH:mm:ssXXX`) or
-  - the property `teamscale.timestamp` in the format `branch:timestamp`. In this case, the timestamp can either be an epoch timestamp or in one of the two formats above. 
+  - the properties `teamscale.commit.branch` and `teamscale.commit.time` (either as an epoch timestamp or in one of the two formats above)
 - `search-git-properties-recursively` Specifies whether to search for git.properties files recursively in folders or archive (jar, war, ear, aar) files. Default: true.
 - `teamscale-message` (optional): the commit message shown within Teamscale for the coverage upload (Default is "Agent
   coverage upload").
@@ -567,12 +568,13 @@ jar {
 ./gradlew jar -Dbranch=master
 ```
 
-## Overwriting the git commit inside `git.properties` files with `teamscale.timestamp`
+## Overwriting the git commit inside `git.properties` files with `teamscale.commit.branch` and `teamscale.commit.time`
+
 There are scenarios, in which you need to use a different branch and time for the upload than provided with `git.commit.id` or `git.branch` and `git.commit.time` in a `git.properties` file.
-This can be achieved by providing a `teamscale.timestamp` property inside the `git.properties` file.
-This property must be provided in the format `branch:timestamp` and the timestamp within that either as epoch timestamp in milliseconds or in one of these two formats: `yyyy-MM-dd'T'HH:mm:ssZ` or `yyyy-MM-dd'T'HH:mm:ssXXX`.
+This can be achieved by providing the properties `teamscale.commt.branch` and `teamsacle.commit.time` inside the `git.properties` file.
+`teamscale.commit.time` must be provided either as epoch timestamp in milliseconds or in one of these two formats: `yyyy-MM-dd'T'HH:mm:ssZ` or `yyyy-MM-dd'T'HH:mm:ssXXX`.
 
-The following explains, how to use the build timestamp instead of the commit timestamp for Maven and Gradle. Thus, we write `teamscale.timestamp=<branch>:<build-times>` into the `git.properties` file.
+The following explains, how to use the build timestamp instead of the commit timestamp for Maven and Gradle. Thus, we write `teamscale.commit.branch=<branch>` and `teamscale.commit.time=<build-time>` into the `git.properties` file.
 
 ### Maven
 * Set up the git-commit-id maven plugin (see also [docs](https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/docs/using-the-plugin.md#basic-configuration--basic-usage-of-the-plugin)):
@@ -601,7 +603,7 @@ The following explains, how to use the build timestamp instead of the commit tim
     ```
 
 * Set up custom entries in the `git.properties` file (see also [advanced docs](https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/docs/using-the-plugin-in-more-depth.md#maven-resource-filtering)):
-  * Add a file called `git.properties` in the resources folder (usually `java/main/ressources`) of your application. The following one contains all default properties + the `teamscale.project` and `teamscale.timestamp` properties.
+  * Add a file called `git.properties` in the resources folder (usually `java/main/ressources`) of your application. The following one contains all default properties + the `teamscale.project`, `teamscale.commit.branch` and `teamscale.commit.time` properties.
     ```properties
     git.tags=${git.tags}
     git.branch=${git.branch}
@@ -628,9 +630,11 @@ The following explains, how to use the build timestamp instead of the commit tim
     git.build.number=${git.build.number}
     git.build.number.unique=${git.build.number.unique}
     teamscale.project=my-teamscale-project-id
-    teamscale.timestamp=${git.branch}:${git.build.time}
+    teamscale.commit.branch=${git.branch}
+    teamscale.commit.time=${git.build.time}
     # alternatively via environment variables
-    # teamscale.timestamp=${@env.BRANCH}:${env.BUILD_TIME}
+    # teamscale.commit.branch=${@env.BRANCH}
+    # teamscale.commit.time=${env.BUILD_TIME}
     ```
   * Add the following to your pom file so the commit id plugin picks up your custom `git.properties` file:
     ```xml
@@ -659,13 +663,16 @@ The following explains, how to use the build timestamp instead of the commit tim
   ```groovy
   gitProperties {
     customProperty "teamscale.project", "my-teamscale-project-id"
-    customProperty "teamscale.timestamp", {
-        // If you have multiple git.properties, you can also add a custom project property at the start of your build. 
-        // See https://stackoverflow.com/a/7029021
-        "${it.branch.current().getName()}:${new Date().toInstant().toEpochMilli()}"
-        // alternatively via environment variables
-        //"${System.getenv("BRANCH")}:${System.getenv("BUILD_TIME")}"
-
+    customProperty "teamscale.commit.branch", {
+      // If you have multiple git.properties, you can also add a custom project property at the start of your build. See https://stackoverflow.com/a/7029021
+      "${it.branch.current().getName()}"
+      // alternatively via environment variable
+      // "${System.getenv("BRANCH")}"
+    }
+    customProperty "teamscale.commit.time", {
+      "${new Date().toInstant().toEpochMilli()}"
+      // alternatively via environment variable
+      // "${System.getenv("BUILD_TIME")}"
     }
   }
   ```

agent/src/main/java/com/teamscale/jacoco/agent/commit_resolution/git_properties/CommitInfo.java

@@ -14,8 +14,8 @@ public class CommitInfo {
 	public CommitDescriptor commit;
 
 	/**
-	 * If the commit property is set via the <code>teamscale.timestamp</code> property in a git.properties file, this
-	 * should be preferred to the revision. For details see <a
+	 * If the commit property is set via the <code>teamscale.commit.branch</code> and <code>teamscale.commit.time</code>
+	 * properties in a git.properties file, this should be preferred to the revision. For details see <a
 	 * href="https://cqse.atlassian.net/browse/TS-38561">TS-38561</a>.
 	 */
 	public boolean preferCommitDescriptorOverRevision = false;

agent/src/main/java/com/teamscale/jacoco/agent/commit_resolution/git_properties/GitMultiProjectPropertiesLocator.java

@@ -72,7 +72,7 @@ void searchFile(File file, boolean isJarFile) {
 					logger.debug(
 							"Found inconsistent git.properties file: the git.properties file in {} either does not specify the" +
 									" Teamscale project (teamscale.project) property, or does not specify the commit " +
-									"(git.commit.id, git.commit.branch + git.commit.time, or teamscale.timestamp) or ." +
+									"(git.commit.id, git.commit.branch + git.commit.time, or teamscale.commit.branch + teamscale.commit.time) or ." +
 									" Will skip this git.properties file and try to continue with the other ones that were found during discovery.",
 							file);
 					continue;

agent/src/main/java/com/teamscale/jacoco/agent/commit_resolution/git_properties/GitPropertiesLocatorUtils.java

@@ -23,7 +23,6 @@
 import java.time.format.DateTimeFormatterBuilder;
 import java.time.format.DateTimeParseException;
 import java.util.ArrayList;
-import java.util.Arrays;
 import java.util.List;
 import java.util.Properties;
 import java.util.jar.JarEntry;
@@ -56,7 +55,13 @@ public class GitPropertiesLocatorUtils {
 	 * You can provide a teamscale timestamp in git.properties files to overwrite the revision. See <a
 	 * href="https://cqse.atlassian.net/browse/TS-38561">TS-38561</a>.
 	 */
-	static final String GIT_PROPERTIES_TEAMSCALE_TIMESTAMP = "teamscale.timestamp";
+	static final String GIT_PROPERTIES_TEAMSCALE_COMMIT_BRANCH = "teamscale.commit.branch";
+
+	/**
+	 * You can provide a teamscale timestamp in git.properties files to overwrite the revision. See <a
+	 * href="https://cqse.atlassian.net/browse/TS-38561">TS-38561</a>.
+	 */
+	static final String GIT_PROPERTIES_TEAMSCALE_COMMIT_TIME = "teamscale.commit.time";
 
 	/** The git.properties key that holds the Teamscale project name. */
 	private static final String GIT_PROPERTIES_TEAMSCALE_PROJECT = "teamscale.project";
@@ -333,9 +338,10 @@ static List<Pair<String, Properties>> findGitPropertiesInArchive(
 	 * Returns the CommitInfo (revision & branch + timestmap) from a git properties file. The revision can be either in
 	 * {@link #GIT_PROPERTIES_GIT_COMMIT_ID} or {@link #GIT_PROPERTIES_GIT_COMMIT_ID_FULL}. The branch and timestamp in
 	 * {@link #GIT_PROPERTIES_GIT_BRANCH} + {@link #GIT_PROPERTIES_GIT_COMMIT_TIME} or in
-	 * {@link #GIT_PROPERTIES_TEAMSCALE_TIMESTAMP} By default, times will be parsed with
-	 * {@link #GIT_PROPERTIES_DEFAULT_GRADLE_DATE_FORMAT} and {@link #GIT_PROPERTIES_DEFAULT_MAVEN_DATE_FORMAT}. An
-	 * additional format can be given with {@code dateTimeFormatter}
+	 * {@link #GIT_PROPERTIES_TEAMSCALE_COMMIT_BRANCH} + {@link #GIT_PROPERTIES_TEAMSCALE_COMMIT_TIME}. By default,
+	 * times will be parsed with {@link #GIT_PROPERTIES_DEFAULT_GRADLE_DATE_FORMAT} and
+	 * {@link #GIT_PROPERTIES_DEFAULT_MAVEN_DATE_FORMAT}. An additional format can be given with
+	 * {@code dateTimeFormatter}
 	 */
 	public static CommitInfo getCommitInfoFromGitProperties(
 			Properties gitProperties, String entryName, File jarFile,
@@ -348,16 +354,14 @@ public static CommitInfo getCommitInfoFromGitProperties(
 
 		// Get branch and timestamp from git.commit.branch and git.commit.id
 		CommitDescriptor commitDescriptor = getCommitDescriptorFromDefaultGitPropertyValues(gitProperties, entryName,
-				jarFile,
-				dateTimeFormatter);
+				jarFile, dateTimeFormatter);
 		// When read from these properties, we should prefer to upload to the revision
 		boolean preferCommitDescriptorOverRevision = false;
 
 
-		// Get branch and timestamp from teamscale.timestamp (TS-38561)
+		// Get branch and timestamp from teamscale.commit.branch and teamscale.commit.time (TS-38561)
 		CommitDescriptor teamscaleTimestampBasedCommitDescriptor = getCommitDescriptorFromTeamscaleTimestampProperty(
-				gitProperties,
-				entryName, jarFile, dateTimeFormatter);
+				gitProperties, entryName, jarFile, dateTimeFormatter);
 		if (teamscaleTimestampBasedCommitDescriptor != null) {
 			// In this case, as we specifically set this property, we should prefer branch and timestamp to the revision
 			preferCommitDescriptorOverRevision = true;
@@ -367,8 +371,8 @@ public static CommitInfo getCommitInfoFromGitProperties(
 		if (StringUtils.isEmpty(revision) && commitDescriptor == null) {
 			throw new InvalidGitPropertiesException(
 					"No entry or invalid value for '" + GIT_PROPERTIES_GIT_COMMIT_ID + "', '" + GIT_PROPERTIES_GIT_COMMIT_ID_FULL +
-							"' and " + GIT_PROPERTIES_TEAMSCALE_TIMESTAMP + "'\n" +
-							"In " + entryName + " in " + jarFile + "." +
+							"', '" + GIT_PROPERTIES_TEAMSCALE_COMMIT_BRANCH + "' and " + GIT_PROPERTIES_TEAMSCALE_COMMIT_TIME + "'\n" +
+							"Location: Entry '" + entryName + "' in jar file '" + jarFile + "'." +
 							"\nContents of " + GIT_PROPERTIES_FILE_NAME + ":\n" + gitProperties);
 		}
 
@@ -400,40 +404,33 @@ private static String getRevisionFromGitProperties(Properties gitProperties) {
 	private static CommitDescriptor getCommitDescriptorFromTeamscaleTimestampProperty(Properties gitProperties,
 			String entryName, File jarFile,
 			DateTimeFormatter dateTimeFormatter) throws InvalidGitPropertiesException {
-		String teamscaleTimestampProperty = gitProperties.getProperty(GIT_PROPERTIES_TEAMSCALE_TIMESTAMP);
-		if (!StringUtils.isEmpty(teamscaleTimestampProperty)) {
-			String[] split = teamscaleTimestampProperty.split(":");
-			if (split.length < 2) {
-				throw new InvalidGitPropertiesException(
-						String.format("%s needs to be in the format <branch>:<timestamp>. You provided %s" +
-										"\nIn " + entryName + " in " + jarFile + "." +
-										"\nContents of " + GIT_PROPERTIES_FILE_NAME + ":\n" + gitProperties,
-								GIT_PROPERTIES_TEAMSCALE_TIMESTAMP, teamscaleTimestampProperty));
-			}
-			String branch = split[0];
-			String timestamp = String.join(":", Arrays.copyOfRange(split, 1, split.length));
-			String teamscaleTimestampRegex = "\\d*(?:p\\d*)?";
-			Matcher teamscaleTimestampMatcher = Pattern.compile(teamscaleTimestampRegex).matcher(timestamp);
-			if (teamscaleTimestampMatcher.matches()) {
-				return new CommitDescriptor(branch, timestamp);
-			}
+		String teamscaleCommitBranch = gitProperties.getProperty(GIT_PROPERTIES_TEAMSCALE_COMMIT_BRANCH);
+		String teamscaleCommitTime = gitProperties.getProperty(GIT_PROPERTIES_TEAMSCALE_COMMIT_TIME);
 
-			long epochTimestamp;
-			try {
-				epochTimestamp = ZonedDateTime.parse(timestamp, dateTimeFormatter).toInstant().toEpochMilli();
-			} catch (DateTimeParseException e) {
-				throw new InvalidGitPropertiesException(
-						"Cannot parse commit time '" + timestamp + "' in the '" + GIT_PROPERTIES_TEAMSCALE_TIMESTAMP +
-								"' property. It needs to be in the date formats '" + GIT_PROPERTIES_DEFAULT_MAVEN_DATE_FORMAT +
-								"' or '" + GIT_PROPERTIES_DEFAULT_GRADLE_DATE_FORMAT + "' or match the Teamscale timestamp format '"
-								+ teamscaleTimestampRegex + "'." +
-								"\nIn " + entryName + " in " + jarFile + "." +
-								"\nContents of " + GIT_PROPERTIES_FILE_NAME + ":\n" + gitProperties, e);
-			}
+		if (StringUtils.isEmpty(teamscaleCommitBranch) || StringUtils.isEmpty(teamscaleCommitTime)) {
+			return null;
+		}
 
-			return new CommitDescriptor(branch, epochTimestamp);
+		String teamscaleTimestampRegex = "\\d*(?:p\\d*)?";
+		Matcher teamscaleTimestampMatcher = Pattern.compile(teamscaleTimestampRegex).matcher(teamscaleCommitTime);
+		if (teamscaleTimestampMatcher.matches()) {
+			return new CommitDescriptor(teamscaleCommitBranch, teamscaleCommitTime);
 		}
-		return null;
+
+		long epochTimestamp;
+		try {
+			epochTimestamp = ZonedDateTime.parse(teamscaleCommitTime, dateTimeFormatter).toInstant().toEpochMilli();
+		} catch (DateTimeParseException e) {
+			throw new InvalidGitPropertiesException(
+					"Cannot parse commit time '" + teamscaleCommitTime + "' in the '" + GIT_PROPERTIES_TEAMSCALE_COMMIT_TIME +
+							"' property. It needs to be in the date formats '" + GIT_PROPERTIES_DEFAULT_MAVEN_DATE_FORMAT +
+							"' or '" + GIT_PROPERTIES_DEFAULT_GRADLE_DATE_FORMAT + "' or match the Teamscale timestamp format '"
+							+ teamscaleTimestampRegex + "'." +
+							"\nLocation: Entry '" + entryName + "' in jar file '" + jarFile + "'." +
+							"\nContents of " + GIT_PROPERTIES_FILE_NAME + ":\n" + gitProperties, e);
+		}
+
+		return new CommitDescriptor(teamscaleCommitBranch, epochTimestamp);
 	}
 
 	private static CommitDescriptor getCommitDescriptorFromDefaultGitPropertyValues(Properties gitProperties,
@@ -448,7 +445,7 @@ private static CommitDescriptor getCommitDescriptorFromDefaultGitPropertyValues(
 			} catch (DateTimeParseException e) {
 				throw new InvalidGitPropertiesException(
 						"Could not parse the timestamp in property '" + GIT_PROPERTIES_GIT_COMMIT_TIME + "'." +
-								"\nIn " + entryName + " in " + jarFile + "." +
+								"\nLocation: Entry '" + entryName + "' in jar file '" + jarFile + "'." +
 								"\nContents of " + GIT_PROPERTIES_FILE_NAME + ":\n" + gitProperties, e);
 			}
 			return new CommitDescriptor(gitBranch, gitTimestamp);

agent/src/main/java/com/teamscale/jacoco/agent/options/AgentOptions.java

@@ -497,7 +497,7 @@ private DelayedTeamscaleMultiProjectUploader createTeamscaleMultiProjectUploader
 			Instrumentation instrumentation) {
 		DelayedTeamscaleMultiProjectUploader uploader = new DelayedTeamscaleMultiProjectUploader(
 				(project, commitInfo) -> {
-					if (commitInfo.preferCommitDescriptorOverRevision) {
+					if (commitInfo.preferCommitDescriptorOverRevision || StringUtils.isEmpty(commitInfo.revision)) {
 						return new TeamscaleUploader(teamscaleServer.withProjectAndCommit(project, commitInfo.commit));
 					}
 					return new TeamscaleUploader(teamscaleServer.withProjectAndRevision(project, commitInfo.revision));
@@ -544,7 +544,8 @@ private DelayedUploader<ProjectAndCommit> createDelayedSingleProjectTeamscaleUpl
 										" Teamscale project '{}' specified in the agent configuration.",
 								teamscaleServer.project, projectAndCommit.getProject(), teamscaleServer.project);
 					}
-					if (projectAndCommit.getCommitInfo().preferCommitDescriptorOverRevision) {
+					if (projectAndCommit.getCommitInfo().preferCommitDescriptorOverRevision ||
+							StringUtils.isEmpty(projectAndCommit.getCommitInfo().revision)) {
 						teamscaleServer.commit = projectAndCommit.getCommitInfo().commit;
 					} else {
 						teamscaleServer.revision = projectAndCommit.getCommitInfo().revision;