Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RATIS-1840. Avoid including build timestamp in artifacts #1114

Merged
merged 3 commits into from
Jul 2, 2024

Conversation

adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

buildnumber-maven-plugin's create-metadata goal generates a properties file similar to:

#Created by build system. Do not modify
#Fri Dec 29 09:18:39 CET 2023
version=3.0.1
revision=Unknown
name=Apache Ratis Common
timestamp=1703837919371

Time of build should be omitted, because it makes builds non-reproducible. The same set of sources and build instructions result in different output each time.

However, buildnumber-maven-plugin doesn't seem to have any way to configure the properties to be included/omitted. We can replace it with a custom properties file. hadoop-maven-plugins is used to get the SCM revision.

https://issues.apache.org/jira/browse/RATIS-1840

How was this patch tested?

Added a step in compile check to verify build is reproducible.

CI:
https://github.com/adoroszlai/ratis/actions/runs/9659530041

@adoroszlai adoroszlai self-assigned this Jun 25, 2024
@adoroszlai adoroszlai changed the title RATIS-1840. Ratis build is not fully reproducible RATIS-1840. Avoid including build timestamp in artifacts Jun 25, 2024
Copy link
Member

@SzyWilliam SzyWilliam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @adoroszlai for working on this!

Verified the unreproducible timestamp in the build (using master)

#Created by build system. Do not modify
#Sat Jun 29 21:58:45 CST 2024
version=3.2.0-SNAPSHOT
revision=c31806dbdcf2f288c57684958447a6ddfe6fc29c
name=Apache Ratis Project Assembly
timestamp=1719669525114

I tried to build Ratis after applying this patch and the original ratis-version.properties is gone. There are no output files (like the original properties file), is that an expected behavior?
Also, I tried to run mvn hadoops:version-info and got errors

on project ratis: The parameters 'source' for goal org.apache.hadoop:hadoop-maven-plugins:3.4.0:version-info are missing or invalid

, am I using it in an incorrect way?

@adoroszlai
Copy link
Contributor Author

adoroszlai commented Jun 29, 2024

Thanks @SzyWilliam for checking this.

Here are my steps:

mvn -DskipTests clean package > /dev/null
find * -name 'ratis-version.properties' | sort
echo # for separator line
cat ratis-common/target/classes/ratis-version.properties # sample

Output on master:

ratis-assembly/target/classes/ratis-version.properties
ratis-client/target/classes/ratis-version.properties
ratis-common/target/classes/ratis-version.properties
ratis-docs/target/classes/ratis-version.properties
ratis-examples/target/classes/ratis-version.properties
ratis-grpc/target/classes/ratis-version.properties
ratis-metrics-api/target/classes/ratis-version.properties
ratis-metrics-default/target/classes/ratis-version.properties
ratis-metrics-dropwizard3/target/classes/ratis-version.properties
ratis-netty/target/classes/ratis-version.properties
ratis-proto/target/classes/ratis-version.properties
ratis-replicated-map/target/classes/ratis-version.properties
ratis-resource-bundle/target/classes/ratis-version.properties
ratis-server-api/target/classes/ratis-version.properties
ratis-server/target/classes/ratis-version.properties
ratis-shell/target/classes/ratis-version.properties
ratis-test/target/classes/ratis-version.properties
ratis-tools/target/classes/ratis-version.properties
target/classes/ratis-version.properties

#Created by build system. Do not modify
#Sat Jun 29 16:40:20 CEST 2024
name=Apache Ratis Common
version=3.2.0-SNAPSHOT
timestamp=1719672020675
revision=c31806dbdcf2f288c57684958447a6ddfe6fc29c

Output with the patch:

ratis-client/target/classes/ratis-version.properties
ratis-common/target/classes/ratis-version.properties
ratis-docs/target/classes/ratis-version.properties
ratis-examples/target/classes/ratis-version.properties
ratis-grpc/target/classes/ratis-version.properties
ratis-metrics-api/target/classes/ratis-version.properties
ratis-metrics-default/target/classes/ratis-version.properties
ratis-metrics-dropwizard3/target/classes/ratis-version.properties
ratis-netty/target/classes/ratis-version.properties
ratis-proto/target/classes/ratis-version.properties
ratis-replicated-map/target/classes/ratis-version.properties
ratis-resource-bundle/target/classes/ratis-version.properties
ratis-server-api/target/classes/ratis-version.properties
ratis-server/target/classes/ratis-version.properties
ratis-shell/target/classes/ratis-version.properties
ratis-test/target/classes/ratis-version.properties
ratis-tools/target/classes/ratis-version.properties
src/main/resources/ratis-version.properties

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name=Apache Ratis Common
version=3.1.0-SNAPSHOT
revision=6d5c926e15aafbf542dfad573e72f6af48bf9a5a

With the patch, we have an extra file in src/main/resources, which is the template for the generated files.

On master, there is no source file, but there are two more output files:

ratis-assembly/target/classes/ratis-version.properties
target/classes/ratis-version.properties

However, I don't think these are ever used, since these modules have packaging=pom, not jar.

Copy link
Member

@SzyWilliam SzyWilliam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

find * -name 'ratis-version.properties' | sort
echo # for separator line
cat ratis-common/target/classes/ratis-version.properties # sample

+1 verified. Thanks @adoroszlai very much for working on this!

@SzyWilliam SzyWilliam merged commit d101678 into apache:master Jul 2, 2024
12 checks passed
@adoroszlai
Copy link
Contributor Author

Thanks @SzyWilliam for reviewing and merging this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants