Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicitly search for bookkeeper-server jar #772

Closed
wants to merge 2 commits into from

Conversation

ivankelly
Copy link
Contributor

@ivankelly ivankelly commented Nov 23, 2017

Rather than listing all jars, and excluding some exceptions, search for
jars which match a specific pattern, namely:

bookkeeper-server-[version].jar

Where [version] may include numbers, dots and optionally -SNAPSHOT.

@ivankelly ivankelly self-assigned this Nov 23, 2017
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@sijie
Copy link
Member

sijie commented Nov 24, 2017

from programming perspective, this change looks good.

However, I didn't encounter this problem in current master:

I tried two ways:

$ mvn clean install -DskipTests

$ cd bookkeeper-server

$ ls target/
apidocs                                           checkstyle-checker.xml                            javadoc-bundle-options
bookkeeper-server-4.6.0-SNAPSHOT-javadoc.jar      checkstyle-result.xml                             maven-archiver
bookkeeper-server-4.6.0-SNAPSHOT-shaded-tests.jar checkstyle-suppressions.xml                       maven-shared-archive-resources
bookkeeper-server-4.6.0-SNAPSHOT-shaded.jar       classes                                           maven-status
bookkeeper-server-4.6.0-SNAPSHOT-sources.jar      generated-sources                                 rat.txt
bookkeeper-server-4.6.0-SNAPSHOT-tests.jar        generated-test-sources                            test-classes
bookkeeper-server-4.6.0-SNAPSHOT.jar              invoker

$ bin/bookkeeper shell simpletest
JMX enabled by default
Ledger ID: 0
396 entries written
1000 entries written to ledger 0
  1. I removed lib/ and target/ from bookkeeper-server. and run steps at 1. all the tests passed.

so I don't think this behavior has been broken since we added sources package. I might be a problem that command would produce different results when running on different shells.

@@ -71,18 +71,18 @@ else
fi

# exclude tests jar
RELEASE_JAR=$(ls ${BK_HOME}/*bookkeeper-server-*.jar 2> /dev/null | grep -v tests | tail -1)
RELEASE_JAR=$(ls ${BK_HOME}/*bookkeeper-server-*.jar 2> /dev/null | grep -v tests | grep -v sources | tail -1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the problem comes from multiple package, or your shell doesn't emit the right result. You need to exclude following packages:

target/bookkeeper-server-4.6.0-SNAPSHOT-javadoc.jar
target/bookkeeper-server-4.6.0-SNAPSHOT-shaded.jar
target/bookkeeper-server-4.6.0-SNAPSHOT-sources.jar

However I would suggest digging into this issue a bit more and come up a command that can produce determistic results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange. Yes, we should make this more deterministic.

My shell(debian stable, bash) collates differently:

apidocs                                            bookkeeper-server-4.7.0-SNAPSHOT-sources.jar  classes                 maven-archiver
bookkeeper-server-4.7.0-SNAPSHOT.jar               bookkeeper-server-4.7.0-SNAPSHOT-tests.jar    generated-sources       maven-shared-archive-resources
bookkeeper-server-4.7.0-SNAPSHOT-javadoc.jar       checkstyle-checker.xml                        generated-test-sources  maven-status
bookkeeper-server-4.7.0-SNAPSHOT-shaded.jar        checkstyle-result.xml                         invoker                 test-classes
bookkeeper-server-4.7.0-SNAPSHOT-shaded-tests.jar  checkstyle-suppressions.xml                   javadoc-bundle-options

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is why I was seeing the shading problem also.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the result returned by ls is unsorted. that means it will have different behaviors on different bash (mac vs linux).

a deterministic way is to ls ... | sort | tail -1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed ls would lexically sort by default, but I guess that would make ls very expensive on large directories. In any case, I think I'll either check that there's a version before the ".jar" or maybe directly use the shaded. This may fix #773 also.

Copy link
Member

@jiazhai jiazhai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the former change of bin/bookkeeper on mac, seems not meet this issue.

bin/bookkeeper tries to figure out the location of the bookkeeper-server
jar, so that a locally built jar can be run in place. However, this has
been broken since we've been generating a sources jar.

This patch excludes the sources jar from the search.
Rather than listing all jars, and excluding some exceptions, search for
jars which match a specific pattern, namely:

  bookkeeper-server-<version>.jar

Where <version> may include numbers, dots and optionally -SNAPSHOT.
@ivankelly ivankelly changed the title bookkeeper script excludes sources jar Explicitly search for bookkeeper-server jar Nov 27, 2017
@ivankelly
Copy link
Contributor Author

@jiazhai ya, I think it was filesystem specific.

@sijie I've updated to search explicitly for the correct jar. This fixes the shading issue also.

Copy link
Contributor

@merlimat merlimat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

A bit of a side note, but can we also remove the " JMX enabled by default line that gets printed from the bash script?

@ivankelly
Copy link
Contributor Author

@merlimat I'll create another PR

@ivankelly ivankelly added this to the 4.6.0 milestone Nov 28, 2017
@ivankelly ivankelly closed this in 90a82c2 Nov 28, 2017
ivankelly added a commit that referenced this pull request Nov 28, 2017
Rather than listing all jars, and excluding some exceptions, search for
jars which match a specific pattern, namely:

  bookkeeper-server-[version].jar

Where [version] may include numbers, dots and optionally -SNAPSHOT.

Author: Ivan Kelly <ivank@apache.org>

Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org>, Matteo Merli <mmerli@apache.org>

This closes #772 from ivankelly/fix-local-bookie

(cherry picked from commit 90a82c2)
Signed-off-by: Ivan Kelly <ivank@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants