Enforce Java 11 in archetypes #1029

msghasan · 2023-01-07T15:10:46Z

Signed-off-by: Maimur Hasan msghasan@gmail.com

Thanks for contributing to StormCrawler, your efforts are appreciated!

Developer Certificate of Origin

By contributing to StormCrawler, you accept and agree to the following terms and conditions (the Developer Certificate of Origin) for your present and future contributions submitted to StormCrawler.
Please refer to the Developer Certificate of Origin section in CONTRIBUTING.md for details.

Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Before opening a PR, please check that:

You've squashed your commits into a single one
You've described what the PR does or at least point to a related issue
You've signed-ff your commits with 'git commit -s'
The code is properly formatted with 'mvn git-code-format:format-code -Dgcf.globPattern=**/*'

Thanks!

Signed-off-by: Maimur Hasan <msghasan@gmail.com>

msghasan · 2023-01-07T15:18:55Z

@jnioche Shall I remove the below plugin from parent pom , as it do not support code above jdk 8

com.cosium.code
git-code-format-maven-plugin
3.4

jnioche · 2023-01-07T16:20:08Z

Let's keep it until we find a replacement for it.

- Did refactoring of the code. Now the class is more reusable anybody and extend the class and can use it as per their business requirement without having to copy paste everything from the existing store method - Secondly if we can use an utilty class in the core project and copy paste the store() method that would have been nice we could have used htis in both places for elastic search and opensearch StatusUpdaterBolt Signed-off-by: Maimur Hasan <msghasan@gmail.com>

sonatype-lift · 2023-01-07T18:16:40Z

...h/src/main/java/com/digitalpebble/stormcrawler/opensearch/persistence/StatusUpdaterBolt.java

+    protected void addToTupleListAndSendToES(String url, Tuple tuple, String sha256hex, IndexRequest request) {
+        waitAckLock.lock();
+        try {
+            final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new LinkedList<>());


JdkObsolete: It is very rare for LinkedList to out-perform ArrayList or ArrayDeque. Avoid it unless you're willing to invest a lot of time into benchmarking. Caveat: LinkedList supports null elements, but ArrayDeque does not.

Suggested change

final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new LinkedList<>());

final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new ArrayList<>());

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

sonatype-lift · 2023-01-07T18:16:42Z

...rc/main/java/com/digitalpebble/stormcrawler/elasticsearch/persistence/StatusUpdaterBolt.java

+    protected void addToTupleListAndSendToES(String url, Tuple tuple, String sha256hex, IndexRequest request) {
+        waitAckLock.lock();
+        try {
+            final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new LinkedList<>());


JdkObsolete: It is very rare for LinkedList to out-perform ArrayList or ArrayDeque. Avoid it unless you're willing to invest a lot of time into benchmarking. Caveat: LinkedList supports null elements, but ArrayDeque does not.

Suggested change

final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new LinkedList<>());

final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new ArrayList<>());

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

sonatype-lift · 2023-01-07T18:16:43Z

...rc/main/java/com/digitalpebble/stormcrawler/elasticsearch/persistence/StatusUpdaterBolt.java

+
+        LOG.debug("Sending to ES buffer {} with ID {}", url, sha256hex);
+
+        connection.addToProcessor(request);


THREAD_SAFETY_VIOLATION: Read/Write race. Non-private method StatusUpdaterBolt.addToTupleListAndSendToES(...) reads without synchronization from this.connection. Potentially races with write in method StatusUpdaterBolt.cleanup().
Reporting because this access may occur on a background thread.

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

sonatype-lift · 2023-01-07T18:16:45Z

...h/src/main/java/com/digitalpebble/stormcrawler/opensearch/persistence/StatusUpdaterBolt.java

+
+        LOG.debug("Sending to ES buffer {} with ID {}", url, sha256hex);
+
+        connection.addToProcessor(request);


THREAD_SAFETY_VIOLATION: Read/Write race. Non-private method StatusUpdaterBolt.addToTupleListAndSendToES(...) reads without synchronization from this.connection. Potentially races with write in method StatusUpdaterBolt.cleanup().
Reporting because this access may occur on a background thread.

ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage

@sonatype-lift ignore Leave out the above finding from this PR

@sonatype-lift ignoreall Leave out all the existing findings from this PR

@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
_{Click here to add LiftBot to another repo.}

Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

jnioche · 2023-01-08T07:24:09Z

git-code-format-maven-plugin

and it works when calling

mvn git-code-format:format-code -Dgcf.globPattern=**/*
with Java 11. So I am not sure why you'd want to remove it.

rzo1 · 2023-01-08T07:48:57Z

@msghasan can you restore the PR to only include the Java Version changes in the archetype?

msghasan · 2023-01-08T14:18:54Z

@msghasan can you restore the PR to only include the Java Version changes in the archetype?

yeah sure

msghasan · 2023-01-08T14:30:05Z

git-code-format-maven-plugin

@jnioche throws error stating this plugin only supports class version till 52 which is java 8 while we try to commit anything.... in to the repository

msghasan · 2023-01-08T15:53:43Z

@rzo1 Reverted the commit that I mistakenly did in this PR please approve the changes...

rzo1 · 2023-01-08T16:02:48Z

@jnioche don't know if squashing will retain signed commits. Aside from that lgtm.

msghasan · 2023-01-08T16:17:36Z

@rzo1 @jnioche Can I go ahead and close this pull request after the changes has been approved?

rzo1 · 2023-01-08T16:18:55Z

A PR is automatically closed if it is merged into master/main.

This reverts commit 81b85a1. Need to push in different PR Signed-off-by: Maimur Hasan <msghasan@gmail.com>

jnioche · 2023-01-09T12:48:50Z

Fixes #1022

jnioche · 2023-01-09T12:56:47Z

Merged, changed the title. Thanks @msghasan

Minor POM Changes

70c398c

Signed-off-by: Maimur Hasan <msghasan@gmail.com>

sonatype-lift bot reviewed Jan 7, 2023

View reviewed changes

rzo1 approved these changes Jan 8, 2023

View reviewed changes

Revert "Changes for Status Updater Bolt"

b910138

This reverts commit 81b85a1. Need to push in different PR Signed-off-by: Maimur Hasan <msghasan@gmail.com>

msghasan force-pushed the master branch from 46b2348 to b910138 Compare January 8, 2023 17:28

jnioche mentioned this pull request Jan 9, 2023

If stormcrawler above 2.5 uses Jdk 11 why the archetypes pom are not updated to 11 #1022

Closed

jnioche changed the title ~~Minor POM Changes for dependency upgrade~~ Enforce Java 11 in archetypes Jan 9, 2023

jnioche merged commit 2bf2a05 into apache:master Jan 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enforce Java 11 in archetypes #1029

Enforce Java 11 in archetypes #1029

msghasan commented Jan 7, 2023

msghasan commented Jan 7, 2023 •

edited

jnioche commented Jan 7, 2023

sonatype-lift bot Jan 7, 2023

sonatype-lift bot Jan 7, 2023

sonatype-lift bot Jan 7, 2023

sonatype-lift bot Jan 7, 2023

jnioche commented Jan 8, 2023

rzo1 commented Jan 8, 2023

msghasan commented Jan 8, 2023

msghasan commented Jan 8, 2023

msghasan commented Jan 8, 2023

rzo1 commented Jan 8, 2023

msghasan commented Jan 8, 2023

rzo1 commented Jan 8, 2023

jnioche commented Jan 9, 2023

jnioche commented Jan 9, 2023

	final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new LinkedList<>());
	final List<Tuple> tupleList = waitAck.get(sha256hex, k -> new ArrayList<>());

Command	Usage
`@sonatype-lift ignore`	Leave out the above finding from this PR
`@sonatype-lift ignoreall`	Leave out all the existing findings from this PR
`@sonatype-lift exclude <file\|issue\|path\|tool>`	Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file


		LOG.debug("Sending to ES buffer {} with ID {}", url, sha256hex);

		connection.addToProcessor(request);

Enforce Java 11 in archetypes #1029

Enforce Java 11 in archetypes #1029

Conversation

msghasan commented Jan 7, 2023

Developer Certificate of Origin

msghasan commented Jan 7, 2023 • edited

jnioche commented Jan 7, 2023

sonatype-lift bot Jan 7, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jan 7, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jan 7, 2023

Choose a reason for hiding this comment

sonatype-lift bot Jan 7, 2023

Choose a reason for hiding this comment

jnioche commented Jan 8, 2023

rzo1 commented Jan 8, 2023

msghasan commented Jan 8, 2023

msghasan commented Jan 8, 2023

msghasan commented Jan 8, 2023

rzo1 commented Jan 8, 2023

msghasan commented Jan 8, 2023

rzo1 commented Jan 8, 2023

jnioche commented Jan 9, 2023

jnioche commented Jan 9, 2023

msghasan commented Jan 7, 2023 •

edited