Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low level Rest Client #18735

Merged
merged 109 commits into from Jun 22, 2016
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
3efbe95
RestClient prototype
s1monw Feb 22, 2016
c9c33a7
Fix checkstyle issues, setup thirdPartyAudit checks and enable forbid…
javanna Apr 8, 2016
5970723
clarify why jarhell and dependency license check are disabled, add ok…
javanna Apr 11, 2016
6ae8be5
Introduce notion of Transport, Connection and ConnectionPool
javanna Apr 28, 2016
2589235
remove Verb enum
javanna May 4, 2016
94cf843
get rid of retry timeout exception
javanna May 4, 2016
f6a5a0a
get rid of Node abstraction
javanna May 4, 2016
ce663e9
get rid of connection selector predicate
javanna May 4, 2016
a472544
move sniff related stuff to sniff package
javanna May 4, 2016
062a216
merge Connection and StatefulConnection into one class, remove generi…
javanna May 6, 2016
bd29dc1
Remove Transport class, move all of it within RestClient class
javanna May 6, 2016
9ffdea9
remove Scheme enum
javanna May 6, 2016
d7c4176
add some javadocs to connection pool classes
javanna May 6, 2016
e85ed3e
remove pinging from static connection pool, can be replaced by a low …
javanna May 6, 2016
e77ab87
return the response as part of ElasticsearchResponseException
javanna May 6, 2016
530ad22
prevent unclosed response entities in ElasticsearchResponseException,…
javanna May 6, 2016
e7fe397
add missing parentheses
javanna May 6, 2016
17a21f0
add curl format trace logging for requests and responses
javanna May 9, 2016
e040d2f
remove ConnectionPool interface
javanna May 9, 2016
9569ebc
add builders for simple creation of RestClient and SniffingConnection…
javanna May 9, 2016
599dad5
remove streams and java 8 only api, build with source and target 1.7
javanna May 9, 2016
b38ef34
remove streams leftover
javanna May 10, 2016
cdffc3d
remove notion of connection pool, turn around dependency between Rest…
javanna May 13, 2016
85a7721
simplify Connection class
javanna May 13, 2016
c0a72c1
add support for headers: default ones and per request
javanna May 13, 2016
2d7a781
fix usage of deprecated apis
javanna May 17, 2016
2735897
use versions from versions.properties
javanna May 17, 2016
3890842
add project substitution for org.elasticsearch:client so that we can …
javanna May 17, 2016
1d06916
adapt params argument, they can only be Strings in performRequest method
javanna May 18, 2016
16ab491
add getFirstHeader method to ElasticsearchResponse
javanna May 18, 2016
9a38d81
Expose whole ElasticsearchResponse within ElasticsearchResponseException
javanna May 18, 2016
6d3f6c7
support missing OPTIONS method, it is supported in elasticsearch core
javanna May 18, 2016
325b723
[TEST] add rest client test dependency and replace usage of HttpReque…
javanna May 18, 2016
eae914a
Replace rest test client with low level RestClient
javanna May 19, 2016
c70e08c
include response body in ElasticsearchResponseException error message
javanna May 20, 2016
e81aad9
remove usage of deprecated api
javanna May 20, 2016
6490355
make host state immutable
javanna May 25, 2016
29b1f8d
make some classes and methods package private
javanna May 27, 2016
7f4807b
add some javadocs
javanna May 27, 2016
044a97c
move client sniffer to its own project
javanna May 27, 2016
3745305
[TEST] be more specific around http method used for sniffing
javanna May 30, 2016
51e487f
[TEST] remove okhttp test dependency
javanna May 30, 2016
83c6e73
add support for PATCH and TRACE methods
javanna May 31, 2016
c9db111
add javadocs on closing responses
javanna Jun 1, 2016
6d66fbd
add toString to DeadHostState class
javanna Jun 1, 2016
35dbdee
check hosts is not null nor empty earlier, remove check from nextHost
javanna Jun 1, 2016
47e5204
[TEST] add setHosts test and rename RestClientBuilderTests to RestCli…
javanna Jun 2, 2016
24ea585
don't use setDefaultHeaders from HttpClient
javanna Jun 3, 2016
9ed2d61
[TEST] rename restClientTests back to RestClientBuilderTests
javanna Jun 3, 2016
4572b69
[TEST] add RestClient unit tests
javanna Jun 3, 2016
13a27a3
[TEST] add RestClient integration test
javanna Jun 3, 2016
4fa824f
[TEST] add a lot of forgotten try with resources to wrap Elasticsearc…
javanna Jun 3, 2016
f5825b8
[TEST] restore needed LogManager.reset to prevent logging to sysout
javanna Jun 3, 2016
23a94bb
[TEST] create standard RestClient at first request and reuse it
javanna Jun 3, 2016
c48b7a7
[TEST] create standard RestClient at first request and reuse it
javanna Jun 3, 2016
29eee32
[TEST] expand RequestLoggerTests to all the supported http methods
javanna Jun 3, 2016
f17f0f9
rename ElasticsearchResponse#getFirstHeader to getHeader
javanna Jun 3, 2016
b891c46
[TEST] remove status matcher and hasStatus assertion
javanna Jun 3, 2016
b15279b
Allow to pass socket facttry registry to createDefaultHttpClient method
javanna Jun 3, 2016
f2318ed
Build: add missing licenses, SHAs and enable dependency licenses check
javanna Jun 3, 2016
56e689e
[TEST] remove unused method
javanna Jun 3, 2016
05e26a4
raise default socket timeout to 10s and default connect timeout to 1s
javanna Jun 3, 2016
a461dd8
Build: add hamcrest and securemock to version.properties
javanna Jun 6, 2016
467dfbd
use TimeUnit and long to keep track of time
javanna Jun 6, 2016
1f7f6e2
wrap log statement with logger.isDebugEnabled
javanna Jun 6, 2016
2cf04c0
wrap entity only when not repeatable and improve RequestLoggerTests
javanna Jun 6, 2016
791db1f
remove TODO around using /_cat/nodes rather than /_nodes, test compat…
javanna Jun 8, 2016
a78fe13
require RestClient when creating Sniffer.Builder
javanna Jun 8, 2016
04d620d
require hosts when creating RestClient.Builder
javanna Jun 8, 2016
be5e2e1
Decouple HostsSniffer from Sniffer
javanna Jun 9, 2016
437c4f2
rename ElasticsearchResponse to Response and ElasticsearchResponseExc…
javanna Jun 9, 2016
85606e8
renamed all time intervals arguments and members to include the time …
javanna Jun 9, 2016
c028bf9
rename deadUntil to deadUntilNanos
javanna Jun 9, 2016
742f9c6
nextHost to return Iterable<HttpHost> rather than Iterator
javanna Jun 9, 2016
853ea93
add comments on retries
javanna Jun 9, 2016
cbdffc7
s/elasticsearchResponse/response
javanna Jun 9, 2016
de8b9fd
use switch for status codes and add comments
javanna Jun 9, 2016
9a4971d
fix line length
javanna Jun 9, 2016
d8c0fad
fix failing tests
javanna Jun 9, 2016
3474a14
fix line length
javanna Jun 9, 2016
33bdab1
Build: temporarily disable dependency licenses check (till our own ge…
javanna Jun 9, 2016
bd77335
[TEST] add SuppresForbidden annotation for client project
javanna Jun 9, 2016
cf6e713
Merge branch 'master' into feature/http_client
javanna Jun 9, 2016
f38ce72
make forbiddenApisTest work for client and client-sniffer
javanna Jun 9, 2016
2856ab0
Build: enable dependency licenses check for client-sniffer, exclude o…
javanna Jun 9, 2016
0af9d2c
Build: add animalsniffer to detect usage of java8 apis in client and …
javanna Jun 10, 2016
a5e329e
[TEST] use animalsniffer annotation @IgnoreJRERequirement (similar to…
javanna Jun 10, 2016
9cbfa98
Merge branch 'master' into feature/http_client
javanna Jun 10, 2016
432efc7
catch Exception rather than Throwable
javanna Jun 10, 2016
6db90da
[TEST] Remove usage of @SuppressForbidden due to sun HttpServer usage…
javanna Jun 10, 2016
3d7186c
make DeadHostState final
javanna Jun 10, 2016
656422c
move shutdownNow within try block
javanna Jun 10, 2016
777d438
[TEST] use jdk-internal bundled signature (rather than the previously…
javanna Jun 10, 2016
50b6f4c
Build: changed forbidden-apis targetCompatibility to 1.7 for client a…
javanna Jun 13, 2016
3cd201e
[TEST] add comment on using animal-sniffer suppress annotation
javanna Jun 13, 2016
8f7b7fb
added comments to clarify RequestLogger and DeadHostState
javanna Jun 13, 2016
116805b
remove TODO around copying hosts when rotating the collection, it's n…
javanna Jun 13, 2016
caa6c96
Build: make client and client-sniffer depend on lucene-test version 5…
javanna Jun 14, 2016
1932f6b
Rename RequestLogger#log methods to distinguish between the two
javanna Jun 14, 2016
cf93e90
remove message parameter from RequestLogger methods
javanna Jun 14, 2016
70fb67c
Build: targetCompatibility and sourceCompatibility are enough to make…
javanna Jun 14, 2016
7c8013f
Build: remove explicit targetCompatibility from forbiddenapis config
javanna Jun 15, 2016
ace3a7b
Merge branch 'master' into feature/http_client
javanna Jun 15, 2016
8c60374
Build: do not load integ test class if --skip-integ-tests-in-disguise…
javanna Jun 17, 2016
af93533
Merge branch 'master' into feature/http_client
javanna Jun 17, 2016
cb4bfcb
Take SniffOnFailureListener out of Sniffer and make FailureListener f…
javanna Jun 21, 2016
886cb37
Merge branch 'master' into feature/http_client
javanna Jun 21, 2016
f0b6abe
rename onSuccess to onResponse
javanna Jun 21, 2016
490d9c8
Merge branch 'master' into feature/http_client
javanna Jun 22, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -19,15 +19,13 @@
package org.elasticsearch.gradle.plugin

import nebula.plugin.publishing.maven.MavenBasePublishPlugin
import nebula.plugin.publishing.maven.MavenManifestPlugin
import nebula.plugin.publishing.maven.MavenScmPlugin
import org.elasticsearch.gradle.BuildPlugin
import org.elasticsearch.gradle.test.RestIntegTestTask
import org.elasticsearch.gradle.test.RunTask
import org.gradle.api.Project
import org.gradle.api.tasks.SourceSet
import org.gradle.api.tasks.bundling.Zip

/**
* Encapsulates build configuration for an Elasticsearch plugin.
*/
Expand Down Expand Up @@ -56,7 +54,7 @@ public class PluginBuildPlugin extends BuildPlugin {
}

project.namingConventions {
// Plugins decalare extensions of ESIntegTestCase as "Tests" instead of IT.
// Plugins declare integration tests as "Tests" instead of IT.
skipIntegTestInDisguise = true
}
}
Expand Down
Expand Up @@ -26,7 +26,6 @@ import org.gradle.api.file.FileCollection
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.InputFiles
import org.gradle.api.tasks.OutputFile

/**
* Runs NamingConventionsCheck on a classpath/directory combo to verify that
* tests are named according to our conventions so they'll be picked up by
Expand Down Expand Up @@ -90,9 +89,11 @@ public class NamingConventionsTask extends LoggedExec {
doFirst {
args('-Djna.nosys=true')
args('-cp', (classpath + extraClasspath).asPath, 'org.elasticsearch.test.NamingConventionsCheck')
args(testClass, integTestClass)
args('--test-class', testClass)
if (skipIntegTestInDisguise) {
args('--skip-integ-tests-in-disguise')
} else {
args('--integ-test-class', integTestClass)
}
/*
* The test framework has classes that fail the checks to validate that the checks fail properly.
Expand Down
Expand Up @@ -44,31 +44,36 @@
*/
public class NamingConventionsCheck {
public static void main(String[] args) throws IOException {
int i = 0;
NamingConventionsCheck check = new NamingConventionsCheck(
loadClassWithoutInitializing(args[i++]),
loadClassWithoutInitializing(args[i++]));
Class<?> testClass = null;
Class<?> integTestClass = null;
Path rootPath = null;
boolean skipIntegTestsInDisguise = false;
boolean selfTest = false;
while (true) {
switch (args[i]) {
case "--skip-integ-tests-in-disguise":
skipIntegTestsInDisguise = true;
i++;
continue;
case "--self-test":
selfTest = true;
i++;
continue;
case "--":
i++;
break;
default:
fail("Expected -- before a path.");
for (int i = 0; i < args.length; i++) {
String arg = args[i];
switch (arg) {
case "--test-class":
testClass = loadClassWithoutInitializing(args[++i]);
break;
case "--integ-test-class":
integTestClass = loadClassWithoutInitializing(args[++i]);
break;
case "--skip-integ-tests-in-disguise":
skipIntegTestsInDisguise = true;
break;
case "--self-test":
selfTest = true;
break;
case "--":
rootPath = Paths.get(args[++i]);
break;
default:
fail("unsupported argument '" + arg + "'");
}
break;
}
check.check(Paths.get(args[i]));

NamingConventionsCheck check = new NamingConventionsCheck(testClass, integTestClass);
check.check(rootPath, skipIntegTestsInDisguise);

if (selfTest) {
assertViolation("WrongName", check.missingSuffix);
Expand All @@ -87,9 +92,9 @@ public static void main(String[] args) throws IOException {
assertNoViolations("Found inner classes that are tests, which are excluded from the test runner", check.innerClasses);
assertNoViolations("Pure Unit-Test found must subclass [" + check.testClass.getSimpleName() + "]", check.pureUnitTest);
assertNoViolations("Classes ending with [Tests] must subclass [" + check.testClass.getSimpleName() + "]", check.notImplementing);
if (!skipIntegTestsInDisguise) {
assertNoViolations("Subclasses of ESIntegTestCase should end with IT as they are integration tests",
check.integTestsInDisguise);
if (skipIntegTestsInDisguise == false) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add the integTestClass != null here too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can't happen to have it null when skipIntegTestsInDisguise is false no?

assertNoViolations("Subclasses of " + check.integTestClass.getSimpleName() +
" should end with IT as they are integration tests", check.integTestsInDisguise);
}
}

Expand All @@ -108,7 +113,7 @@ public NamingConventionsCheck(Class<?> testClass, Class<?> integTestClass) {
this.integTestClass = integTestClass;
}

public void check(Path rootPath) throws IOException {
public void check(Path rootPath, boolean skipTestsInDisguised) throws IOException {
Files.walkFileTree(rootPath, new FileVisitor<Path>() {
/**
* The package name of the directory we are currently visiting. Kept as a string rather than something fancy because we load
Expand Down Expand Up @@ -143,7 +148,7 @@ public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IO
String className = filename.substring(0, filename.length() - ".class".length());
Class<?> clazz = loadClassWithoutInitializing(packageName + className);
if (clazz.getName().endsWith("Tests")) {
if (integTestClass.isAssignableFrom(clazz)) {
if (skipTestsInDisguised == false && integTestClass.isAssignableFrom(clazz)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd just check if integTestClass != null rather than make a new boolean.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I disagree, I didn't add a new boolean but just made work the existing one, no need to load the class if those checks are disabled.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with merging as is and hacking on it later. I don't like that the boolean which means "I don't care if integ tests sneak into the regular test phase" also means "there aren't integ tests so don't bother looking" but there are worse problems in the world.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure... I see it differently though. we simply don't need to load the class if we don't do any check. We maybe could simply rename the option to "skipIntegTestCheck" or something. let's postpone this though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++

integTestsInDisguise.add(clazz);
}
if (Modifier.isAbstract(clazz.getModifiers()) || Modifier.isInterface(clazz.getModifiers())) {
Expand Down
7 changes: 5 additions & 2 deletions client-sniffer/build.gradle
Expand Up @@ -61,8 +61,11 @@ forbiddenApisTest {

//JarHell is part of es core, which we don't want to pull in
jarHell.enabled=false
//NamingConventionCheck is part of test-framework, which we don't want to pull in as it depends on es core
namingConventions.enabled=false

namingConventions {
//we don't have integration tests
skipIntegTestInDisguise = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer settings integTestClass to null or defaulting it to null and setting it our stuff somewhere else in the build.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried defaulting it to null, but ended up having to set it in too many projects. Seems like we should rather disable in the client exception only? I went for setting it to null here explicitly but those null values become trickier to handle and understand, I went for the boolean that we already had cause it's more explicit and considered it a bug that when the check is disabled we still load the class.

}

dependencyLicenses {
dependencies = project.configurations.runtime.fileCollection {
Expand Down
7 changes: 5 additions & 2 deletions client/build.gradle
Expand Up @@ -59,8 +59,11 @@ forbiddenApisTest {

//JarHell is part of es core, which we don't want to pull in
jarHell.enabled=false
//NamingConventionCheck is part of test-framework, which we don't want to pull in as it depends on es core
namingConventions.enabled=false

namingConventions {
//we don't have integration tests
skipIntegTestInDisguise = true
}

thirdPartyAudit.excludes = [
//commons-logging optional dependencies
Expand Down