Fix for java sdk to force the use of suns https Handler

[pax95]

There are some problems using the sdk in application servers eg. Weblogic.

This pull request fixes 2 problems.

1. It forces the use of Suns Https handler.
2. It null checks values before setting the RequestProperty.

Comments :

1. imho the sdk should support Client certificates when constructing the Truststore. This is not included in the pull request since it would require api changes.
2. The sdk might declare better dependencies. I have to declare dependencies on

groupId -> org.codehaus.woodstox
artifactId -> stax2-api
version -> 3.1.1

groupId -> org.codehaus.woodstox
artifactId -> woodstox-core-asl
version -> 4.1.5

To be independent on jdk/app server library.

[madhadron]

Two issues from my end:

I can't use the com.sun.* libraries in the SDK, first because we support multiple JDKs, and second because they're deprecated and there's no guarantee they won't be changed to something else entirely (com.oracle.*?) in future releases. Is there some generic access mechanism to get the default handler for the platform that you could use instead?

For nulls in RequestProperty, can you give me a case where the show up so I can add it to the test suite, or at least track down the error in the code that's producing them? A null showing up there is not an acceptable value, and means that something's gone wrong.

[pax95]

Hi Fred

Yeah I understand your point regarding the com.sun....ssl.Handler. Since
the problem relates to running Splunk reader in a applicationserver
(Weblogic in my case). Application servers will in many cases provide there
own implementation of http-/https handlers, and ppl might get into
problems.
Many other oss projects have tackled this by providing dependencies that
they can control. eg. xmlstreaming asf. and not rely on the jdk.

As for the null handeling if get the following entries in the header map
->
HttpService line 293
{Authorization=null, Content-Type=application/x-www-form-urlencoded}

As you see Authorization is null and will fail on Weblogic implementation
of http handler.
It's hard to provide a unittest since this might only happen on Weblogic.

mandag den 20. maj 2013 skrev Fred Ross :

Two issues from my end:

I can't use the com.sun.* libraries in the SDK, first because we support
multiple JDKs, and second because they're deprecated and there's no
guarantee they won't be changed to something else entirely (com.oracle.*?)
in future releases. Is there some generic access mechanism to get the
default handler for the platform that you could use instead?

For nulls in RequestProperty, can you give me a case where the show up so
I can add it to the test suite, or at least track down the error in the
code that's producing them? A null showing up there is not an acceptable
value, and means that something's gone wrong.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/45#issuecomment-18168791
.

[madhadron]

On May 21, 2013, at 5:56 AM, Preben Asmussen notifications@github.com wrote:

Yeah I understand your point regarding the com.sun....ssl.Handler. Since
the problem relates to running Splunk reader in a applicationserver
(Weblogic in my case). Application servers will in many cases provide there
own implementation of http-/https handlers, and ppl might get into
problems.

Would providing a constructor where the user of the SDK can specify an SSL handler, as in

ServiceArgs serviceArgs = new ServiceArgs();
serviceArgs.setUsername("blah");
…
serviceArgs.setSSLHandler(new sun.net.www.protocol.https.Handler());

Service service = Service.connect(serviceArgs);

be a good fix?

As for the null handeling if get the following entries in the header map
->
HttpService line 293
{Authorization=null, Content-Type=application/x-www-form-urlencoded}

As you see Authorization is null and will fail on Weblogic implementation
of http handler.
It's hard to provide a unittest since this might only happen on Weblogic.

Aha. Okay, now I see where to fix it so you won't get those nulls. I'll open a bug for us to fix it here, and if you want to do it straight away in your copy, go to splunk/com/splunk/Service.java, look at the send method. Replace the line

request.getHeader().put("Authorization", token);

with

if (token != null) {
request.getHeader().put("Authorization", token);
}

We use null as the marker for "this service is not logged in yet."

Frederick Ross
http://madhadron.com
http://github.com/madhadron/

[pax95]

Hi Fred

Yeah - the proposed solutions seems fine. Will you provide a branch in your
gitrepo for me to test out the fixes ?

Besides that there are still some spooks regarding the xml streaming
classes when i run under Weblogic since it has it's own implementations of
STaX. When calling connect I get an ->
java.lang.RuntimeException: next() may not be called when XMLStreamReader
is at END_DOCUMENT

This might me due to some problems with the Weblogic stuff and I tried to
overcome this limitation with replacing the whole Weblogic stuff with ->
woodstox-core-asl version 4.1.2 that brings in StaX 2. This solves the
connect problem and i'm now able to connect and post events to Splunk.
I'm still not able to read though. The error I get is :

com.ctc.wstx.exc.WstxEOFException: Unexpected EOF; was expecting a close
tag for element
at [row,col {unknown-source}]: [1,27]
java.lang.RuntimeException: com.ctc.wstx.exc.WstxEOFException: Unexpected
EOF; was expecting a close tag for element
at [row,col {unknown-source}]: [1,27]
at
com.splunk.ResultsReaderXml.advanceStreamToNextSet(ResultsReaderXml.java:430)
at com.splunk.ResultsReader.getNextElement(ResultsReader.java:105)
at com.splunk.ResultsReader.getNextEvent(ResultsReader.java:66)
at
org.apache.camel.component.splunk.support.SplunkDataReader.extractData(SplunkDataReader.java:288)
at
org.apache.camel.component.splunk.support.SplunkDataReader.runQuery(SplunkDataReader.java:263)
at
org.apache.camel.component.splunk.support.SplunkDataReader.nonBlockingSearch(SplunkDataReader.java:228)
at
org.apache.camel.component.splunk.support.SplunkDataReader.read(SplunkDataReader.java:76)
at
org.apache.camel.component.splunk.SplunkConsumer.poll(SplunkConsumer.java:45)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:139)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:909)
at java.lang.Thread.run(Thread.java:662)

So for the moment I'm not able to read from Splunk.

An other way to bypass all xml streaming problems is to use JSON, but that
has it limitations in ResultsReader.getNextEvent() is blocking if there is
no events to read.
This will result in stuck threads and ugly thread dumps. I think you can
find this problem described in SplunkAnswers :-(

Best,
Preben

torsdag den 23. maj 2013 skrev Fred Ross :

On May 21, 2013, at 5:56 AM, Preben Asmussen notifications@github.com
wrote:

Yeah I understand your point regarding the com.sun....ssl.Handler. Since
the problem relates to running Splunk reader in a applicationserver
(Weblogic in my case). Application servers will in many cases provide
there
own implementation of http-/https handlers, and ppl might get into
problems.

Would providing a constructor where the user of the SDK can specify an SSL
handler, as in

ServiceArgs serviceArgs = new ServiceArgs();
serviceArgs.setUsername("blah");
…
serviceArgs.setSSLHandler(new sun.net.www.protocol.https.Handler());

Service service = Service.connect(serviceArgs);

be a good fix?

As for the null handeling if get the following entries in the header map
->
HttpService line 293
{Authorization=null, Content-Type=application/x-www-form-urlencoded}

As you see Authorization is null and will fail on Weblogic
implementation
of http handler.
It's hard to provide a unittest since this might only happen on
Weblogic.

Aha. Okay, now I see where to fix it so you won't get those nulls. I'll
open a bug for us to fix it here, and if you want to do it straight away in
your copy, go to splunk/com/splunk/Service.java, look at the send method.
Replace the line

request.getHeader().put("Authorization", token);

with

if (token != null) {
request.getHeader().put("Authorization", token);
}

We use null as the marker for "this service is not logged in yet."

Frederick Ross
http://madhadron.com
http://github.com/madhadron/

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/45#issuecomment-18372606
.

[pax95]

Fred

Any comments on this ?

2013/5/24 Preben Asmussen preben.asmussen@gmail.com

Hi Fred

Yeah - the proposed solutions seems fine. Will you provide a branch in
your gitrepo for me to test out the fixes ?

Besides that there are still some spooks regarding the xml streaming
classes when i run under Weblogic since it has it's own implementations of
STaX. When calling connect I get an ->
java.lang.RuntimeException: next() may not be called when XMLStreamReader
is at END_DOCUMENT

This might me due to some problems with the Weblogic stuff and I tried to
overcome this limitation with replacing the whole Weblogic stuff with ->
woodstox-core-asl version 4.1.2 that brings in StaX 2. This solves the
connect problem and i'm now able to connect and post events to Splunk.
I'm still not able to read though. The error I get is :

com.ctc.wstx.exc.WstxEOFException: Unexpected EOF; was expecting a close
tag for element
at [row,col {unknown-source}]: [1,27]
java.lang.RuntimeException: com.ctc.wstx.exc.WstxEOFException: Unexpected
EOF; was expecting a close tag for element
at [row,col {unknown-source}]: [1,27]
at
com.splunk.ResultsReaderXml.advanceStreamToNextSet(ResultsReaderXml.java:430)
at com.splunk.ResultsReader.getNextElement(ResultsReader.java:105)
at com.splunk.ResultsReader.getNextEvent(ResultsReader.java:66)
at
org.apache.camel.component.splunk.support.SplunkDataReader.extractData(SplunkDataReader.java:288)
at
org.apache.camel.component.splunk.support.SplunkDataReader.runQuery(SplunkDataReader.java:263)
at
org.apache.camel.component.splunk.support.SplunkDataReader.nonBlockingSearch(SplunkDataReader.java:228)
at
org.apache.camel.component.splunk.support.SplunkDataReader.read(SplunkDataReader.java:76)
at
org.apache.camel.component.splunk.SplunkConsumer.poll(SplunkConsumer.java:45)
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:139)
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:91)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:909)
at java.lang.Thread.run(Thread.java:662)

So for the moment I'm not able to read from Splunk.

An other way to bypass all xml streaming problems is to use JSON, but that
has it limitations in ResultsReader.getNextEvent() is blocking if there is
no events to read.
This will result in stuck threads and ugly thread dumps. I think you can
find this problem described in SplunkAnswers :-(

Best,
Preben

torsdag den 23. maj 2013 skrev Fred Ross :

On May 21, 2013, at 5:56 AM, Preben Asmussen notifications@github.com

wrote:

Yeah I understand your point regarding the com.sun....ssl.Handler.
Since
the problem relates to running Splunk reader in a applicationserver
(Weblogic in my case). Application servers will in many cases provide
there
own implementation of http-/https handlers, and ppl might get into
problems.

Would providing a constructor where the user of the SDK can specify an
SSL handler, as in

ServiceArgs serviceArgs = new ServiceArgs();
serviceArgs.setUsername("blah");
…
serviceArgs.setSSLHandler(new sun.net.www.protocol.https.Handler());

Service service = Service.connect(serviceArgs);

be a good fix?

As for the null handeling if get the following entries in the header
map
->
HttpService line 293
{Authorization=null, Content-Type=application/x-www-form-urlencoded}

As you see Authorization is null and will fail on Weblogic
implementation
of http handler.
It's hard to provide a unittest since this might only happen on
Weblogic.

Aha. Okay, now I see where to fix it so you won't get those nulls. I'll
open a bug for us to fix it here, and if you want to do it straight away in
your copy, go to splunk/com/splunk/Service.java, look at the send method.
Replace the line

request.getHeader().put("Authorization", token);

with

if (token != null) {
request.getHeader().put("Authorization", token);
}

We use null as the marker for "this service is not logged in yet."

Frederick Ross
http://madhadron.com
http://github.com/madhadron/

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/45#issuecomment-18372606
.

[madhadron]

Yes, sorry, I've been swamped. Planning on getting to this today.

[pax95]

Thats ok.
I just realized that the new version 5.0.3 had some bugfixes including this
->

• REST always returns a result when in previous versions 0 results were
  returned. (SPL-64751)

I'm just on it to give it a try. So forth it seems that the
ResultReaderJson is not blocking anymore when no results is present. So It
seems to work with JSON.

mandag den 3. juni 2013 skrev Fred Ross :

Yes, sorry, I've been swamped. Planning on getting to this today.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/45#issuecomment-18854843
.

[madhadron]

Excellent. I've also just pushed the fixes we talked about to the develop branch, so you can download and build from there to get them.