Skip to content

SOLR-17433: Set default http request timeout to infinite#4626

Open
VishnuPriyaChandraSekar wants to merge 1 commit into
apache:mainfrom
VishnuPriyaChandraSekar:SOLR-1743-changed-default-request-timeout
Open

SOLR-17433: Set default http request timeout to infinite#4626
VishnuPriyaChandraSekar wants to merge 1 commit into
apache:mainfrom
VishnuPriyaChandraSekar:SOLR-1743-changed-default-request-timeout

Conversation

@VishnuPriyaChandraSekar

Copy link
Copy Markdown
Contributor

Description

In earlier versions of SolrJ 9.4, the legacy HttpSolrClient was able to support long running streaming operation (> 10 mins). However, in SolrJ 9.4+, the http client throws exception exactly after 10 mins.

https://issues.apache.org/jira/browse/SOLR-17433

Solution

In case of long streaming operation, the server sends the response in chunks and request timeout determines how long the client can wait to receive all the chunks. The default request timeout was set to 10 mins which caused the streaming operations to timeout abruptly.

In order to support long running streams, the request timeout must be infinite (as we don't know how long streams will take to complete). Jetty excepts either 0 or -1 however JDK expects no value to represent the infinite timeout. The following changes were made to implement the behavior

  • HttpSolrClient: Set the default request timeout to infinite by setting it to zero. This helps streams to continue streaming until they exhaust the content.
  • HttpJdkSolrClient: Prevented passing over the request timeout to JDK's HttpClient if it is zero.

Tests

  • Updated ConcurrentUpdateSolrClientTestBase.testSocketTimeoutOnCommit to explicitly set a request timeout. The test previously relied on the default 1 ms request timeout (from idle timeout) to trigger an HTTP client timeout. After the default timeout was changed to infinite, the client no longer timed out, causing the test's 10-second timeout to fail first. This change restores the intended test behavior by overriding the request timeout explicitly.
  • Updated HttpJdkSolrClientTest.testTimeout to explicitly set a request timeout. Previously, request timeout was based on idle timeout. After the infinite request timeout, the client no longer timed out. Fixed the test by overriding the request timeout explicitly
  • Updated LB2SolrClientTest.testTimeoutExceptionMarksServerAsZombie to explicitly set a request timeout as the default value causes the client to wait infinitely.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • [X ] I have added tests for my changes.
  • I have added documentation for the Reference Guide
  • I have added a changelog entry for my change

* Set the default request timeout to infinite by setting it to zero. This helps streams to continue streaming until they exhaust the content.
* Prevent passing over the request timeout to JDK's HttpClient if the request timeout is zero. This is because JDK expects no value to represent infinite request timeout.
* Updated ConcurrentUpdateSolrClientTestBase.testSocketTimeoutOnCommit to explicitly set a request timeout. The test previously relied on the default 1 ms request timeout (from idle timeout)
to trigger an HTTP client timeout. After the default timeout was changed to infinite, the client no longer timed out, causing the test's 10-second timeout to fail first.
This change restores the intended test behavior by overriding the request timeout explicitly.
* Updated HttpJdkSolrClientTest.testTimeout to explicitly set a request timeout. Previously, request timeout was based on idle timeout. After the infinite request timeout, the client no longer timed out.
Fixed the test by overriding the request timeout explicitly
* Updated LB2SolrClientTest.testTimeoutExceptionMarksServerAsZombie to explicitly set a request timeout as the default request timeout causes the client to wait infinitely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant