Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] pdf timeout in bad internet environment #48

Closed
zhongnansu opened this issue May 13, 2021 · 5 comments
Closed

[BUG] pdf timeout in bad internet environment #48

zhongnansu opened this issue May 13, 2021 · 5 comments
Assignees
Labels
bug Something isn't working discuss need discussion SA untriaged
Milestone

Comments

@zhongnansu
Copy link
Member

Describe the bug
pdf timeout in bad internet environment

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

@zhongnansu zhongnansu added bug Something isn't working Beta untriaged SA and removed Beta labels May 13, 2021
@zhongnansu zhongnansu added this to the may-bug-bash milestone May 14, 2021
@zhongnansu zhongnansu self-assigned this May 14, 2021
@zhongnansu
Copy link
Member Author

zhongnansu commented May 14, 2021

Some info:

  • we have set the timeout for Chriomium to be 1 min(default 30s)
  • Node.js default server timeout is 2 min
  • I tried to increase the Chromium timeout to be 2 min/100s/80s, and foud that it failed to return any response to the client browser. And sometimes it keeps retrying. (I tryied to trigger a timeout by changing the css id selector #dashBoardView to something else, so that puppeteer will never find it to do the screenshot)@joshuali925 @davidcui-amzn do you have any suggestion on how to increase the timeout safely?

@joshuali925
Copy link
Member

@zhongnansu I don't see why 2 min would behave differently from 1 min or 30s. Would it fail to return response if timeout is 1 min or 30s and the css id selector is something other than #dashBoardView?

@zhongnansu
Copy link
Member Author

zhongnansu commented May 20, 2021

@zhongnansu I don't see why 2 min would behave differently from 1 min or 30s. Would it fail to return response if timeout is 1 min or 30s and the css id selector is something other than #dashBoardView?

Normally when I change the css id selector, it should return an error response saying something like "can't find the selector, timeout after XXs"

  • when I set timeout to 1 min(current value), it returns "can't find selector, timeout after 60s"
  • when I set timeout to 2min, it returns "socket hang up", I think this is because of node.js server default value
  • When I set timeout to 100000ms, browser failed to get any response. And from the kibana error log, I found it keeps printing the timeout error log again and again, which either means the client is retrying, or the server is doing something. I am confused by this behavior

@zhongnansu zhongnansu added the discuss need discussion label May 20, 2021
@joshuali925
Copy link
Member

joshuali925 commented May 21, 2021

The 2 min socket hang up is likely from http on server side, I don't think we can override that https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/core/server/http/http_config.ts#L77

the 100000ms issue seems to be caused by some kind of auto-retry requests from client. I added a log when a request hits server API

server    log   [00:31:02.019] [info][opendistroReportsKibana][plugins] request from Client API
server    log   [00:31:02.641] [info][opendistroReportsKibana][plugins] domain enables security, use session cookie to access
server    log   [00:31:02.651] [info][opendistroReportsKibana][plugins] original queryUrl http://localhost:5601/app/dashboards#/view/7adfa750-4c81-11e8-b3d7-01146121b73d?_g=(filters:!(),refreshInterval:(pause:!f,value:900000),time:(from:'2021-05-20T00:31:01.467Z',to:'2021-05-21T00:31:01.467Z'))&_a=(description:'Analyze%20mock%20flight%20data%20for%20ES-Air,%20Logstash%20Airways,%20Kibana%20Airlines%20and%20JetBeats',filters:!(),fullScreenMode:!f,options:(hidePanelTitles:!f,useMargins:!t),query:(language:kuery,query:''),timeRestore:!t,title:'%5BFlights%5D%20Global%20Flight%20Dashboard',viewMode:view)
server    log   [00:31:06.305] [error][data][elasticsearch] [version_conflict_engine_exception]: [search-telemetry:search-telemetry]: version conflict, required seqNo [7521], primary term [39]. current document has seqNo [7522] and primary term [39]
server    log   [00:31:16.290] [error][data][elasticsearch] [version_conflict_engine_exception]: [search-telemetry:search-telemetry]: version conflict, required seqNo [7551], primary term [39]. current document has seqNo [7552] and primary term [39]
server    log   [00:31:23.837] [info][opendistroReportsKibana][plugins] page url http://localhost:5601/app/dashboards
server    log   [00:33:02.021] [info][opendistroReportsKibana][plugins] request from Client API
server    log   [00:33:05.433] [error][opendistroReportsKibana][plugins] Failed to generate report: TimeoutError: waiting for selector "#dashboaiewport" failed: timeout 100000ms exceeded

at 00:31:02.019 server received first request. Before reaching 100 seconds timeout, server received a second request from browser at 00:33:02.021. Then at 00:33:05.433 timeout happens but client did not receive response. Client only sees error if timeout happens before retry request

This is what it seems to be happening, need to look further to see where the 2 min retry requests come from


Actually I tested with context menu with fetch, so it's probably something on server side

@zhongnansu
Copy link
Member Author

zhongnansu commented May 24, 2021

I tried again with 100000ms, but this time I can get the error response, saying it failed due to a timeout(monitored from devtool). if it's above 100000ms, the "retry" thing happens. Need to figure out where the constraint comes from.

some other findings:
All previous tests are based on download from context menu with fetch API.
I tried another approach to download from saved reports, which doesn't triggered by the fetch API I think, and it works for timeout > 100s, no auto-retry detected.

kavilla pushed a commit to kavilla/dashboards-reports that referenced this issue Jul 12, 2023
Signed-off-by: Rupal Mahajan <maharup@amazon.com>
(cherry picked from commit 9ce6c2576432e33bf9e570d96eafe848d8ef705b)

Co-authored-by: Rupal Mahajan <maharup@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working discuss need discussion SA untriaged
Projects
None yet
Development

No branches or pull requests

2 participants