New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Connection resets and "500 Internal Server error" #11
Comments
@deiruch, Inside logs I fount next two types of errors. it is strange, but files is exist on the server and I have no problems to open it. first error. The error only with this file. After some research I find out that this file is just regular jp2 image and not are video. Is It suppose to be like this?:
Second error. Many different files. I look into files and everything looks fine. Is it possible that something with esajpip server?:
|
I've committed a change to log the reason for failure. Could you please install it? |
Works again nicely. Thanks for the quick reaction!! |
@bogdanni, just installed new update for jpip. |
This is occurring again. All my requests fail with a "500 Internal Server Error". Repro: Download the current beta release of JHelioviewer (3.0.5) and start it. http://www.jhelioviewer.org/download_archive.html) |
Hopefully now there is more info in the logs. |
restarted service again. I found this type of errors:
@bogdanni, I sent you full log as well. |
@ddialer, are you sure the commit with extra logging was checked out, compiled and installed? I expected a fourth error in the sequence file_manager -> index_manager -> client_manager. @deiruch, in our experience, the requested region (size+offset) should be clipped to be within the requested resolution size, otherwise it leads to those errors: "A new data chunk could not be generated". |
@bogdanni Do you have an example of an invalid request? JHV only generates requests where fsiz=rsiz. If I read the spec correctly, this should always work - regardless of the values used, because rsiz is specified in a coordinate system of size fsiz. |
@bogdanni, sorry, buy mistake compiled from old source. I just tried to compile new source and I got this error: Helioviewer-Project/JHelioviewer-SWHV#1 |
Recompiled from new source. After closer examination of:
Looks like file api.helioviewer.org/jp2/movies/SDO_AIA_AIA_193_F2016-03-13T20.00.34Z_T2016-03-13T23.00.34ZB60L.jpx is just regular JP2 image and not are video. Is it suppose to be like this or that can be are problem? |
Most likely can be a problem, the server uses file extension to determine the way it parses the file. |
@deiruch, I'm just saying that if the client makes requests where rsiz+roff are outside of fsiz, the esajpip server may issue errors like that, no matter what the standard says. SWHV JHV has code to clip, e.g. |
@bogdanni This is the relevant piece of code from JHV:
As far as I understand, rsiz+roff are impossible to lie outside fsiz. Or did I not understand your suggestion? Perhaps an example of an offending request would be helpful. |
Simon, that looks fine. Could be an older client (v2) that led to those errors. |
This problem is still occurring. Is there anything in the logs? What you get back from the server is this:
I think it'd be useful if the server returned some more details about what went wrong to diagnose the issue. Note: I used HTTP instead of JPIP to make clicking in a browser easier |
@deiruch, we are just switched to the new API v3 and server was rebooted. I just finished linking all the directories and now everything should work normal. |
This broke users of the older API like SWHV (http://swhv.oma.be/download/) or classical JHV (2.2). |
Confirmed: Right now, the classical JHV (2.2), which is still used by many people, unfortunately does not work anymore. It would be great if this could be fixed, at least for a transition period. |
I just tested SWHV from the link you provided and its seems working fine. |
@ddialer, with SWHV you have to try to load data from the GSFC server in the dialog box for new layer. |
@ddialer, with JHV 2.2, if you are on a Mac, you have to set in System Preferences -> Security & Privacy -> General -> Allow apps downloaded from: Anywhere. The app is not signed. |
The file size of JHV 2.2.1 is indeed small (3.6 MB) as all libraries are loaded at start-up rather than being included in the executable.
Could it be that you are on OS X and your security settings are not set to ‘Allow apps downloaded from anywhere’? In 2012, when JHV 2.2.1 was released, we did not sign the application with a Mac Developer Key yet, which necessitates this change of security settings. In the default setting, OS X indeed complains the the application is ‘damaged’ - which is, however, not the case. Hope that helps! Daniel
|
I can confirm Bogdan's comment: Using the about-to-be-released version of JHV (http://swhv.oma.be/download/, http://www.jhelioviewer.org), users cannot load data from the GSFC server! :-( |
Should be fixed. Problem was on apache redirect rule to the api v1. |
@ddialer : Thanks! Unfortunately, I still get a "connection reset" error on both JHV 2.2.1 and 2.10. The entries of the Observatory/Instrument/Dector fields, however, are visible now, so the problem might have moved to a different place (JPIP server?). |
This happens with JHV 3.0.5 as well. It's at the JPIP server level. |
Restarted JPIP server. Server was overloaded with connections. @bogdanni, there is new type of errors I found in the log
|
I am running JHV 2.10.0 Rev 6818 using GSFC with no issues. Thanks! On Wed, Mar 30, 2016 at 12:23 PM, Serge Zahniy notifications@github.com
|
@ddialer, some JP2 files made with IDL were not transcoded when ingested into the HV server. |
@wafels, I'm not sure what files is effected. but there is two movie files that giving that errors: The problem I have right now is I getting wall of this errors and they all open new connection and never closing the previous until server is overloaded. |
@wafels, It's a bit difficult to use since the IDL JP2 files have the XML with a terminating NULL which is formally forbidden by the XML standard (but not truly a problem for the use in HV/JHV). I will try to come up with a recipe for installation. Note also |
@ddialer : Thanks a lot! I have checked JHV 2.10.0, 3.0.5 and 2.2.1, and all work like a charm with the GSFC server now. |
@ddialer : your server is down again. Could you please check out the latest revision from 'esajpip-SWHV', I may have solved the problem. I believe sockets were leaked, our server was also experiencing some errors from a client program apparently connected from an ESA IP address... |
@bogdanni Done. JPIP server restarted and updated |
The server again is refusing all JPIP connections at the moment... |
@deiruch, I restarted jpip server. Inside logs, I found this error. Before the error, everything is looking fine
|
Briefly looking through the code, I'm not sure the logic is correct on the error path. |
@ddialer Today GSFC esajpip server returns 500 for several current SDO datasets. Maybe the transcode stage failed? |
@bogdanni, I restarted the jpip server. Transcode seems to work fine. But a few days ago I received some other strange errors. I'm not sure if this errors caused by jpip or our server.
|
That's new. |
Maybe something like the following: |
both addresses return "??:0" |
I still receive 500s here. Are there new errors? Maybe try to run it inside gdb? |
I just get back to logs and there is two type of errors:
And new one:
|
I fixed the missing file error. And restarted JPIP again. May it will resolve the socket error |
Still there. Anything changed in the environment, like the OS etc. ? |
Funny that SWAP works. I still suspect transcoding problems, should get a message about PLT missing. |
Strange. I don't see any errors in the logs. No, nothing changed. We only rebooted server this monday. Also I sent email to you with full JPIP log for last two hours |
Hold on. maybe you are right. seems there is problem with transcoding |
Yes, at least some files are not transcoded: 2016-10-21 09:40:22,511: /tmp/esajpip_0.3/esajpip/esajpip/src/jpeg2000/file_manager.cc:204: ERROR: The code-stream does not include any PLT marker 2016-10-21 10:24:20,758: /tmp/esajpip_0.3/esajpip/esajpip/src/jpeg2000/file_manager.cc:204: ERROR: The code-stream does not include any PLT marker 2016-10-21 10:24:32,016: /tmp/esajpip_0.3/esajpip/esajpip/src/jpeg2000/file_manager.cc:204: ERROR: The code-stream does not include any PLT marker |
I'm reinserting some data. may take couple hours to finish. There was an transcode issue |
|
@bogdanni is it just SDO data need to be transcoded or other sources as well? |
|
https://github.com/Helioviewer-Project/hvJP2K
In some screen(1): |
SOHO LASCO also. |
I reinserted all SDO and SOHO data that was broken. should work now |
Closing since last comment indicates this was fixed. |
The API currently often replies with connection resets and "500 Internal Server Error"s instead of actual data. The same requests succeeded yesterday.
Unfortunately the server still no provides details when an internal server error occurs.
Am I doing anything wrong? Can I diagnose the problem? Can you check the logs?
The text was updated successfully, but these errors were encountered: