Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

THREDDS - Simple joinExisting failing in 4.2.9 #1

Closed
asascience opened this Issue Nov 16, 2011 · 10 comments

Comments

Projects
None yet
3 participants
@ghost

ghost commented Nov 16, 2011

A very simple joinExisting aggregation that was working with 4.2.x is not working with 4.2.9. May be related to scanning recursively for directories when using a scan element? I’ve tested on both Tomcat 6 and 7 using both JDK 6 and 7, the problem is never present in 4.2.6 but always present in 4.2.9.

I've restarted the Tomcat servers and cleared the cache directories multiple times.

Working:
Version 4.2.20110404.1849
Build Date = 2011-04-04 18:49:47
Build Name = 20110404.1849

Not Working:
Version 4.2.9
Build Date = 2011-11-08 17:58:27
Build Name = 9

Stack trace, catalog file, and directory listing are here: https://gist.github.com/1370929

Live TDS is here: http://tds.maracoos.org/thredds/MODIS.html

Collaborator

JohnLCaron commented Nov 16, 2011

hi asa:

is there anything funny like symbolic linking in /data/modis ??

john

Collaborator

JohnLCaron commented Nov 21, 2011

On 11/16/2011 2:09 PM, Applied Science Associates wrote:

nothing of the such:

https://gist.github.com/1370929#file_directory+listing


Reply to this email directly or view it on GitHub:
#1 (comment)

hi kyle:

did you get another message from me on this? i thought i sent on friday.

anyway, ive made a diagnostic version of tds to track this problem down:

ftp://ftp.unidata.ucar.edu/pub/thredds/4.2/4.2.asaTest/

can you run and reproduce the problem and send me the output it should
generate from catalina.out?

thanks,john

@ghost

ghost commented Nov 21, 2011

Tested out the test WAR but I'm not getting any additional information in catalina.out or threddsServlet.log (just the same error message as before)

The new WAR is live here: http://tds.maracoos.org:8080 (port is important... :80 runs a different Tomcat)

@ghost

ghost commented Nov 21, 2011

I removed the lost+found directory in the root of the scan directory and it worked.

@ghost

ghost commented Nov 21, 2011

Steps to reproduce problem:

]$ mkdir /data/modis/lost+found
]$ chown root:root /data/modis/lost+found
]$ chmod 700 /data/mode/lost+found

Restart Tomcat.

Access OPeNDAP HTML page. It fails.

While tomcat is still running;

]$ rm -rf /data/modis/lost+found

Refresh OPeNDAP HTML page, it works!

Collaborator

JohnLCaron commented Nov 21, 2011

still cant reproduce.

Are you on linux?

Can you send me the stack trace from the new server. Are you sure no extra stuff is in catalina.out?

thanks

Collaborator

JohnLCaron commented Nov 21, 2011

On 11/21/2011 3:31 PM, Applied Science Associates wrote:

Steps to reproduce problem:

]$ mkdir /data/modis/lost+found
]$ chown root:root /data/modis/lost+found
]$ chmod 700 /data/mode/lost+found

does this make the directory not readable from tomcat?
or are you running as root?

@ghost

ghost commented Nov 29, 2011

Does this help?

http://www.youtube.com/watch?v=7cNOva2aDGI

I tried to include everything I could think of.

DennisHeimbigner added a commit that referenced this issue Apr 30, 2012

tkunicki added a commit to tkunicki/thredds that referenced this issue Aug 2, 2012

Merge pull request #1 from tkunicki-usgs/v4.3.11-CIDA
removed calls to abort() with commons httpclient HTTPMethodBase instances

kwilcox commented Oct 3, 2012

This works with the latest 4.3 dev branch, OK to close!

@JohnLCaron JohnLCaron closed this May 20, 2014

DennisHeimbigner added a commit that referenced this issue Nov 5, 2015

Fix for Github issue #272
The problem was that java.net.URI and java.net.URL
could not parse url strings that were based on
backslash escape rather than %xx escape.
The General Rule is this:
1. any url given to e.g. HTTPSession or HTTPMethod
   or HTTPFactory is assumed to be the url
   that is wanted on the server side.
2. #1 means that, again as a general rule, urlstrings
   given to those classes should not be %xx escaped
   unless that is what you want on the server side.
3. The key to understand is that
   a. those classes will internally %xx encode the url string they are
      given even if the url string already is %xx encoded.
   b. on the server side, the %xx encoding will be performed
      on the incoming url string. This means that the %xx encoded
      string you originally sent is the one given to the servlet.
Bottom line: do not do %xx encoding yourself unless your server side
code is expected to see a %xx encoded url.

Anyway, the fix is to create a procedure -- HTTPUtil.parseToURI() --
that can properly create a java.net.URI object from a backslash
escaped url string. Any time you need to create a URI object,
you should do so using that procedure.

DennisHeimbigner added a commit that referenced this issue Feb 27, 2016

Part 1 of the dap4 refactor.
1. Move all cdm dependent code into dap4/d4cdm
2. Modify various gradle files to support #1.
3. Misc. other mods to support #1.

lesserwhirls pushed a commit that referenced this issue Sep 22, 2016

Merge pull request #1 from Unidata/master
Merge pull request #617 from msdsoftware/master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment