Skip to content
This repository has been archived by the owner on Sep 1, 2022. It is now read-only.

THREDDS - Simple joinExisting failing in 4.2.9 #1

Closed
asascience opened this issue Nov 16, 2011 · 10 comments
Closed

THREDDS - Simple joinExisting failing in 4.2.9 #1

asascience opened this issue Nov 16, 2011 · 10 comments

Comments

@asascience
Copy link

A very simple joinExisting aggregation that was working with 4.2.x is not working with 4.2.9. May be related to scanning recursively for directories when using a scan element? I’ve tested on both Tomcat 6 and 7 using both JDK 6 and 7, the problem is never present in 4.2.6 but always present in 4.2.9.

I've restarted the Tomcat servers and cleared the cache directories multiple times.

Working:
Version 4.2.20110404.1849
Build Date = 2011-04-04 18:49:47
Build Name = 20110404.1849

Not Working:
Version 4.2.9
Build Date = 2011-11-08 17:58:27
Build Name = 9

Stack trace, catalog file, and directory listing are here: https://gist.github.com/1370929

Live TDS is here: http://tds.maracoos.org/thredds/MODIS.html

@JohnLCaron
Copy link
Collaborator

hi asa:

is there anything funny like symbolic linking in /data/modis ??

john

@asascience
Copy link
Author

nothing of the such:

https://gist.github.com/1370929#file_directory+listing

@JohnLCaron
Copy link
Collaborator

On 11/16/2011 2:09 PM, Applied Science Associates wrote:

nothing of the such:

https://gist.github.com/1370929#file_directory+listing


Reply to this email directly or view it on GitHub:
#1 (comment)

hi kyle:

did you get another message from me on this? i thought i sent on friday.

anyway, ive made a diagnostic version of tds to track this problem down:

ftp://ftp.unidata.ucar.edu/pub/thredds/4.2/4.2.asaTest/

can you run and reproduce the problem and send me the output it should
generate from catalina.out?

thanks,john

@asascience
Copy link
Author

Tested out the test WAR but I'm not getting any additional information in catalina.out or threddsServlet.log (just the same error message as before)

The new WAR is live here: http://tds.maracoos.org:8080 (port is important... :80 runs a different Tomcat)

@asascience
Copy link
Author

I removed the lost+found directory in the root of the scan directory and it worked.

@asascience
Copy link
Author

Steps to reproduce problem:

]$ mkdir /data/modis/lost+found
]$ chown root:root /data/modis/lost+found
]$ chmod 700 /data/mode/lost+found

Restart Tomcat.

Access OPeNDAP HTML page. It fails.

While tomcat is still running;

]$ rm -rf /data/modis/lost+found

Refresh OPeNDAP HTML page, it works!

@JohnLCaron
Copy link
Collaborator

still cant reproduce.

Are you on linux?

Can you send me the stack trace from the new server. Are you sure no extra stuff is in catalina.out?

thanks

@JohnLCaron
Copy link
Collaborator

On 11/21/2011 3:31 PM, Applied Science Associates wrote:

Steps to reproduce problem:

]$ mkdir /data/modis/lost+found
]$ chown root:root /data/modis/lost+found
]$ chmod 700 /data/mode/lost+found

does this make the directory not readable from tomcat?
or are you running as root?

@asascience
Copy link
Author

Does this help?

http://www.youtube.com/watch?v=7cNOva2aDGI

I tried to include everything I could think of.

DennisHeimbigner added a commit that referenced this issue Apr 30, 2012
tkunicki added a commit to tkunicki/thredds that referenced this issue Aug 2, 2012
removed calls to abort() with commons httpclient HTTPMethodBase instances
@kwilcox
Copy link

kwilcox commented Oct 3, 2012

This works with the latest 4.3 dev branch, OK to close!

DennisHeimbigner added a commit that referenced this issue Nov 5, 2015
The problem was that java.net.URI and java.net.URL
could not parse url strings that were based on
backslash escape rather than %xx escape.
The General Rule is this:
1. any url given to e.g. HTTPSession or HTTPMethod
   or HTTPFactory is assumed to be the url
   that is wanted on the server side.
2. #1 means that, again as a general rule, urlstrings
   given to those classes should not be %xx escaped
   unless that is what you want on the server side.
3. The key to understand is that
   a. those classes will internally %xx encode the url string they are
      given even if the url string already is %xx encoded.
   b. on the server side, the %xx encoding will be performed
      on the incoming url string. This means that the %xx encoded
      string you originally sent is the one given to the servlet.
Bottom line: do not do %xx encoding yourself unless your server side
code is expected to see a %xx encoded url.

Anyway, the fix is to create a procedure -- HTTPUtil.parseToURI() --
that can properly create a java.net.URI object from a backslash
escaped url string. Any time you need to create a URI object,
you should do so using that procedure.
DennisHeimbigner added a commit that referenced this issue Feb 27, 2016
1. Move all cdm dependent code into dap4/d4cdm
2. Modify various gradle files to support #1.
3. Misc. other mods to support #1.
lesserwhirls pushed a commit that referenced this issue Sep 22, 2016
Merge pull request #617 from msdsoftware/master
lesserwhirls pushed a commit that referenced this issue Mar 20, 2019
Get latest Unidata commits
hvandam2 pushed a commit that referenced this issue May 2, 2019
setting up a merge with unidata/thredds
hvandam2 pushed a commit that referenced this issue May 2, 2019
Merge pull request #1 from Unidata/5.0.0
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants