mapfishapp - datadir #1038

pmauduit · 2015-08-31T16:06:57Z

This PR proposes the datadirization of the mapfishapp webapp:

Exports the contexts, addons, JS configuration (including GEOR_custom.js) into the datadir
Makes the rest of the webapp configurable outside of the webapp WAR (using the same mechanisms as previously)
Allows mapfishapp to be easily customized by server-side addons (it allows automatic scanning of jars which classes begining with org.georchestra.mapfishapp.addons.*, as well as a applicationContext.xml to be defined. See https://github.com/pmauduit/osm2geor/tree/master/java as an example of use-case)

As the other PR, it requires the introduction of the georchestra-commons module. Travis-ci will fail until #1035 is merged.

Note for the reviewers: A lot of files are related to default addons inclusion, the majority of files added to src/deb/resources/etc are not necessary to be reviewed IMHO.

TODO:

Test compilation
testsuite
runtime test with georchestra.datadir
runtime test without georchestra.datadir

pmauduit · 2015-09-01T10:03:13Z

Regression detected without the georchestra.datadir option: Images for the addons are broken.

pmauduit · 2015-09-01T12:44:34Z

Ok fixed ; everything seems to go well now (without the georchestra.datadir variable)

- Exports the contexts, addons, JS configuration (including GEOR_custom.js) into the datadir - Makes the rest of the webapp configurable outside of the webapp WAR (using the same mechanisms as previously) - Allows mapfishapp to be easily customized by server-side addons (it allows automatic scanning of jars which classes begin with org.georchestra.mapfishapp.addons) As the other PR, it requires the introduction of the georchestra-module.

Fixing path to the addons in the webapp when no georchestra.datadir is provided.

pmauduit · 2015-09-07T18:08:11Z

(FYI I pushed --force after rebasing / integration of the georchestra-commons module)

fvanderbiest · 2015-09-08T07:18:15Z

mapfishapp/pom.xml

+              <packageDescription>Debian package for the geOrchestra's Viewer</packageDescription>
+              <projectOrganization>geOrchestra</projectOrganization>
+              <maintainerName>Pierre Mauduit</maintainerName>
+              <maintainerEmail>pierre.mauduit@gmail.com</maintainerEmail>


maintainer = PSC / psc@georchestra.org

As said, I fear that I had to regenerate a GPG key so that I can publish the signed packages on build.georchestra.org ; it will deserve a try in my georchestra-datadir branch, and see if the CI is not broken by the change.

Anyway, I've no objection for such a change

Let's see: https://build.georchestra.org/ci/job/georchestra-generic-wars/55/ / pmauduit@6f372fc

Sync GEOR_custom.js with the one provided by the template configuration.

s/config//

pmauduit · 2015-09-08T14:48:59Z

I will do extra tests by merging back the branch into my own one.

pmauduit · 2015-09-08T15:56:44Z

Runtime tests: Ok (basic tests, mainly on the addons management, with and without georchestra.datadir).

landryb · 2015-09-14T07:49:09Z

I'm testing this, with georchestra.datadir set and some customizations done in /etc/georchestra/mapfishapp/js/GEOR_custom.js and mapfishapp/w?s.servers.json, but it seems the values returned by the w?s.servers.json webservice come from the json files in the webapp, and GEOR_custom.js is fetched from the webapp dir too. Interestingly, i see no addon at all listed in 'tools' window, even if some are listed in the default GEOR_custom.js found/used within the webapp. Guess i'll need to dig a bit more....

the mapfishapp/log4j/log4j.properties and mapfishapp/mapfishapp.properties files are properly read/applied from the datadir.

/mapfishapp/addons/ and /mapfishapp/contexts/ yield 404's..

maybe relevant info from mapfishapp.log :

2015-09-14 09:08:23 BaseMapServlet [INFO] app is null, setting it as default configPath: WEB-INF/print/config.yaml
2015-09-14 09:08:23 BaseMapServlet [INFO] app is null, setting it as default configPath: WEB-INF/print/config.yaml
2015-09-14 09:08:23 BaseMapServlet [INFO] Attempting to locate app config file: 'WEB-INF/print/config.yaml in the webapplication.
2015-09-14 09:08:23 BaseMapServlet [INFO] Attempting to locate app config file: 'WEB-INF/print/config.yaml in the webapplication.
2015-09-14 09:08:23 BaseMapServlet [INFO] Unable to find config file in web application using getRealPath.  Adding a / because that is often dropped
2015-09-14 09:08:23 BaseMapServlet [INFO] Unable to find config file in web application using getRealPath.  Adding a / because that is often dropped
2015-09-14 09:08:23 BaseMapServlet [INFO] Loading app from: /data/tomcat/georchestra/webapps/mapfishapp/WEB-INF/print/config.yaml
2015-09-14 09:08:23 BaseMapServlet [INFO] Loading app from: /data/tomcat/georchestra/webapps/mapfishapp/WEB-INF/print/config.yaml
2015-09-14 09:08:23 BaseMapServlet [INFO] Loading configuration file: /data/tomcat/georchestra/webapps/mapfishapp/WEB-INF/print/config.yaml
2015-09-14 09:08:23 BaseMapServlet [INFO] Loading configuration file: /data/tomcat/georchestra/webapps/mapfishapp/WEB-INF/print/config.yaml

landryb · 2015-09-14T08:06:04Z

Hm, i was mistaken, at mapfishapp loading /ws/app/js/GEOR_custom.js is called and it effectively returns the content of the file found in the datadir.

landryb · 2015-09-14T08:11:34Z

/ws/addons/ returns

{"success":false, "error":"fileupload_error_ioError", "msg": "null"}

java.lang.NullPointerException at org.georchestra.mapfishapp.ws.AddonController.buildAddonSpecs(AddonController.java:188)

/ws/contexts/ returns the default geobretagne context, which seems expected, although it isnt loaded at initial mapfishapp display. It seems nothing calls /ws/contexts/ nor /ws/addons/ at startup....

landryb · 2015-09-14T08:14:04Z

As for the w?s servers, by default mapfishapp/wms.servers.json is loaded/queried (which returns the default list from the webapp) while i think it should be mapfishapp/ws/wms.servers.json (which contains my custom list..)

pmauduit · 2015-09-14T09:52:27Z

TODO:

Remove contexts and addons from the provided GEOR_custom.js
hardenning contexts / addons to avoid NPE in case of non existing directory in the datadir

landryb · 2015-09-14T12:44:55Z

For the w?s server list, i think GEOR_LayerBrowser.js just lacks /ws/ being prepended to GEOR.config.OGC_SERVERS_URL[this.id] in line 59, otherwise the list comes from the webapp.

Tested this, works..

--- a/mapfishapp/src/main/webapp/app/js/GEOR_LayerBrowser.js
+++ b/mapfishapp/src/main/webapp/app/js/GEOR_LayerBrowser.js
@@ -56,7 +56,7 @@ GEOR.LayerBrowser = Ext.extend(Ext.Panel, {
         this.serverStore = new Ext.data.Store({
             proxy: new Ext.data.HttpProxy({
                 url: GEOR.util.getValidURI(
-                    GEOR.config.OGC_SERVERS_URL[this.id]
+                    "ws/" + GEOR.config.OGC_SERVERS_URL[this.id]
                 ),
                 method: 'GET',

landryb · 2015-09-14T13:19:07Z

Maybe only local to my setup, but there are some encoding issues in various places:

in the addons list, their description (coming from webapp/app/addons//.json) is double-encoded, yielding ?? instead of accentued letters. This is not the case with default contexts where the accents coming from datadir/contexts/*.wmc are properly rendered - and the addons description is fine in http://sdi.georchestra.org/mapfishapp/?lang=fr
saving a context also double-encodes accentued letters (looking bad in mapfishapp db too, although the db is UTF8), badly displayed when re-sending the wmc. Tested the same, works on sdi.georchestra.org.

landryb · 2015-09-14T13:33:11Z

More details on the encodings:

if i browse to /mapfishapp/ws/contexts/ in firefox, an accentued in displayed as a single ? but they're properly displayed in the wmc browser. curl -i on the url says that content-type is app/js;charset=UTF-8
if i browse to /mapfishapp/ws/addons/ in firefox, an accentued letter is displayed as two ??, and they're also displayed this way (ie badly) in the addons browser. curl -i also says that content-type is app/js;charset=UTF-8
if i browse to /mapfishapp/ws/wmts.servers.json in firefox, an accentued letter is displayed as utf vomit (ie GÃ©oPortail) and is displayed 'correctly' in the server list dropdown menu. curl -i doesnt specify a content-type, and the accent is properly rendered in curl output in my term

pmauduit · 2015-09-14T14:21:13Z

if i browse to /mapfishapp/ws/contexts/ in firefox, an accentued in displayed as a single ? but they're properly displayed in the wmc browser. curl -i on the url says that content-type is app/js;charset=UTF-8

Have you set the URIencoding option onto your tomcat connectors ?

<Connector URIEncoding="UTF-8"  ...>

landryb · 2015-09-15T12:32:18Z

I had URIEncoding="UTF-8" set on all the tomcat connectors, but the issue was somewhere else. Setting LANG=C.UTF-8 (or any of the other UTF-8 locales available, check locale -a output) in /etc/default/tomcat-* worked around the issue. Guess that's what i get for not having it set in /etc/default/locale......

landryb · 2015-09-15T15:04:25Z

Small buglet in addons handling.. added openls,magnifier,cadastrapp,streetview & annotation in /etc/georchestra/mapfishapp/addons, loaded /ws/addons -> extractorapp was in the list (and enabled) although not present in the datadir - i think this was caused by cadastrapp not having a config.json file (oversight from me) - i had the error message, but somehow some addon was randomly taken from the webapp..

2015-09-15 16:50:54 ws [ERROR] Addon cadastrapp does not have a config.json configuration file, skipping it.

landryb · 2015-09-15T15:22:13Z

okay, after more explanation from @pmauduit it makes sense to have extractor in the list of app since it's in the webapp, and enabled by default. To disable it, one just needs to make a copy of it in the datadir, and set enable to false in its config.json.

landryb · 2015-09-16T07:07:45Z

And with the current logic in https://github.com/georchestra/georchestra/blob/mapfishapp-datadir/mapfishapp/src/main/java/org/georchestra/mapfishapp/ws/AddonController.java#L200 (and also the two loops over addons arrays in constructAddonsSpec()) it doesnt seem possible to 'disable' an addon that is enabled in the webapp. If i copy extractor to my datadir and change enabled to false in config.json, it will still be listed in the available & enabled addons.

Do not check too early if the addon is activated or not, check it at the very end of the process. This allows to deactivate addons provided by default in mapfishapp, simply by copying it into the datadir and explicitely setting it as disabled in the config.json spec.

- Hardenning tests when trying to find files on the FS - Aslo allows to set images from different formats (JPEG) for context capions. Tests: utests added for NPE avoidance runtime tests for the context captions

Now using the controller systematically, irregardless if georchestra.datadir is configured or not. Getting the relevant file in both cases (like it is done for other controllers).

pmauduit · 2015-09-18T15:39:28Z

Done today:

Removing remaining comments in the standard mapfishapp addons
Fixing bug with disabled addons in the overriden georchestra.datadir path
Rework to harden context / addon controller a little bit (avoids NPE)
fixing bug with the WxS access: same access for both configuration (with & without datadir)

landryb · 2015-09-21T13:47:20Z

Retested tip of the PR, all the previous issues i had are now fixed, i've been able to disable extractor addon and the wxs server lists are fetched from the datadir. Thanks!

fvanderbiest · 2015-10-13T07:38:47Z

Good, please merge.

mapfishapp - datadir

pmauduit force-pushed the mapfishapp-datadir branch from e89b0fa to c526173 Compare September 1, 2015 09:30

pmauduit added 2 commits September 7, 2015 19:57

mapfishapp datadir - fixing path

cc26a6c

Fixing path to the addons in the webapp when no georchestra.datadir is provided.

pmauduit force-pushed the mapfishapp-datadir branch from 40b6213 to cc26a6c Compare September 7, 2015 17:57

fvanderbiest reviewed Sep 8, 2015
View reviewed changes

pmauduit added 3 commits September 8, 2015 16:19

mapfishapp datadir - taking into account comments

3d35f35

Sync GEOR_custom.js with the one provided by the template configuration.

mapfishapp datadir - taking remarks into account (WxS)

14b84a4

s/config//

mapfishapp datadir - Sync config.yaml

0f98436

pmauduit added 5 commits September 18, 2015 12:07

mfapp addons - removing comments in json files

ee1bc3d

mfapp datadir - contexts & addons controller rework

b156143

- Hardenning tests when trying to find files on the FS - Aslo allows to set images from different formats (JPEG) for context capions. Tests: utests added for NPE avoidance runtime tests for the context captions

mapfishapp datadir - repairing testsuite

fad9f74

mfapp datadir - fixing WxS access

97a1243

Now using the controller systematically, irregardless if georchestra.datadir is configured or not. Getting the relevant file in both cases (like it is done for other controllers).

pmauduit added a commit that referenced this pull request Oct 13, 2015

Merge pull request #1038 from georchestra/mapfishapp-datadir

f6c57ed

mapfishapp - datadir

pmauduit merged commit f6c57ed into master Oct 13, 2015

fvanderbiest deleted the mapfishapp-datadir branch October 13, 2015 07:49

fvanderbiest mentioned this pull request Oct 31, 2015

generic WARs #94

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mapfishapp - datadir #1038

mapfishapp - datadir #1038

pmauduit commented Aug 31, 2015

pmauduit commented Sep 1, 2015

pmauduit commented Sep 1, 2015

pmauduit commented Sep 7, 2015

fvanderbiest Sep 8, 2015

pmauduit Sep 8, 2015

pmauduit Sep 8, 2015

pmauduit commented Sep 8, 2015

pmauduit commented Sep 8, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

pmauduit commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

pmauduit commented Sep 14, 2015

landryb commented Sep 15, 2015

landryb commented Sep 15, 2015

landryb commented Sep 15, 2015

landryb commented Sep 16, 2015

pmauduit commented Sep 18, 2015

landryb commented Sep 21, 2015

fvanderbiest commented Oct 13, 2015

Navigation Menu

mapfishapp - datadir #1038

mapfishapp - datadir #1038

Conversation

pmauduit commented Aug 31, 2015

pmauduit commented Sep 1, 2015

pmauduit commented Sep 1, 2015

pmauduit commented Sep 7, 2015

fvanderbiest Sep 8, 2015

Choose a reason for hiding this comment

pmauduit Sep 8, 2015

Choose a reason for hiding this comment

pmauduit Sep 8, 2015

Choose a reason for hiding this comment

pmauduit commented Sep 8, 2015

pmauduit commented Sep 8, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

pmauduit commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

landryb commented Sep 14, 2015

pmauduit commented Sep 14, 2015

landryb commented Sep 15, 2015

landryb commented Sep 15, 2015

landryb commented Sep 15, 2015

landryb commented Sep 16, 2015

pmauduit commented Sep 18, 2015

landryb commented Sep 21, 2015

fvanderbiest commented Oct 13, 2015