Preventing DOS situation with WMS-T service specifying effective huge TIME range #4524

Closed
akrherz opened this Issue Nov 13, 2012 · 8 comments

Projects

None yet

3 participants

@akrherz
Contributor
akrherz commented Nov 13, 2012

I'm running the WMS-T service shown as the example in the MS documentation here :)

http://mapserver.org/ogc/wms_time.html#wms-time

The problem is that when a user specifies a CGI variable of 'TIME=2011', the database query ends up returning every raster for the year 2011 (a raster every 5 minutes). This causes mapserver to do a lot of work and eventually DOSs the server...

The documentation notes that setting the metadata wms_timeformat should help limit the potential timestamp queries coming to the server, but this feature may not be implemented for PostGIS backed WMS-T queries?

Note that this functionality is only available on layers based on Shapefiles
and OGR.

tbonfort kindly helped me with this on IRC and asked if I might have a suggestion on how to deal with this. I am not sure if the issue here is if the postgis backend is not supporting the wms_timeformat setting and implementing that would resolve this or if some syntax to the SQL query is needed to limit the number of rows that could be potentially returned. In my mind, the wms_timeformat would be best and I would want to make sure a request has at least the minutes specified in the TIME CGI variable.

thanks

@akrherz
Contributor
akrherz commented Nov 13, 2012

I got 6.2.0-rc running and debugged the resulting SQL from just a &TIME=2010

select "datetime",encode(ST_AsBinary(ST_Force_2D("the_geom"),'NDR'),'hex') as geom,"oid" 
from nexrad_n0r_tindex where the_geom && 
ST_GeomFromText(<snip>,find_srid('','nexrad_n0r_tindex','the_geom')) and 
(datetime >= date_trunc('year',date '2010-01-01') and 
datetime < date_trunc('year',date '2010-01-01') + interval '1 year') 
and (datetime >= date_trunc('year',date '2010-01-01') and 
datetime < date_trunc('year',date '2010-01-01') + interval '1 year')

interesting. Still digging :)

@akrherz
Contributor
akrherz commented Nov 13, 2012

I think the problem is that msSetLimitedPattersToUse() is never getting called, because the ms_limited_pattern is not getting set early enough. The only place it gets set is within msSetLimitedPattersToUse() . I think this circular logic is the problem , but my head is spinning at the moment.

@akrherz
Contributor
akrherz commented Feb 7, 2013

Here's a test case.

ms4524.sql

create table ms4524(
   datetime timestamptz,
   filepath varchar
);
ALTER TABLE ms4524 SET WITH oids;
SELECT AddGeometryColumn('ms4524', 'geom', 4326, 'MULTIPOLYGON', 2);
GRANT SELECT on ms4524 to apache;

INSERT into ms4524(datetime, filepath, geom) VALUES 
  ('2013-01-01 00:00+00', '/tmp/bogus.png', 
  ST_GeomFromText('MULTIPOLYGON(((-126 50,-66 50,-66 24,-126 24,-126 50)))',
    4326)); 
INSERT into ms4524(datetime, filepath, geom) VALUES 
  ('2013-01-01 00:05+00', '/tmp/bogus.png', 
  ST_GeomFromText('MULTIPOLYGON(((-126 50,-66 50,-66 24,-126 24,-126 50)))',
    4326)); 
INSERT into ms4524(datetime, filepath, geom) VALUES 
  ('2013-01-01 00:10+00', '/tmp/bogus.png', 
  ST_GeomFromText('MULTIPOLYGON(((-126 50,-66 50,-66 24,-126 24,-126 50)))',
    4326)); 

ms4524.map

MAP
#
# Start of map file
#
NAME "nexrad_base_reflect"
STATUS ON
SIZE 800 600
EXTENT -171 15 -66 70
UNITS DD
IMAGECOLOR -1 -1 -1

PROJECTION
   "init=epsg:4326"
END

WEB
  MINSCALE 90000
  MAXSCALE 4650000
  IMAGEPATH "/tmp/"
  IMAGEURL "/tmp/"
  LOG "/tmp/ms4524.log"
  METADATA
   "wms_title" "IEM WMS Service"
   "wms_onlineresource" "http://mesonet.agron.iastate.edu/cgi-bin/wms/nexrad/n0q-t.cgi?"
   "wms_srs" "EPSG:4326 EPSG:900913 EPSG:102100 EPSG:3857"
   "wms_abstract" "IEM generated CONUS composite of NWS WSR-88D level III base reflectivity."
   "wms_keywords" "NEXRAD,N0Q"
   "wms_accessconstraints" "None"
   "wms_contactperson" "Daryl Herzmann"
   "wms_contactorganization" "Iowa State University"
   "wms_enable_request" "*"
  END
END

#  Tile Index
LAYER
  STATUS ON
  NAME "time_idx"
  TYPE POLYGON
  DATA "geom from ms4524"
  FILTER "datetime = '2006-06-23 03:10+00'"
  METADATA
    "wms_title" "NEXRAD BASE REFLECT"
    "wms_srs"   "EPSG:4326 EPSG:900913 EPSG:102100 EPSG:3857"
    "wms_extent" "-171 15 -66 70"
    "wms_timeextent" "2011-02-16/2013-12-31/PT5M"
    "wms_timeitem" "datetime"
    "wms_timedefault" "2011-02-21T19:30:00Z"
    "wms_timeformat" "YYYY-MM-DDTHH:MM:SSZ"
  END
  CONNECTION "dbname=postgis"
  CONNECTIONTYPE postgis
END

# raster layer
LAYER
  NAME "nexrad-n0q-wmst"
  TYPE RASTER
  STATUS ON
  DEBUG ON
  DUMP TRUE
  PROJECTION
    "init=epsg:4326"
  END
  METADATA
    "wms_title" "NEXRAD BASE REFLECT"
    "wms_srs"   "EPSG:4326 EPSG:900913 EPSG:102100 EPSG:3857"
    "wms_extent" "-171 15 -66 70"
    "wms_timeextent" "2011-02-16/2013-12-31/PT5M"
    "wms_timeitem" "datetime"
    "wms_timedefault" "2011-02-21T19:30:00Z"
    "wms_timeformat" "YYYY-MM-DDTHH:MM:SSZ"
  END
  OFFSITE 0 0 0
  TILEITEM "filepath"
  TILEINDEX "time_idx"
END

END

Example request:

http://localhost/cgi-bin/mapserv/mapserv?map=/tmp/ms4524.map&LAYERS=nexrad-n0q-wmst&TRANSPARENT=TRUE&FORMAT=image%2Fpng&TIME=2013&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&SRS=EPSG%3A4326&BBOX=-67.5,0,-45,22.5&WIDTH=256&HEIGHT=256

resulting sql

select "datetime","filepath",encode(ST_AsBinary(ST_Force_2D("geom"),'NDR'),'hex') as geom,"oid" from ms4524 where geom && ST_GeomFromText('POLYGON((-67.4560546875 0.0439453125,-67.4560546875 22.4560546875,-45.0439453125 22.4560546875,-45.0439453125 0.0439453125,-67.4560546875 0.0439453125))',find_srid('','ms4524','geom')) and (datetime >= date_trunc('year',date '2013-01-01') and datetime < date_trunc('year',date '2013-01-01') + interval '1 year')

thanks

@aboudreault aboudreault pushed a commit that closed this issue Feb 8, 2013
Alan Boudreault Fix WMS Time functionnality, so preventing DOS situation of big range…
…. (closes #4524)
3bed2f4
@mkofahl mkofahl pushed a commit to faegi/mapserver that referenced this issue Apr 9, 2013
Alan Boudreault Fix WMS Time functionnality, so preventing DOS situation of big range…
…. (closes #4524)
fb5bdc6
@aboudreault aboudreault was assigned Apr 13, 2013
@akrherz
Contributor
akrherz commented Apr 14, 2014

With mapserver 6.4.1, it appears this request is able to bring about the DOS type situation as well

TIME=2006-12-31T17:36:00Z/2008-12-31T03:12:00Z

@aboudreault
Member

@dmorissette you might want to take a look at this.

@akrherz
Contributor
akrherz commented Jun 15, 2015

Any updates on this issue? I am testing out current master and see the DOS condition is still there :(

@tbonfort
Member

I still believe there is nothing much we can do on the mapserver side, as from what I understand its behavior in this case is correct. The logic of ordering and limiting the number of features from the tileindex is very specific to each scenario, and can't really be hardcoded into mapserver. I suggest you either use MAXFEATURES on your tileindex layer, or use a LIMIT+ORDER BY clause in your sql query.

@akrherz
Contributor
akrherz commented Jun 17, 2015

@tbonfort Thank you for your response. For your suggestions, I could not make MAXFEATURES work and I found no method to inject a LIMIT clause into the DATA setting without breaking the layer entirely. Could you kindly suggest a DATA setting that you had in mind?

Perhaps a PROCESSING or METADATA option could be added to disable this functionality entirely? With 6.4.1, I simply made this change

--- mappostgis.c-orig   2015-06-17 07:59:44.793284497 -0500
+++ mappostgis.c    2015-06-15 14:28:40.717011966 -0500
@@ -3258,7 +3258,8 @@
         strlcat(buffer, bufferTmp, buffer_size);
       } else if(numranges == 2) {
         /* we have a range */
-        createPostgresTimeCompareRange(timefield, aranges[0], aranges[1], bufferTmp, buffer_size);
+        createPostgresTimeCompareSimple(timefield, atimes[i], bufferTmp, buffer_size);
+        //createPostgresTimeCompareRange(timefield, aranges[0], aranges[1], bufferTmp, buffer_size);
         strlcat(buffer, bufferTmp, buffer_size);
       } else {
         return MS_FALSE;

I see the code has changed for 7.0.0, so will try to manually hack it out of the code again.

EDIT: I got MAXFEATURES to work, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment