What is the maximum WCS export size

[msmitherdc]

for a 64 bit instance

I'm getting messages like

agg2CreateImage(): Memory allocation error. mapagg.cpp: 796: Out of memory allocating 4164540520 bytes.

when making large (5 sq mile) wcs requests (1.0.0) against a tile index with resolution of (0.1 m).

[7o9]

Same problem here, but with regular WMS request.

MapServer:
MapServer version 6.2.0 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG SUPPORTS=PROJ SUPPORTS=GD SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=ICONV SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_CLIENT SUPPORTS=THREADS SUPPORTS=GEOS INPUT=JPEG INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE

Debian GNU/Linux 6.0.7 (squeeze)
64 bit
16 GB RAM

<?xml version='1.0' encoding="ISO-8859-1" standalone="no" ?>
<ServiceExceptionReport version="1.3.0" xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.3.0/exceptions_1_3_0.xsd">
<ServiceException>
msDrawMap(): Image handling error. Unable to initialize image.
agg2CreateImage(): Memory allocation error. mapagg.cpp: 796: Out of memory allocating 3600000000 bytes.

</ServiceException>
</ServiceExceptionReport>

[tbonfort]

@7o9 what's the request URL ?

[7o9]

wget -O test.png "http://localhost/cgi-bin/mapserv-6.2?map=/home/data/ordnancesurvey/os.map&VERSION=1.3.0&REQUEST=GetMap&SERVICE=WMS&LAYERS=nationalparks&STYLES=&CRS=EPSG:27700&BBOX=138422.00557103055,38500,811577.9944289695,641500&WIDTH=30000&HEIGHT=30000&FORMAT=image/png&BGCOLOR=0xffffff&TRANSPARENT=TRUE&EXCEPTIONS=XML"

[7o9]

Works fine up about 23000 by 23000 pixels but we need up to ~70000 by 100000. Processing times is exceptionally fast, needs about 10 seconds for a map that needs 20 minutes rendering using ArcSomething... It is used for high resolution printing.

[tbonfort]

Is there anything else running on the server? i.e. do you have the 3.5 Gbytes of ram available for that allocation ?

[7o9]

Hehe, yes, lots a virgin RAM waiting to be ... Ah, well. Yes.

[tbonfort]

also note that your 70000*100000 requests are each going to require 28Gb of memory, which is clearly going to fail on a 16Gb server

[tbonfort]

@7o9 maybe check your ulimits ?

[7o9]

What is the formula for calculating the required RAM?

[tbonfort]

width_height_4 , result in bytes

[7o9]

Thanks. We can adjust resolution to fit the server / stock up on RAM. But this does not solve the original problem. The request only requires 3.6 GB as per your formula and there are definitely ~15GB available?!

[7o9]

After some playing around I think we have a hard limit of 2GB somewhere in MapServer. I can request up to 23k square images, anything more will break things. Makes sense?

[tbonfort]

yes, makes sense... that's what my "check your ulimits" idea was for

[7o9]

Ah, I misunderstood you then. I thought you suggested limits of image size... What is "ulimits" & where can I check / set them (outing myself as a noob).

[tbonfort]

I've never tinkered with them personally. check out https://wiki.debian.org/Limits , and more precisely
For example, apache2 limits are configured in /etc/apache2/envvars

[7o9]

...later, rechecked all:

MapServer is 64bit binary running on a 64 bit system with sufficient RAM.

The Apache processes are running as user www-data and:
www-data@ns3100632:~$ ulimit -v
returns "unlimited"

But we do get a segmentation fault:
mapserv-6.2[5824]: segfault at 7f5943f76008 ip 00007f5a7045fb39 sp 00007fffa54b4880 error 4 in libc-2.11.3.so[7f5a703e9000+159000]

So I still suspect that it is some hard-coded limit in MapServer or one of its libraries.

[7o9]

This issue is very easy to reproduce, simply call a WMS request with a width and height exceeding 25000 pixel. Always works, like in: always breaks.

[tbonfort]

@7o9, please check #4796 for a possible fix. cc @rouault

[jratike80]

Questions belong IMHO better to mapserver-users mailing list. Three years later I asked the same question because I do not follow Github issues so closely.
http://osgeo-org.1560.x6.nabble.com/What-resourse-blocks-any-bigger-GeoTIFF-output-from-WCS-td5218584.html

I suppose that this ticket does not deal with any exact bug so I am closing it.