crash with many threads in java #4231

Closed
mapserver-bot opened this Issue Apr 4, 2012 · 31 comments

Projects

None yet

4 participants

@mapserver-bot

Reporter: nbrachet
Date: 2012/03/09 - 02:09
Trac URL: http://trac.osgeo.org/mapserver/ticket/4231

'''How to reproduce:'''

Modify

`````` MapThread.javato remove the call toSystem.gc()```
. For this problem to reproduce one thread has to delete the mapObj object, while simultaneously another one deletes a layerObj object (from the same mapObj object), which happens when garbage collection runs.

Run with many threads for a while... because this is a threading issue it can work for a while without problem. In our testing we would crash within a half hour on a 8-core machine running 20 threads with JRE 1.6.

'''Analysis:'''

Reference counting isn't thread-safe.

layerObj objects obtained from a call to mapObj.get(int) are really shared between the layerObj and the mapObj objects. When garbage collection runs it may actually delete the layerObj object simultaneously from 2 different threads: one deleting the layerObj, the other one deleting the mapObj.
freeLayer()
calls
MS_REFCNT_DECR_IS_NOT_ZERO()
, itself implemented in terms of
MS_REFCNT_DECR()
which is a simple macro around operator++... not thread-safe.
In some (rare) case both threads think the ref counter went to zero because of their own call to
MS_REFCNT_DECR
and they end up both calling free() causing the crash.

'''Solution:'''

A simple solution is to use atomic int operations,
__sync_fetch_and_add
/
__sync_sub_and_fetch
for GCC,
InterlockedIncrement
/
InterlockedDecrement
on WIN32, etc...
Attached is a patch against 6.0.2 that implements this change for GCC.

@mapserver-bot

Author: unicoletti
Date: 2012/03/09 - 16:33
Thanks for the patch, I'll look into it asap.
I'm wondering what options are you passing to the vm? Perhaps ''UseParallelGC'' or something like that?

@mapserver-bot

Author: unicoletti
Date: 2012/03/09 - 16:37
Changing the component as Mapscript because this change potentially impacts other mapscript implementations as well.
cc'ing tamas maybe he can come up with a fix for c# too.

@mapserver-bot

Author: nbrachet
Date: 2012/03/09 - 16:41
Replying to [comment:3 unicoletti]:

Thanks for the patch, I'll look into it asap.
I'm wondering what options are you passing to the vm? Perhaps ''UseParallelGC'' or something like that?

Not specifically but it first happened with the -server option (which turns UseParallelGC on right?). I also reproduced on a default (-client no UseParallelGC) JVM.

I am running 64-bit JVM. Not that it should matter.

@mapserver-bot

Author: unicoletti
Date: 2012/03/09 - 16:47
Replying to [comment:5 nbrachet]:

Replying to [comment:3 unicoletti]:

Thanks for the patch, I'll look into it asap.
I'm wondering what options are you passing to the vm? Perhaps ''UseParallelGC'' or something like that?

Not specifically but it first happened with the -server option (which turns UseParallelGC on right?). I also reproduced on a default (-client no UseParallelGC) JVM.

I think that starting from 1.6 -server does just that.
Patch looks good to me.

I am running 64-bit JVM. Not that it should matter.

@mapserver-bot

Author: rouault
Date: 2012/03/09 - 21:57
You could borrow http://trac.osgeo.org/gdal/browser/trunk/gdal/port/cpl_atomic_ops.cpp from GDAL that has implementation of atomic incrementation for various platforms/compilers

@unicolet unicolet was assigned Apr 5, 2012
@unicolet
Contributor

@nbrachet if you could you open a pull request with the proposed patch I will pull it in in time for inclusion in 6.2.
As for other platforms they will have to wait a little more, as I am currently busy with a gazillion things.

@unicolet unicolet added a commit that referenced this issue May 5, 2012
@unicolet unicolet fix concurrency issue in maintaining ref counter, at least for GCC. O…
…ther platforms still vulnerable. #4231
3a0be39
@unicolet
Contributor
unicolet commented May 5, 2012

I have applied the patch for gcc for the time being.

@unicolet unicolet added a commit that referenced this issue May 5, 2012
@unicolet unicolet fix concurrency issue in maintaining ref counter for Windows platform…
…s. Thanks @rouault for providing examples #4231
fcccb57
@jmckenna
Member
jmckenna commented May 7, 2012

@unicolet I am following this issue now. Let me know here when you have an update (as reported on the -dev mailing list).

@jmckenna
Member
jmckenna commented May 8, 2012

@unicolet I've cloned your 'windows_ref_count_atomic' repoository. I get many many errors when I try to compile that on windows, such as:

C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : error C2733: second
C linkage of overloaded function 'operator &' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : see declarat
ion of 'operator &'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : warning C4190: '|' h
as C-linkage specified, but returns UDT 'Iu16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(444) : see declarat
ion of 'Iu16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : error C2733: second
C linkage of overloaded function 'operator |' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : see declarat
ion of 'operator |'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : warning C4190: '^' h
as C-linkage specified, but returns UDT 'Iu16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(444) : see declarat
ion of 'Iu16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : error C2733: second
C linkage of overloaded function 'operator ^' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : see declarat
ion of 'operator ^'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : warning C4190: 'andn
ot' has C-linkage specified, but returns UDT 'Iu16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(444) : see declarat
ion of 'Iu16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : error C2733: second
C linkage of overloaded function 'andnot' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(722) : see declarat
ion of 'andnot'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : warning C4190: '&' h
as C-linkage specified, but returns UDT 'Is16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(355) : see declarat
ion of 'Is16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : error C2733: second
C linkage of overloaded function 'operator &' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : see declarat
ion of 'operator &'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : warning C4190: '|' h
as C-linkage specified, but returns UDT 'Is16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(355) : see declarat
ion of 'Is16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : error C2733: second
C linkage of overloaded function 'operator |' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : see declarat
ion of 'operator |'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : warning C4190: '^' h
as C-linkage specified, but returns UDT 'Is16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(355) : see declarat
ion of 'Is16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : error C2733: second
C linkage of overloaded function 'operator ^' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : see declarat
ion of 'operator ^'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : warning C4190: 'andn
ot' has C-linkage specified, but returns UDT 'Is16vec4' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(355) : see declarat
ion of 'Is16vec4'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : error C2733: second
C linkage of overloaded function 'andnot' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(723) : see declarat
ion of 'andnot'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : warning C4190: '&' h
as C-linkage specified, but returns UDT 'I32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(133) : see declarat
ion of 'I32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : error C2733: second
C linkage of overloaded function 'operator &' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : see declarat
ion of 'operator &'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : warning C4190: '|' h
as C-linkage specified, but returns UDT 'I32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(133) : see declarat
ion of 'I32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : error C2733: second
C linkage of overloaded function 'operator |' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : see declarat
ion of 'operator |'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : warning C4190: '^' h
as C-linkage specified, but returns UDT 'I32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(133) : see declarat
ion of 'I32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : error C2733: second
C linkage of overloaded function 'operator ^' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : see declarat
ion of 'operator ^'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : warning C4190: 'andn
ot' has C-linkage specified, but returns UDT 'I32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(133) : see declarat
ion of 'I32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : error C2733: second
C linkage of overloaded function 'andnot' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(724) : see declarat
ion of 'andnot'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : warning C4190: '&' h
as C-linkage specified, but returns UDT 'Iu32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(246) : see declarat
ion of 'Iu32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : error C2733: second
C linkage of overloaded function 'operator &' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : see declarat
ion of 'operator &'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : warning C4190: '|' h
as C-linkage specified, but returns UDT 'Iu32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(246) : see declarat
ion of 'Iu32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : error C2733: second
C linkage of overloaded function 'operator |' not allowed
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : see declarat
ion of 'operator |'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : warning C4190: '^' h
as C-linkage specified, but returns UDT 'Iu32vec2' which is incompatible with C
        C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(246) : see declarat
ion of 'Iu32vec2'
C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\ivec.h(725) : error C2733: second
C linkage of overloaded function 'operator ^' not allowed
@jmckenna
Member
jmckenna commented May 8, 2012

@unicolet latest pull compiles on Windows with no issues. Java/CSharp/PHP/Python mapscripts also compile.

@unicolet
Contributor
unicolet commented May 8, 2012

Instructions for testing the fix:

build mapserver and java mapscript remember to use --with-threads options when configuring:

 configure --with-threads [your options here]
 make
 cd mapscript/java 
 make

On windows you probably have to use nmake instead of make.
Now from the java mapscript directory run the threadtests target included in the Makfile:

 make threadtests

if the program runs to its completion w/o errors or crashing then it is considered to be successful.

@jmckenna
Member
jmckenna commented May 8, 2012
  • On Windows, the makefile used is /mapscript/java/makefile.vc
  • I then executed the command: nmake /f Makefile.vc threadtests
  • results are below:
Microsoft (R) Program Maintenance Utility Version 9.00.30729.01
Copyright (C) Microsoft Corporation.  All rights reserved.

        "C:\Progra~1\Java\jdk1.6.0_13"\bin\javac -cp ./mapscript.jar -d tests/threadtest/
tests/threadtest/*.java
-------------------------------------------------------------------------
   You can use your own map! Copy the following command in your shell
        and change the file to the map file (the last argument)
-------------------------------------------------------------------------
        "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 7 running...
Thread 8 running...
Thread 5 running...
Thread 6 running...
Thread 3 running...
Thread 4 running...
Thread 2 running...
Thread 1 running...
Thread 9 running...
Exception in thread "Thread-0" Exception in thread "Thread-7" java.lang.NoClassDefFoundErr
or: Could not initialize class edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-6" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-1" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-5" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-9" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-4" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-2" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-3" java.lang.UnsatisfiedLinkError: D:\build\mapserver-buildkit
-2008\mapserver-unicolet\mapscript\java\javamapscript.dll: Can't find dependent libraries
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1778)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1703)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1030)
        at edu.umn.gis.mapscript.mapscriptJNI.<clinit>(mapscriptJNI.java:23)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-8" java.lang.NoClassDefFoundError: Could not initialize class
edu.umn.gis.mapscript.mapscriptJNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
java.lang.NoClassDefFoundError: Could not initialize class edu.umn.gis.mapscript.mapscript
JNI
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
@unicolet
Contributor

@jmckenna the error above is because java can't load javamapscript.dll due to missing dependencies:

 Exception in thread "Thread-3" java.lang.UnsatisfiedLinkError: D:\build\mapserver-buildkit -2008\mapserver-unicolet\mapscript\java\javamapscript.dll: Can't find dependent libraries

could you make sure that javamapscript.dll is in the mapscript\java directory, that all dependencies are met (ie they are in path) and then run again:

 nmake /f Makefile.vc threadtests
@jmckenna
Member

@unicolet

I've done as you requested. Results below:


        "C:\Progra~1\Java\jdk1.6.0_13"\bin\javac -cp ./mapscript.jar -d tests/threadtest/
tests/threadtest/*.java
-------------------------------------------------------------------------
   You can use your own map! Copy the following command in your shell
        and change the file to the map file (the last argument)
-------------------------------------------------------------------------
        "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 5 running...
Thread 4 running...
Thread 3 running...
Thread 1 running...
Thread 2 running...
Thread 9 running...
Thread 8 running...
Thread 7 running...
Thread 6 running...
Exception in thread "Thread-2" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326t#r╚F╜♫╚F╜
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-5" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-1" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-7" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-0" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-4" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-3" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-8" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-6" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
Exception in thread "Thread-9" java.lang.UnknownError: msProcessProjection(): Projection l
ibrary error. proj error "No such file or directory" for "init=epsg:4326:(null)"
        at edu.umn.gis.mapscript.mapscriptJNI.new_mapObj(Native Method)
        at edu.umn.gis.mapscript.mapObj.<init>(mapObj.java:283)
        at MapThread.run(MapThread.java:31)
@jmckenna
Member

Setting PROJ_LIB inside the mapfile allows us to step over that issue (the issue of Java MapScript not using the PROJ_LIB env variable). Next results are:

        "C:\Progra~1\Java\jdk1.6.0_13"\bin\javac -cp ./mapscript.jar -d tests/threadtest/
tests/threadtest/*.java
-------------------------------------------------------------------------
   You can use your own map! Copy the following command in your shell
        and change the file to the map file (the last argument)
-------------------------------------------------------------------------
        "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 5 running...
Thread 4 running...
Thread 9 running...
Thread 3 running...
Thread 1 running...
Thread 2 running...
Thread 8 running...
Thread 7 running...
Thread 6 running...
Thread 0-0 querying...

That querying is still going...........

@jmckenna
Member

Latest results from pull:

        "C:\Progra~1\Java\jdk1.6.0_13"\bin\javac -cp ./mapscript.jar -d tests/threadtest/
tests/threadtest/*.java
-------------------------------------------------------------------------
   You can use your own map! Copy the following command in your shell
        and change the file to the map file (the last argument)
-------------------------------------------------------------------------
        "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 5 running...
Thread 4 running...
Thread 3 running...
Thread 2 running...
Thread 1 running...
Thread 9 running...
Thread 8 running...
Thread 7 running...
Thread 6 running...
Thread 3-0 using geos to create a buffer...

stuck, no cpu activity..

@unicolet
Contributor

The fix for GCC has been pushed to master.
The fix for VC has been commented out instead : it builds, but hangs, see previous comments by @jmckenna .

@tbonfort
Member

undefined ref to __sync_sub_and_fetch

this is with:

gcc (GCC) 4.1.2 20070626 (Red Hat 4.1.2-13)

@unicolet
Contributor

@tbonfort I have worked around it and added a configure check for it, it's probably safer.

It's on my fork in the windows_ref_count_atomic branch. Diff:

https://github.com/unicolet/mapserver/compare/windows_ref_count_atomic

Could you test it?

@tbonfort
Member

@unicolet , this was reported to me by a client, I have no means to test this myself.

@unicolet
Contributor

@tbonfort perhaps then you could ask your client?

@unicolet
Contributor

The issue is fixed on all platforms where GCC is used and __sync_fetch_and_add is available.
For Windows there is a tentative fix, but since I can't test it (don't have Windows build environment) I have commented it out.

@unicolet unicolet closed this May 23, 2012
@unicolet unicolet added a commit to unicolet/mapserver that referenced this issue Jun 23, 2012
@unicolet unicolet fix #4231 for Windows too 6081792
@unicolet unicolet reopened this Jun 23, 2012
@unicolet
Contributor

I have a fix for Windows (it builds and runs with VS Express 10), could anyone test it before I merge it?

#4357

@jmckenna
Member

I've tried to build Java mapscript on Windows (MSVC 2008). Using branch 'windows_ref_count_atomic' I get the following errors:


mapscript_wrap.c(1498) : warning C4090: 'function' : different 'const' qualifiers
mapscript_wrap.c(1795) : error C2039: 'paginate' : is not a member of 'layer_obj'
d:\build\mapserver-buildkit-2008\mapserver-unicolet\mapserver.h(1450) : see declar
ation of 'layer_obj'
mapscript_wrap.c(1816) : error C2039: 'paginate' : is not a member of 'layer_obj'
d:\build\mapserver-buildkit-2008\mapserver-unicolet\mapserver.h(1450) : see declar
ation of 'layer_obj'
mapscript_wrap.c(1836) : error C2039: 'paginate' : is not a member of 'layer_obj'
d:\build\mapserver-buildkit-2008\mapserver-unicolet\mapserver.h(1450) : see declar
ation of 'layer_obj'
NMAKE : fatal error U1077: '"C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.EXE"'
: return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\nmake.EXE
"' : return code '0x2'

Stop.

@unicolet
Contributor

@jmckenna that is another error:

#4360

which I have already fixed, just pull again from upstream master

@jmckenna
Member

@unicolet I have updated your 'windows_ref_count_atomic' branch and here are my results of the threadtests:

        "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 6 running...
Thread 8 running...
Thread 5 running...
Thread 4 running...
Thread 3 running...
Thread 1 running...
Thread 2 running...
Thread 9 running...
Thread 7 running...
Thread 7-0 querying...
Thread 6-0 using geos to create a buffer...
Thread 4-0 querying...
Thread 9-0 using geos to create a buffer...
Thread 8-0 querying...
Thread 3-0 using geos to create a buffer...
Thread 0-0 using geos to create a buffer...
Thread 5-0 using geos to create a buffer...
Thread 1-0 querying...
Thread 2-0 using geos to create a buffer...
 numresults: 0
 numresults: 0
 numresults: 0
 numresults: 0
Thread 8-1 using geos to create a buffer...
Thread 2-1 using geos to create a buffer...
Thread 9-1 querying...
Thread 6-1 using geos to create a buffer...
Thread 7-1 using geos to create a buffer...
Thread 5-1 querying...
Thread 1-1 using geos to create a buffer...
Thread 3-1 querying...
Thread 0-1 querying...
 numresults: 0
Thread 4-1 querying...
 numresults: 0
Thread 2-2 querying...
 numresults: 0
Thread 1-2 querying...
Thread 8-2 querying...
 numresults: 0
Thread 9-2 querying...
 numresults: 0
 numresults: 0
Thread 7-2 using geos to create a buffer...
Thread 6-2 using geos to create a buffer...
Thread 2-3 using geos to create a buffer...
 numresults: 0
Thread 3-2 querying...
 numresults: 0
 numresults: 0
Thread 5-2 using geos to create a buffer...
Thread 7-3 using geos to create a buffer...
Thread 2-4 using geos to create a buffer...
 numresults: 0
Thread 6-3 using geos to create a buffer...
Thread 0-2 using geos to create a buffer...
Thread 4-2 querying...
Thread 2 done.
Thread 1-3 using geos to create a buffer...
Thread 8-3 using geos to create a buffer...
Thread 9-3 querying...
Thread 6-4 querying...
 numresults: 0
Thread 3-3 querying...
Thread 5-3 using geos to create a buffer...
Thread 7-4 using geos to create a buffer...
 numresults: 0
Thread 8-4 using geos to create a buffer...
Thread 1-4 using geos to create a buffer...
 numresults: 0
Thread 3-4 querying...
 numresults: 0
Thread 0-3 querying...
Thread 6 done.
 numresults: 0
Thread 1 done.
Thread 8 done.
Thread 7 done.
Thread 5-4 querying...
Thread 4-3 using geos to create a buffer...
 numresults: 0
Thread 3 done.
Thread 9-4 using geos to create a buffer...
 numresults: 0
Thread 5 done.
Thread 4-4 using geos to create a buffer...
Thread 9 done.
Thread 0-4 using geos to create a buffer...
Thread 4 done.
Thread 0 done.
@unicolet
Contributor

looks good, will merge later today
thanks Jeff
Il giorno 27/giu/2012 18:36, "Jeff McKenna" <
reply@reply.github.com>
ha scritto:

@unicolet I have updated your 'windows_ref_count_atomic' branch and here
are my results of the threadtests:

       "C:\Progra~1\Java\jdk1.6.0_13"\bin\java -cp
tests/threadtest/;./mapscript.jar -Dja
va.library.path=. MapTest -t 10 -i 5 ../../tests/test.map
Thread 0 running...
Thread 6 running...
Thread 8 running...
Thread 5 running...
Thread 4 running...
Thread 3 running...
Thread 1 running...
Thread 2 running...
Thread 9 running...
Thread 7 running...
Thread 7-0 querying...
Thread 6-0 using geos to create a buffer...
Thread 4-0 querying...
Thread 9-0 using geos to create a buffer...
Thread 8-0 querying...
Thread 3-0 using geos to create a buffer...
Thread 0-0 using geos to create a buffer...
Thread 5-0 using geos to create a buffer...
Thread 1-0 querying...
Thread 2-0 using geos to create a buffer...
 numresults: 0
 numresults: 0
 numresults: 0
 numresults: 0
Thread 8-1 using geos to create a buffer...
Thread 2-1 using geos to create a buffer...
Thread 9-1 querying...
Thread 6-1 using geos to create a buffer...
Thread 7-1 using geos to create a buffer...
Thread 5-1 querying...
Thread 1-1 using geos to create a buffer...
Thread 3-1 querying...
Thread 0-1 querying...
 numresults: 0
Thread 4-1 querying...
 numresults: 0
Thread 2-2 querying...
 numresults: 0
Thread 1-2 querying...
Thread 8-2 querying...
 numresults: 0
Thread 9-2 querying...
 numresults: 0
 numresults: 0
Thread 7-2 using geos to create a buffer...
Thread 6-2 using geos to create a buffer...
Thread 2-3 using geos to create a buffer...
 numresults: 0
Thread 3-2 querying...
 numresults: 0
 numresults: 0
Thread 5-2 using geos to create a buffer...
Thread 7-3 using geos to create a buffer...
Thread 2-4 using geos to create a buffer...
 numresults: 0
Thread 6-3 using geos to create a buffer...
Thread 0-2 using geos to create a buffer...
Thread 4-2 querying...
Thread 2 done.
Thread 1-3 using geos to create a buffer...
Thread 8-3 using geos to create a buffer...
Thread 9-3 querying...
Thread 6-4 querying...
 numresults: 0
Thread 3-3 querying...
Thread 5-3 using geos to create a buffer...
Thread 7-4 using geos to create a buffer...
 numresults: 0
Thread 8-4 using geos to create a buffer...
Thread 1-4 using geos to create a buffer...
 numresults: 0
Thread 3-4 querying...
 numresults: 0
Thread 0-3 querying...
Thread 6 done.
 numresults: 0
Thread 1 done.
Thread 8 done.
Thread 7 done.
Thread 5-4 querying...
Thread 4-3 using geos to create a buffer...
 numresults: 0
Thread 3 done.
Thread 9-4 using geos to create a buffer...
 numresults: 0
Thread 5 done.
Thread 4-4 using geos to create a buffer...
Thread 9 done.
Thread 0-4 using geos to create a buffer...
Thread 4 done.
Thread 0 done.

Reply to this email directly or view it on GitHub:
#4231 (comment)

@unicolet unicolet added a commit that closed this issue Jun 27, 2012
@unicolet unicolet fix #4231 for Windows too 6081792
@unicolet unicolet closed this in 6081792 Jun 27, 2012
@tbonfort tbonfort reopened this Jul 17, 2012
@tbonfort
Member

__sync_sub_and_fetch is still causing trouble, at least with gcc 4.1.2.1:

./.libs/libmapserver.so: undefined reference to `__sync_sub_and_fetch_4

this seems spot on: http://trac.wxwidgets.org/ticket/4542

@unicolet
Contributor

@tbonfort I'm surprised this has not been caught by the configure step since it no longer depends on compiler version, but will instead try to compile a small program against it. I'll check on it asap.

@unicolet
Contributor
unicolet commented Nov 6, 2012

Closing: definitely fixed in pull request 4502

@unicolet unicolet closed this Nov 6, 2012
@mkofahl mkofahl pushed a commit to faegi/mapserver that referenced this issue Apr 9, 2013
@unicolet unicolet improve __sync_sub_and_fetch detection #4231 c45e544
@mkofahl mkofahl pushed a commit to faegi/mapserver that referenced this issue Apr 9, 2013
@unicolet unicolet improve __sync_sub_and_fetch detection #4231 c3552d7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment