Thread Safety - Signal 11 faults if using multiple threads

[lawless-m]

I was advised that API is changing and to update my code from this

en2lalo(e, n) = Proj4.transform(Projection(Proj4.epsg[27700]), Projection(Proj4.epsg[4326]), [e, n])

to this

en2lalo(e, n) = Proj4.Transformation("EPSG:27700", "EPSG:4326", always_xy=true)([e,n])

And using mutliple threads (eleven) I get a received signal: 11 error

Commenting out the Threads.@threads made it work again

##Threads.@threads
    for i in 1:length(readers)
        dbs[i] = generate(readers[i])
    end

The threaded version processed 39 million records using 11 threads, so I am confident it is not the data or my code.

[visr]

Ah for multiple threading it is important to use separate contexts per thread.

https://proj.org/development/reference/datatypes.html#c.PJ_CONTEXT

So something like

ctx = proj_context_clone()
trans = Proj4.Transformation("EPSG:27700", "EPSG:4326"; always_xy=true, ctx)

Although I haven't tried this myself, so I'm curious if you can create a working example.

[visr]

Regarding this function:

en2lalo(e, n) = Proj4.Transformation("EPSG:27700", "EPSG:4326", always_xy=true)([e,n])

I think this creates a Transformation object each time, which is quite expansive to do for each point. So it's better to create it once (once per thread), and reuse for each projection.

[lawless-m]

Anyway, thanks. Making the en2lalo definition per thread completely solved the Sig. 11 problem.

I didn't do the context thing, I'm processing 11 files, 1 per thread, and combining the results afterwards

so instead of

en2lalo = ....

function process_file(f)
   for line in f
    en2lalo(row.e, row.n)
   end
end

Threads.@threads for f in file
    process_file(f)
end

I did

function process_file(f)
   en2lalo = ....
   for line in f
    en2lalo(row.e, row.n)
   end
end

Threads.@threads for f in file
    process_file(f)
end

And it worked great