readcsv locks file in windows

[aviks]

          _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-rc2
 _/ |\__'_|_|_|\__'_|  |  Commit b372a68 2013-10-26 02:06:56 UTC
|__/                   |  i686-w64-mingw32

julia> a=[1 2 3 4 5]
1x5 Array{Int32,2}:
 1  2  3  4  5

julia> writecsv("test.csv", a)

julia> readcsv("test.csv")
1x5 Array{Float64,2}:
 1.0  2.0  3.0  4.0  5.0

julia> writecsv("test.csv", a)
ERROR: could not open file test.csv
 in open at io.jl:312

Closing and restarting the Julia REPL allows us to write once again

C:\Vega>c:\julia\julia-b372a68743\julia.bat
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-rc2
 _/ |\__'_|_|_|\__'_|  |  Commit b372a68 2013-10-26 02:06:56 UTC
|__/                   |  i686-w64-mingw32

julia> a=[1 2 3 4 5]
1x5 Array{Int32,2}:
 1  2  3  4  5

julia> writecsv("test.csv", a)

The behaviour is correct on MacOSX (a build a couple of days old)

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.2.0-rc1-prerelease+4163
 _/ |\__'_|_|_|\__'_|  |  Commit as/pkgfix/5259264* 2013-10-22 10:33:18 UTC
|__/                   |  x86_64-apple-darwin11.4.2

julia> a=[1 2 3 4 5]
1x5 Array{Int64,2}:
 1  2  3  4  5

julia> writecsv("test.csv", a)

julia> writecsv("test.csv", a)

julia> readcsv("test.csv")

1x5 Array{Float64,2}:
 1.0  2.0  3.0  4.0  5.0

julia> writecsv("test.csv", a)

julia> 

[JeffBezanson]

readcsv uses mmap, and we are probably not calling munmap until the array is finalized. Could you try calling gc() after readcsv to see if that fixes the following writecsv?

[wlbksy]

same bug post on julia-user

[tanmaykm]

Is it a Windows only issue? Works fine in OSX with the latest master.

Does calling gc() help?
It is also possible to call readcsv with named argument use_mmap=false.

[aviks]

I'd worked around it using use_mmap=false

Using gc() also fixes it.

julia> a=[1 2 3 4 5]
1x5 Array{Int32,2}:
 1  2  3  4  5

julia> writecsv("test.csv", a)

julia> readcsv("test.csv")

1x5 Array{Float64,2}:
 1.0  2.0  3.0  4.0  5.0

julia> writecsv("test.csv", a)
ERROR: could not open file test.csv
 in open at io.jl:312

julia> gc()

julia> writecsv("test.csv", a)

julia>

@tanmaykm yes, it works fine in OSX. This is a windows only problem.

[JeffBezanson]

We should probably add a finalize function to perform early finalization, and a with function to run a block guaranteeing finalization done before it exits.

[vtjnash]

This is essentially an issue with the windows file system being pretty awful. Mac and linux don't have that problem.

[StefanKarpinski]

To be clear, the Mac filesystem is awful too, just in different ways :-P

[vtjnash]

The Mac filesystem is case insensitive by choice (e.g. forced by Adobe), it isn't a fundamental limitation in the OS :-P

[StefanKarpinski]

True, but that's not the only issue with HFS+.

[vtjnash]

It appears that node/libuv has fixed this issue, if we were using libuv files: https://github.com/joyent/node/issues/1449

[JeffBezanson]

Just ran into this. It would be nice to fix it. Should we make use_mmap=false the default on windows, or add finalize?

[tkelman]

Being able to manually force a finalize could be useful in other contexts too, as long as it's not too dangerous (can use of a manually finalized object be made into an error without too much trouble?).

Windows mmap functionality is generally buggy in a number of packages aside from use in readcsv, I've been meaning to file one or more new issues on that.

[tanmaykm]

I think it is best not to use mmap by default on Windows for now. Have sent a PR updating the defaults.

[stevengj]

Now that we have a function finalize(a) that can force the munmap to be called, we should probably revisit this.