Support for direct access mode (#2)

Enabled by passing `access="direct"` and `recl` keyword options to the FortranFile constructor. read and write operations on such files must include the `rec` keyword to select which record to read/write.

We assume that such files use no record markers at all, which is what gfortran and ifort seem to be doing. The ifort documentation mentions a "vms" option, which uses a single control byte at the start of each record, but this is currently not supported here.

Closes #1.

docs/src/files.md

@@ -3,8 +3,8 @@
 ## Terminology
 
 When opening a file in Fortran, you can specify its *access mode*.
-The default and most commonly used mode is *sequential access*, and
-this is the only mode currently supported by this package.
+The default and most commonly used mode is *sequential access*.
+This package additionally supports *direct access* mode.
 (If the Fortran program uses *stream access* mode, then the file
 contains plain binary data, which can be easily read with Julia's
 built-in facilities.)
@@ -31,6 +31,11 @@ recollection and may be incorrect):
   the default.
 All these kinds of record markers are supported by this package.
 
+In direct access mode, all records have the same, fixed size. This record size
+must be specified when opening the file. Records can be accessed in random
+order, by specifying the number of the record to be read/written in each
+`READ` or `WRITE` statement.
+
 
 ## Opening files
 
@@ -92,3 +97,17 @@ probably corresponds to
 integer::lun
 open(newunit=lun, file="data.bin", form="unformatted", action="readwrite", position="append", status="unknown")
 ```
+
+#### Opening a file read-only in direct access mode
+
+```julia
+f = FortranFile("data.bin", "r", access="direct", recl=640)
+```
+for reading a file containing 640-byte records, and corresponds to
+```fortran
+integer::lun
+open(newunit=lun, file="data.bin", form="unformatted", action="read", status="old", access="direct", recl=640)
+```
+if compiled with gfortran; ifort measures `recl` not in bytes, but in 4-byte longwords, unless the
+compiler switch `-assume byterecl` is used.
+

docs/src/index.md

@@ -10,13 +10,14 @@ Currently the following features are implemented and working:
 * Sequential Access mode
   * 4-byte record markers, with subrecord support (allowing records larger than 2 GiB)
   * 8-byte record markers (used by early versions of gfortran)
+* Direct Access mode 
+  * fixed-size records without any record markers
 * Most standard Fortran datatypes, including arrays and strings
 * "Inhomogeneous" records, i.e. records made from multiple different datatypes
 * Byte-order conversion (little endian ⟷ big endian) 
 
 The following features are not (yet) supported:
 
-* Direct Access mode
 * Derived Type I/O
 * Equivalents of BACKSPACE and ENDFILE
 

docs/src/read.md

@@ -8,8 +8,12 @@ read
 
 The following examples show how to write Julia code that corresponds to
 certain Fortran `READ` statements. The Julia code assumes that `f` refers
-to an opened `FortranFile`, while the Fortran code assumes that `lun` refers
-to a logical unit number for a connected file.
+to an opened `FortranFile` in sequential access mode, while the Fortran
+code assumes that `lun` refers to a logical unit number for a connected file.
+
+For direct access mode, each `read` call additionally needs to specify the
+number of the record to read, by using the `rec` keyword argument.
+E.g. to read the first record, use `read(f, rec=1, ...)`.
 
 #### Reading a single scalar
 

docs/src/tests.md

@@ -23,7 +23,8 @@ The tests perform the following steps:
    skipped or read incompletely.
 1. Use the Julia code to write the data to an output file.
 1. Check that the input and output file are identical.
-This sequence of steps is performed for each of the tested record marker types,
-and each of the supported byte orders,
+This sequence of steps is performed for each of the tested record types
+(variable-length records with various types of record markers, and
+fixed-length records for direct access mode), and each of the supported byte orders,
 using the appropriate gfortran compiler options to adjust the Fortran output.
 

docs/src/write.md

@@ -8,8 +8,12 @@ write
 
 The following examples show how to write Julia code that corresponds to
 certain Fortran `WRITE` statements. The Julia code assumes that `f` refers
-to an opened `FortranFile`, while the Fortran code assumes that `lun` refers
-to a logical unit number for a connected file.
+to an opened `FortranFile` in sequential access mode, while the Fortran
+code assumes that `lun` refers to a logical unit number for a connected file.
+
+For direct access mode, each `write` call additionally needs to specify the
+number of the record to write, by using the `rec` keyword argument.
+E.g. to write the first record, use `write(f, rec=1, ...)`.
 
 #### Writing scalars
 

src/FortranFiles.jl

@@ -10,6 +10,7 @@ include("types.jl")
 include("file.jl")
 include("withoutsubrecords.jl")
 include("withsubrecords.jl")
+include("fixedlengthrecords.jl")
 include("string.jl")
 include("read.jl")
 include("write.jl")

src/file.jl

@@ -15,10 +15,16 @@ Wrap the given `IO` stream as a `FortranFile` containing Fortran "unformatted"
 * `access` for specifying the access mode; a `String` being one of
   * "sequential": sequential access as in Fortran, where records have leading
     and trailing record markers. This is the default.
-  * [nothing else at the moment]
+  * "direct": direct access as in Fortran, where records have fixed length
+    and can be accessed in random order. The "recl" keyword must be given
+    to specify the record length. `read` and `write` operations on these files
+    must use the `rec` keyword argument to specify which record to read/write.
 * `marker`: for specifying the type of record marker; one of
   * `RECMRK4B`: 4-byte record markers (with support for subrecords) [default]
   * `RECMRK8B`: 8-byte record markers
+  This is ignored for direct access files.
+* `recl`: for specifying the record length if access=="direct".
+  The record length is counted in bytes and must be specified as an Integer.
 * `convert`: for specifying the byte-order of the file data; one of
   * "native": use the host byte order [default]
   * "big-endian": use big-endian byte-order
@@ -27,14 +33,22 @@ Wrap the given `IO` stream as a `FortranFile` containing Fortran "unformatted"
 The returned `FortranFile` can be used with Julia's `read` and `write`
 functions. See their documentation for more information.
 """
-function FortranFile(io::IO; access = "sequential", marker = RECMRKDEF, convert = "native")
+function FortranFile(io::IO; access = "sequential", marker = RECMRKDEF, recl::Integer = 0, convert = "native")
    conv = get_convert(convert)
    if access == "sequential"
+      if recl != 0
+         error("sequential-access with fixed-length records not supported")
+      end
       acctyp = SequentialAccess(marker)
-      return FortranFile(io, acctyp, conv)
+   elseif access == "direct"
+      if recl == 0
+         error("must specify record length for direct-access files (use recl keyword argument)")
+      end
+      acctyp = DirectAccess(Int64(recl))
    else
       error("unsupported access mode \"$(access)\"")
    end
+   return FortranFile(io, acctyp, conv)
 end
 
 

src/fixedlengthrecords.jl

@@ -0,0 +1,62 @@
+import Base: close, unsafe_read, unsafe_write
+
+type FixedLengthRecord{T,C} <: Record
+   io       :: IO    # underlying I/O stream
+   reclen   :: T     # length of this record
+   nleft    :: T     # bytes left in this record
+   writable :: Bool  # whether this record is writable
+   convert  :: C     # convert method
+end
+
+function Record{C}( f::FortranFile{DirectAccess,C} )
+## constructor for readable records
+   conv = f.convert
+   reclen = f.acctyp.reclen
+   FixedLengthRecord(f.io, reclen, reclen, false, conv)
+end
+
+function Record{C}( f::FortranFile{DirectAccess,C}, towrite::Integer )
+## constructor for writable records
+   conv = f.convert
+   reclen = f.acctyp.reclen
+   if towrite > reclen
+      error("attempting to write record of $(towrite) bytes into a file of record length $(reclen) bytes")
+   end
+   FixedLengthRecord(f.io, reclen, reclen, true, conv)
+end
+
+function gotorecord( f::FortranFile{DirectAccess}, recnum::Integer )
+   reclen = f.acctyp.reclen
+   pos = (recnum-1) * reclen
+   seek(f.io, pos)
+end
+
+function unsafe_read( rec::FixedLengthRecord, p::Ptr{UInt8}, n::UInt )
+   if (n > rec.nleft); error("attempting to read beyond record end"); end
+   unsafe_read( rec.io, p, n )
+   rec.nleft -= n
+   nothing
+end
+
+function unsafe_write( rec::FixedLengthRecord, p::Ptr{UInt8}, n::UInt )
+   if (n > rec.nleft); error("attempting to write beyond record end"); end
+   nwritten = unsafe_write( rec.io, p, n )
+   rec.nleft -= n
+   return nwritten
+end
+
+function close( rec::FixedLengthRecord )
+   if rec.nleft != 0
+      if rec.writable
+         # Fortran standard 9.6.4.5.2 point 7:
+         # "If the file is connected for direct access and the values specified by the
+         #  output list do not fill the record, the remainder of the record is undefined."
+         # Following gfortran, we fill it with zeros.
+         write(rec.io, zeros(UInt8, rec.nleft))
+      else
+         skip(rec.io, rec.nleft)
+      end
+   end
+   nothing
+end
+

src/read.jl

@@ -2,6 +2,7 @@ import Base: read
 
 """
     read(f::FortranFile [, spec [, spec [, ...]]])
+    read(f::FortranFile, rec=N, [, spec [, spec [, ...]]])
 
 Read data from a `FortranFile`. Like the READ statement in Fortran, this
 reads a completely record, regardless of how man `spec`s are given. Each
@@ -14,25 +15,40 @@ reads a completely record, regardless of how man `spec`s are given. Each
 * an array, for reading into pre-allocated arrays; `DataType` and size
   of the array are implied through its Julia type.
 
+For direct-access files, the number of the record to be read must be
+specified with the `rec` keyword (N=1 for the first record).
+
 Return value:
 * if no `spec` is given: nothing (the record is skipped over)
 * if one `spec` is given: the scalar or array requested
 * if more `spec`s are given: a tuple of the scalars and arrays requested, in order
 """
-function read( f::FortranFile )
+function read( f::FortranFile, specs...)
+   fread(f, specs...)
+end
+
+function read( f::FortranFile{DirectAccess}, specs... ; rec::Integer=0 )
+   if rec == 0
+      error("direct-access files require specifying the record to be read (use rec keyword argument)")
+   end
+   gotorecord(f, rec)
+   fread( f, specs... )
+end
+
+function fread( f::FortranFile )
    rec = Record(f)
    close(rec)
    return nothing
 end
 
-function read( f::FortranFile, spec )
+function fread( f::FortranFile, spec )
    rec = Record(f)
    data = read_spec(rec, spec)
    close(rec)
    return data
 end
 
-function read( f::FortranFile, specs... )
+function fread( f::FortranFile, specs... )
    rec = Record(f)
    data = map( spec->read_spec(rec,spec), specs)
    close(rec)

src/types.jl

@@ -13,6 +13,14 @@ show(io::IO, a::SequentialAccess) =
    print(io, "sequential-access, ", a.recmrktyp)
 
 
+immutable DirectAccess <: AccessMode
+   reclen :: Int64
+end
+
+show(io::IO, a::DirectAccess) =
+   print(io, "direct-access, $(a.reclen)-byte records")
+
+
 immutable WithoutSubrecords{T} <: RecordMarkerType
 end
 

src/write.jl

@@ -2,20 +2,36 @@ import Base: write
 
 """
     write(f::FortranFile, items...)
+    write(f::FortranFile, rec=N, items...)
 
 Write a data record to a `FortranFile`. Each `item` should be a scalar
 of a Fortran-compatible datatype (e.g. `Int32`, `Float64`, `FString{10}`),
 or an array of such scalars. If no `item`s are given, an empty record is
 written. Returns the number of bytes written, **not** including the space
 taken up by the record markers.
+
+For direct-access files, the number of the record to be written must be
+specified with the `rec` keyword (N=1 for the first record).
 """
-function write( f::FortranFile )
+function write(f::FortranFile, items...)
+   fwrite(f, items...)
+end
+
+function write(f::FortranFile{DirectAccess}, items...; rec::Integer=0)
+   if rec==0
+      error("direct-access files require specifying the record to be written (use rec keyword argument)")
+   end
+   gotorecord(f, rec)
+   fwrite(f, items...)
+end
+
+function fwrite( f::FortranFile )
    rec = Record(f, 0)
    close(rec)
    return 0
 end
 
-function write( f::FortranFile, vars... )
+function fwrite( f::FortranFile, vars... )
    # how much data to write?
    towrite = sum( sizeof_var(var) for var in vars )
    rec = Record(f, towrite)

test/codegen/Makefile

@@ -7,15 +7,25 @@ RANDOMEXT := i1 i2 i4 i8 r4 r8 c8 c16 str
 RANDOMSRC := $(RANDOMEXT:%=random%.f90)
 RANDOMOBJ := $(RANDOMEXT:%=random%.o)
 
-GENDATAOBJ := $(RANDOMOBJ) randutil.o gendata$(SUFFIX).o
+GENDATASEQOBJ := $(RANDOMOBJ) randutil.o gendataseq$(SUFFIX).o
+GENDATADIRECTOBJ := $(RANDOMOBJ) randutil.o gendatadirect$(SUFFIX).o
 
 JGENSRC := jwrite.jl jread.jl jskip.jl
 FGENSRC := fdecl.f90 fwrite.f90
 
-gendata$(SUFFIX).x: $(GENDATAOBJ)
+gendataseq: gendataseq$(SUFFIX).x
+gendatadirect: gendatadirect$(SUFFIX).x
+
+gendataseq$(SUFFIX).x: $(GENDATASEQOBJ)
+	$(LINK) -o $@ $+
+
+gendatadirect$(SUFFIX).x: $(GENDATADIRECTOBJ)
 	$(LINK) -o $@ $+
 
-gendata$(SUFFIX).o: gendata.F90 $(FGENSRC)
+gendataseq$(SUFFIX).o: gendataseq.F90 $(FGENSRC)
+	$(FC) $(FFLAGS) $(XFLAGS) -c -o $@ $<
+
+gendatadirect$(SUFFIX).o: gendatadirect.F90
 	$(FC) $(FFLAGS) $(XFLAGS) -c -o $@ $<
 
 $(RANDOMSRC): .random-stamp