Skip to content
Permalink
Browse files

Removing processing of extraneous blank fields and enlarged arrays

  • Loading branch information
ccrook committed Mar 21, 2015
1 parent 0beb170 commit 99e6a6253fbdf688188e008af0412945f2f60007
@@ -35,7 +35,9 @@ be double, otherwise the type will be text.
QGIS can also read the types from an OGR CSV driver compatible "csvt" file.
This is a file alongside the data file, but with a "t" appended to the file name.
The file should just contain one line which lists the type of each field.
Valid types are "integer", "real", "string", "date", "time", and "datetime". The date, time, and datetime types are treated as strings in QGIS.
Valid types are "integer", "long", "longlong", "real",
"string", "date", "time", and "datetime".
The date, time, and datetime types are treated as strings by the delimited text provider.
Each type may be followed by a width and precision, for example "real(10.4)".
The list of types are separated by commas, regardless of the delimiter used in the data file. An
example of a valid format file would be:
@@ -215,9 +217,11 @@ feature id to each record which is the line number in the source file on which
the record starts.
</p>
<p>
Each attribute also has a data type, one of string (text), integer, or real number.
Each attribute also has a data type, one of string (text), integer, longlong,
or real number.
The data type is inferred from the content of the fields - if every non blank value
is a valid integer then the type is integer, otherwise if it is a valid real
is a valid integer then the type is integer, otherwise if it is a valid long long
nubmer then the type is longlong, otherwise if it is a valid real
number then the type is real, otherwise the type is string. Note that this is
based on the content of the fields - quoting fields does not change the way they
are interpreted.
@@ -562,29 +562,46 @@ void QgsDelimitedTextProvider::scanFile( bool buildIndexes )
{

QString &value = parts[i];
if ( value.isEmpty() ) {
// Ignore empty fields - spreadsheet generated CSV files often
// have random empty fields at the end of a row
if ( value.isEmpty() )
continue;

// Expand the columns to include this non empty field if necessary

while ( couldBeInt.size() <= i )
{
isEmpty.append( true );
couldBeInt.append( false );
couldBeLongLong.append( false );
couldBeDouble.append( false );
continue;
}
// try to convert attribute values to integer, long and double
isEmpty.append( false );
// try all possible formats
couldBeInt.append( true );
couldBeLongLong.append( true );
couldBeDouble.append( true );

// If this column has been empty so far then initiallize it
// for possible types

if ( isEmpty[i] )
{
isEmpty[i] = false;
couldBeInt[i] = true;
couldBeLongLong[i] = true;
couldBeDouble[i] = true;
}

// Now test for still valid possible types for the field
// Types are possible until first record which cannot be parsed

if ( couldBeInt[i] )
{
value.toInt( &couldBeInt[i] );
}

if ( couldBeLongLong[i] )
if ( couldBeLongLong[i] && ! couldBeInt[i] )
{
value.toLongLong( &couldBeLongLong[i] );
}
if ( couldBeDouble[i] )

if ( couldBeDouble[i] && ! couldBeLongLong[i] )
{
if ( ! mDecimalPoint.isEmpty() )
{
@@ -627,7 +644,7 @@ void QgsDelimitedTextProvider::scanFile( bool buildIndexes )
fieldType = QVariant::LongLong; //QVariant doesn't support long
typeName = "longlong";
}
else if ( csvtTypes[i] == "real" )
else if ( csvtTypes[i] == "real" || csvtTypes[i] == "double" )
{
fieldType = QVariant::Double;
typeName = "double";

0 comments on commit 99e6a62

Please sign in to comment.
You can’t perform that action at this time.