Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This has been addressed earlier but I am getting the error again - "No header record found” on reading a csv stream" #1634

Open
ankku opened this issue Dec 25, 2020 · 1 comment
Labels

Comments

@ankku
Copy link

ankku commented Dec 25, 2020

Below is the method i am trying to read a stream and I am getting the error as No header record found. I have checked the stream and its return length of stream. and also to verify that my stream is valid I checked with writing it in StreamReader and then using readtoEnd() and it gives me all contents. Now I cant do readToEnd() for large files so going for Stream. below is the method How I am reading blobs and passing stream. I am using the CSVHelper version 15 here

 public Type DataType
        {
            get
            {
                switch( Type.ToUpper() )
                {
                    case "STRING":
                        return typeof(string);

                    case "INT":
                        return typeof( int );

                    case "BOOL":
```
                    case "BOOLEAN":
                        return typeof( bool );

                    case "FLOAT":
                    case "SINGLE":
                    case "DOUBLE":
                        return typeof( double );

                    case "DATETIME":
                        return typeof( DateTime );

                    default:
                        throw new NotSupportedException( $"CSVColumn data type '{Type}' not supported" );
                }
            }
        }
// This is the method where the stream is passed and the error is received at line "csv.ReadHeader()"
```
private IEnumerable<Dictionary<string, EntityProperty>> ReadCSV(Stream source, IEnumerable<TableField> cols)          

{              
         using (TextReader reader = new StreamReader(source, Encoding.UTF8))
                {
               
                    var cache = new TypeConverterCache();
                    cache.AddConverter<float>(new CSVSingleConverter());
                    cache.AddConverter<double>(new CSVDoubleConverter());
                    var csv = new CsvReader(reader,
                        new CsvHelper.Configuration.CsvConfiguration(global::System.Globalization.CultureInfo.InvariantCulture)
                        {
                            Delimiter = ";",
                            HasHeaderRecord = true,
                            CultureInfo = global::System.Globalization.CultureInfo.InvariantCulture,
                            TypeConverterCache = cache
                        });
                    csv.Read();
                    csv.ReadHeader();


                    var map = (
                            from col in cols
                            from src in col.Sources()
                            let index = csv.GetFieldIndex(src, isTryGet: true)
                            where index != -1
                            select new { col.Name, Index = index, Type = col.DataType }).ToList();

                    while (csv.Read())
                    {
                        yield return map.ToDictionary(
                            col => col.Name,
                            col => EntityProperty.CreateEntityPropertyFromObject(csv.GetField(col.Type, col.Index)));
                    }
               
                }
```
// This is the method from where the stream is being returned to ReadCsv() method above
```
public async Task<Stream> ReadStream(string containerName, string digestFileName, string fileName, string connectionString)
        {
            string data = string.Empty;
            string fileExtension = Path.GetExtension(fileName);
            var contents = await DownloadBlob(containerName, digestFileName, connectionString);              
               
           
            return contents;
        }

// method where blob is read as stream
 public async Task<Stream> DownloadBlob(string containerName, string fileName, string connectionString)
        {
           
            Microsoft.Azure.Storage.CloudStorageAccount storageAccount = Microsoft.Azure.Storage.CloudStorageAccount.Parse(connectionString);
            CloudBlobClient serviceClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = serviceClient.GetContainerReference(containerName);
            CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
            if (!blob.Exists())
            {
                throw new Exception($"Unable to upload data in table store for document");
            }
           
            return await blob.OpenReadAsync();
           
        }`
```
@ankku ankku added the bug label Dec 25, 2020
@JoshClose
Copy link
Owner

Read returns a bool. If that bool is false, meaning there is no more data, you'll get that error when calling ReadHeader because there is no header record. Basically it's an empty file.

I can't think of any other reason this would happen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants