New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception on ReadFromEnumerable with NULL values data #2489

Open
jwood803 opened this Issue Feb 10, 2019 · 1 comment

Comments

Projects
None yet
2 participants
@jwood803
Copy link
Contributor

jwood803 commented Feb 10, 2019

System information

  • OS version/distro:
    OS Name: Windows
    OS Version: 10.0.17134
    OS Platform: Windows
    RID: win10-x64
  • .NET Version (eg., dotnet --info):
    .NET Core SDK (reflecting any global.json):
    Version: 3.0.100-preview-009812

Issue

  • What did you do?
    Loaded the wine data into a database and read data to use in ReadFromEnumerable

  • What happened?
    Got the below exception when loading the data into ReadFromEnumerable. I believe this is because there are null values within the data.

     System.ArgumentOutOfRangeException: 'Could not determine an IDataView type for member 
     Parameter name: rawType'
    
  • What did you expect?

Source code / logs

Have this ML.NET code:

var dbData = ReadFromDatabase();

var context = new MLContext();

var mlData = context.Data.ReadFromEnumerable(dbData);

And a preview of dbData where I think it's failing at:

[
  {
    "Type": "white",
    "FixedAcidity": null,
    "VolatileAcidity": 1.0,
    "CitricAcid": 0.0,
    "ResidualSugar": 1.0,
    "Chlorides": 0.0,
    "FreeSulfurDioxide": 29.0,
    "TotalSulfureDioxide": 75.0,
    "Density": 1.0,
    "Ph": 3.0,
    "Sulphates": 0.0,
    "Alcohol": 13.0,
    "Quality": 8.0
  }
]

Also, is this the correct way to handle missing data when reading from a database instead of from a CSV file? Or should I be handling it another way?

@Ivanidzo4ka

This comment has been minimized.

Copy link
Member

Ivanidzo4ka commented Feb 11, 2019

You are right, it's related to nullable class. This cryptic message comes from here:

else if (!itemType.TryGetDataKind(out _))

where we don't know how to map Nullable to internal type.

@TomFinley I think we discussed that long time ago, and decided what user should handle that case (Nullable<Type>) by themselves. Is it still the case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment