Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CreateEnumerable from key column #5270

Closed
go2ready opened this issue Jul 1, 2020 · 2 comments
Closed

CreateEnumerable from key column #5270

go2ready opened this issue Jul 1, 2020 · 2 comments
Assignees
Labels
P3 Doc bugs, questions, minor issues, etc. question Further information is requested

Comments

@go2ready
Copy link

go2ready commented Jul 1, 2020

System information

  • OS version/distro: Windows 10
  • .NET Version (eg., dotnet --info): 4.7

Issue

  • What did you do?
    Trying to create Enumerable from IDataView which contains Column with Type Key<UInt32, 0-1059>, into a uint type.
  • What happened?
    Encountered error:
    "Can't bind the IDataView column 'ImpressionIdKey' of type 'Key<UInt32, 0-1059>' to field or property 'ImpressionIdKey' of type 'System.UInt32'."
  • What did you expect?
    I want to be able to export key value in its uint format as enumerable

Source code / logs

        var a = mlContext.Data.CreateEnumerable<ProcessedData>(
            colSelTrainingData, reuseRowObject: false);

...

        private class ProcessedData
        {
            public float[] Feature { get; set; }

            public float BackProClick { get; set; }

            public uint ImpressionIdKey { get; set; }
        }
@antoniovs1029 antoniovs1029 added P3 Doc bugs, questions, minor issues, etc. question Further information is requested labels Jul 1, 2020
@antoniovs1029
Copy link
Member

antoniovs1029 commented Jul 1, 2020

Hi, @go2ready . It should be possible for you to create an enumerable from a key column. I don't know why you're getting that exception. Can you please provide a full stacktrace, and also a .zip containing code and data to reproduce your error so I can look closer? Thanks.

For reference, I've just ran this toy example, and it worked as expected, showing you can create enumerables from key columns:

Click to toggle toy example

using System;
using Microsoft.ML;
using Microsoft.ML.Data;
using System.Linq;

namespace TextLoaderSample
{
    class Program
    {
        public class ModelInput
        {
            public int Id { get; set; }

            public string Description { get; set; }

            public float Num1 { get; set; }

            public float Num2 { get; set; }

        }

        public class ModelOutput
        {
            public uint KeyDescription { get; set; }
        }

        static void Main(string[] args)
        {
            MLContext mlContext = new MLContext(seed: 1);

            var inputList = new[]
            {
                new ModelInput(){Id = 0, Description = "lion", Num1 = 12.333f, Num2 = 13.44f},
                new ModelInput(){Id = 1, Description = "house", Num1 = 12.333f, Num2 = 13.44f},
            };

            IDataView inputDV = mlContext.Data.LoadFromEnumerable(inputList);

            var pipeline = mlContext.Transforms.Conversion.Hash("KeyDescription", "Description");
            var outputDV = pipeline.Fit(inputDV).Transform(inputDV); // "KeyDescription" column is type "Key<UInt32, 0-2147483647>"

            var outputEnum = mlContext.Data.CreateEnumerable<ModelOutput>(outputDV, reuseRowObject: false);
            var outputArray = outputEnum.ToArray();
        }
    }
}

@antoniovs1029 antoniovs1029 self-assigned this Jul 1, 2020
@go2ready
Copy link
Author

go2ready commented Jul 1, 2020

Thanks @antoniovs1029 for providing the code snippet, I tired this morning it worked, must be something I have been doing wrong on the pipeline :) Really appreciated

@go2ready go2ready closed this as completed Jul 1, 2020
@dotnet dotnet locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
P3 Doc bugs, questions, minor issues, etc. question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants