Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate graph code for evaluate.cs #2470

Open
beccamc opened this issue Feb 1, 2023 · 4 comments
Open

Generate graph code for evaluate.cs #2470

beccamc opened this issue Feb 1, 2023 · 4 comments
Assignees
Labels
Priority:2 Work that is important, but not critical for the release Stale
Milestone

Comments

@beccamc
Copy link
Contributor

beccamc commented Feb 1, 2023

Generate graphs for evaluating models.

Regression

  • Compare actual to predicted values
@beccamc beccamc added this to the April 2023 milestone Feb 6, 2023
@beccamc
Copy link
Contributor Author

beccamc commented Feb 21, 2023

Here is the code for the R Squared explanation graph. This should be added for Regression scenarios.

  1. We will need to a reference to Plotly.Net to the project

    <PackageReference Include="Plotly.NET" Version="3.0.1" />

  2. Code to add to Evaluate.cs file.

        /// <summary>
        /// R Squared is a measure of variation between the values predicted by the model and the true values. 
        /// In a "perfect" model, there would be no variation between predictions and true values. 
        /// 
        /// Here we will plot the predicted values vs the true values for the trained model. This RegressionChart.html 
        /// is then saved to the location specified by <param name="folderPath"></param>. 
        /// 
        /// See more information on R Squared at https://en.wikipedia.org/wiki/Coefficient_of_determination. 
        /// </summary>
        /// <param name="trainData">IDataView used to train the model.</param>
        /// <param name="model">Model used for predictions.</param>
        /// <param name="labelColumnName">Name of the predicted label column.</param>
        /// <param name="folderPath">Folder path to save the RegressionChart.html file into.</param>
        public static void PlotRSquaredValues(IDataView trainData, ITransformer model, string labelColumnName, string folderPath)
        {
            // Number of rows to display in charts.
            int numberOfRows = 1000;
            // Use the model to make batch predictions on training data
            var testResults = model.Transform(trainData);

            // Get the actual values from the dataset
            var trueValues = testResults.GetColumn<float>(labelColumnName).Take(numberOfRows); ;

            // Get the predicted values from the test results
            var predictedValues = testResults.GetColumn<float>("Score").Take(numberOfRows);

            // Setup what the graph looks like
            var title = Title.init(Text: "R-Squared Plot");
            var layout = Layout.init<IConvertible>(Title: title, PlotBGColor: Plotly.NET.Color.fromString("#e5ecf6"));
            var xAxis = LinearAxis.init<IConvertible, IConvertible, IConvertible, IConvertible, IConvertible, IConvertible>(
                    Title: Title.init("True Values"),
                    ZeroLineColor: Plotly.NET.Color.fromString("#ffff"),
                    GridColor: Plotly.NET.Color.fromString("#ffff"),
                    ZeroLineWidth: 2);
            var yAxis = LinearAxis.init<IConvertible, IConvertible, IConvertible, IConvertible, IConvertible, IConvertible>(
                    Title: Title.init("Predicted Values"),
                    ZeroLineColor: Plotly.NET.Color.fromString("#ffff"),
                    GridColor: Plotly.NET.Color.fromString("#ffff"),
                    ZeroLineWidth: 2);

            // We will plot the line that shows the perfect result. Setup that line here.
            var maximumValue = Math.Max(trueValues.Max(), predictedValues.Max());
            var perfectX = new[] { 0, maximumValue };
            var perfectY = new[] { 0, maximumValue };



            // Create the scatterplot that shows the true values vs the predicted values
            var trueAndPredictedValues = Chart2D.Chart.Scatter<float, float, string>(x: trueValues, y: predictedValues, mode: StyleParam.Mode.Markers)
                            .WithLayout(layout)
                            .WithXAxis(xAxis)
                            .WithYAxis(yAxis);

            // Setup the line that shows what a perfect prediction would look like
            var perfectLineGraph = Chart2D.Chart.Line<float, float, string>(x: perfectX, y: perfectY)
                            .WithLayout(layout)
                            .WithLine(Line.init(Width: 1.5));

            var chartWithValuesAndIdealLine = Chart.Combine(new[] { trueAndPredictedValues, perfectLineGraph });
            var chartFilePath = folderPath + "\\RegressionChart.html";

            chartWithValuesAndIdealLine.SaveHtml(chartFilePath);
        }

@beccamc
Copy link
Contributor Author

beccamc commented Feb 21, 2023

@luisquintanilla Similar to our discussion on SQL library additions. In order to get graphing we need to add Plotly.NET dependency. We probably don't want to add this dependency every time someone trains. Is this graph something you can only get when generating one of our Consumption projects?

@github-actions github-actions bot added the Stale label Mar 24, 2023
@luisquintanilla
Copy link
Contributor

luisquintanilla commented Apr 10, 2023

If we're okay with adding additional dependencies, I'm okay with adding it to the code-behind in the project the ML model is added to as well as the consumption projects.

@JakeRadMSFT @zewditu @LittleLittleCloud thoughts on adding additional dependencies to projects?

@github-actions github-actions bot removed the Stale label Apr 11, 2023
@luisquintanilla luisquintanilla modified the milestones: May 2023, June 2023 Apr 17, 2023
@github-actions github-actions bot added the Stale label May 18, 2023
@luisquintanilla luisquintanilla added the Priority:2 Work that is important, but not critical for the release label May 22, 2023
@LittleLittleCloud
Copy link
Contributor

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:2 Work that is important, but not critical for the release Stale
Projects
None yet
Development

No branches or pull requests

4 participants