machine-learning/E2E-Forecasting using SSA with Luna Dataset.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Forecasting using SSA with Luna Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Forecasting the next-hour load using SSA on Luna dataset\n",
        "\n",
        "This example shows how to train a time-series forecasting model using the Luna dataset. In this notebook you will learn:\n",
        "- How to run hyper-parameter optimization and search for the best model for your data using SSA and built-in `AutoMLExperiment` class."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Install NuGet packages for training ML.NET models and plotting:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div><div><strong>Restore sources</strong><ul><li><span>https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json</span></li></ul></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Microsoft.Data.Analysis, 0.20.0-preview.22458.1</span></li><li><span>Microsoft.ML.AutoML, 0.20.0-preview.22458.1</span></li><li><span>Plotly.NET.CSharp, 0.0.1</span></li><li><span>Plotly.NET.Interactive, 3.0.2</span></li></ul></div></div>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Loading extensions from `Microsoft.Data.Analysis.Interactive.dll`"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Loading extensions from `Microsoft.ML.AutoML.Interactive.dll`"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/markdown": [
              "Loading extensions from `Plotly.NET.Interactive.dll`"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "// using nightly-build\n",
        "#i \"nuget:https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json\"\n",
        "#r \"nuget: Plotly.NET.Interactive, 3.0.2\"\n",
        "#r \"nuget: Plotly.NET.CSharp, 0.0.1\"\n",
        "#r \"nuget: Microsoft.ML.AutoML, 0.20.0-preview.22458.1\"\n",
        "#r \"nuget: Microsoft.Data.Analysis, 0.20.0-preview.22458.1\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Import packages"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "using static Microsoft.DotNet.Interactive.Formatting.PocketViewTags;\n",
        "using Microsoft.Data.Analysis;\n",
        "using System;\n",
        "using System.IO;\n",
        "using Microsoft.ML;\n",
        "using Microsoft.ML.AutoML;\n",
        "using Microsoft.ML.Data;\n",
        "using Plotly.NET;\n",
        "using Microsoft.ML.Transforms.TimeSeries;\n",
        "using Microsoft.ML.SearchSpace;\n",
        "using System.Diagnostics;"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Import Dataset\n",
        "\n",
        "Luna is a time-series dataset which records the metric of hourly-active usage of an internal-used service on Azure. It has two columns: `DateTime` and `load` where `load` column records the metric of hourly-active usage. Luna shows a strong weekly seasonal pattern given its nature as an Azure service, and a slow trend (growth with time) as well. Those features allow you to build a forecasting model to predict the next hour load metric to adjust the size of computing power accordingly.\n",
        "\n",
        "In the code block below, we show how to load dataset into `DataFrame`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Download or Locate Data\n",
        "The following code tries to locate the data file in a few known locations or it will download it from the known GitHub location."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "using System;\n",
        "using System.IO;\n",
        "using System.Net;\n",
        "\n",
        "string EnsureDataSetDownloaded(string fileName)\n",
        "{\n",
        "\n",
        "\t// This is the path if the repo has been checked out.\n",
        "\tvar filePath = Path.Combine(Directory.GetCurrentDirectory(),\"data\", fileName);\n",
        "\n",
        "\tif (!File.Exists(filePath))\n",
        "\t{\n",
        "\t\t// This is the path if the file has already been downloaded.\n",
        "\t\tfilePath = Path.Combine(Directory.GetCurrentDirectory(), fileName);\n",
        "\t}\n",
        "\n",
        "\tif (!File.Exists(filePath))\n",
        "\t{\n",
        "\t\tusing (var client = new WebClient())\n",
        "\t\t{\n",
        "\t\t\tclient.DownloadFile($\"https://raw.githubusercontent.com/dotnet/csharp-notebooks/main/machine-learning/data/{fileName}\", filePath);\n",
        "\t\t}\n",
        "\t\tConsole.WriteLine($\"Downloaded {fileName}  to : {filePath}\");\n",
        "\t}\n",
        "\telse\n",
        "\t{\n",
        "\t\tConsole.WriteLine($\"{fileName} found here: {filePath}\");\n",
        "\t}\n",
        "\n",
        "\treturn filePath;\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Luna.csv found here: c:\\Users\\xiaoyuz\\source\\repos\\csharp-notebooks\\machine-learning\\data\\Luna.csv\r\n"
          ]
        }
      ],
      "source": [
        "var dataPath = EnsureDataSetDownloaded(@\"Luna.csv\");\n",
        "var df = DataFrame.LoadCsv(dataPath);\n",
        "var loads = df[\"load\"].Cast<float?>();"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Plotting last three weeks of Luna"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "<div>\n",
              "    <div id=\"639a2709-aec4-4780-9fc6-969b6a0da29c\"><!-- Plotly chart will be drawn inside this DIV --></div>\r\n",
              "<script type=\"text/javascript\">\r\n",
              "\r\n",
              "            var renderPlotly_639a2709aec447809fc6969b6a0da29c = function() {\r\n",
              "            var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-2.6.3.min'}}) || require;\r\n",
              "            fsharpPlotlyRequire(['plotly'], function(Plotly) {\r\n",
              "\r\n",
              "            var data = [{\"type\":\"scatter\",\"name\":\"load\",\"mode\":\"lines\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503],\"y\":[1244682.0,993425.0,784059.0,659575.0,618020.0,587536.0,656175.0,719350.0,782053.0,377668.0,41744.0,41259.0,59032.0,84875.0,129043.0,131641.0,289836.0,324920.0,491942.0,469317.0,271060.0,790696.0,28481.0,25086.0,1166220.0,1375564.0,10131135.0,13797705.0,720077.0,443333.0,424187.0,438539.0,447839.0,490204.0,572431.0,760359.0,925447.0,1015902.0,1036829.0,1005726.0,1018290.0,879402.0,857873.0,788077.0,705716.0,680384.0,511480.0,519391.0,458420.0,437325.0,389174.0,341664.0,330065.0,331946.0,309777.0,364589.0,381155.0,392688.0,494031.0,591913.0,729360.0,839978.0,879891.0,808652.0,826177.0,789247.0,701204.0,737962.0,719838.0,708549.0,653841.0,698137.0,717881.0,766936.0,699714.0,630697.0,632418.0,672336.0,791854.0,947877.0,1025792.0,1100498.0,1265976.0,1762765.0,2582223.0,3525729.0,3889515.0,3948050.0,3745920.0,3394873.0,3383188.0,3285317.0,3036515.0,2517527.0,1626746.0,1475707.0,1427133.0,1270143.0,961586.0,821726.0,742111.0,758311.0,890960.0,980905.0,1078892.0,1151264.0,1301542.0,1768634.0,2669396.0,3642494.0,4075830.0,3983445.0,3640118.0,3609375.0,3488198.0,3332503.0,2979552.0,2492888.0,1655599.0,1507789.0,1312478.0,1173607.0,951998.0,847015.0,789866.0,797501.0,933284.0,1033695.0,1040847.0,1140580.0,1333189.0,1796120.0,2559730.0,3191097.0,3413288.0,3656260.0,3480386.0,3316502.0,3241861.0,3180167.0,2945007.0,2486358.0,1625965.0,1512705.0,1383126.0,1249839.0,991129.0,852538.0,1094460.0,787062.0,895449.0,1032035.0,1175511.0,1134514.0,1429905.0,1861162.0,2761320.0,3532327.0,3770989.0,3949709.0,3757184.0,3454248.0,3406394.0,3285810.0,3019567.0,2507959.0,1695169.0,1547924.0,1396631.0,1239369.0,974823.0,866852.0,848025.0,820796.0,907761.0,1021858.0,1060416.0,1144608.0,1368263.0,1783984.0,2297043.0,2715306.0,2731480.0,2881350.0,2821269.0,2725415.0,2740686.0,2708103.0,2506036.0,2101946.0,1344942.0,1225594.0,1021260.0,876134.0,651912.0,556682.0,503380.0,466372.0,448718.0,525430.0,517454.0,560738.0,662374.0,774495.0,900277.0,1031882.0,1115998.0,1064012.0,984538.0,859138.0,786860.0,748414.0,711653.0,643469.0,522330.0,507266.0,482923.0,479022.0,416528.0,354112.0,362443.0,332783.0,366223.0,404525.0,411918.0,423643.0,509883.0,661678.0,797081.0,869201.0,939310.0,860191.0,896668.0,790226.0,830925.0,797267.0,730847.0,710141.0,622980.0,710878.0,731457.0,790050.0,673955.0,607948.0,659456.0,651493.0,775742.0,926070.0,1036212.0,1109769.0,1374389.0,1900993.0,2830465.0,3730390.0,4063176.0,4075144.0,3921920.0,3698129.0,3498237.0,3271959.0,3081571.0,2585828.0,1781072.0,1585949.0,1449976.0,1251014.0,942425.0,828102.0,779533.0,766892.0,827139.0,999466.0,1027640.0,1151513.0,1382976.0,1891899.0,2860401.0,3876586.0,4236497.0,4281556.0,3795041.0,3662663.0,3485182.0,3293806.0,3159509.0,2625952.0,1780764.0,1561361.0,1381776.0,1189885.0,972750.0,840883.0,905355.0,855915.0,964200.0,1112917.0,1142336.0,1281667.0,1589824.0,2062153.0,2837300.0,3586612.0,3760053.0,3965720.0,3728705.0,3677397.0,3653702.0,3518544.0,3403602.0,2870308.0,1998886.0,2141190.0,1973818.0,1674698.0,1426299.0,1286974.0,1196719.0,1165946.0,1313885.0,1497647.0,1512771.0,1694122.0,2050635.0,2616435.0,3738119.0,5171418.0,5571983.0,5607505.0,5428267.0,5170756.0,4995878.0,4852746.0,4353343.0,3670371.0,2442195.0,2475439.0,2140878.0,1895147.0,1579756.0,1402062.0,1322785.0,1165658.0,1131081.0,1244647.0,1313523.0,1327327.0,1506570.0,1799622.0,2468453.0,3322579.0,3490262.0,3537042.0,3357287.0,3054315.0,2878427.0,2817847.0,2560142.0,2086403.0,1311481.0,1131604.0,835595.0,706414.0,564374.0,501562.0,460359.0,407497.0,524727.0,513550.0,498451.0,471818.0,568307.0,736493.0,880628.0,980110.0,1056655.0,960999.0,944985.0,792694.0,730271.0,704140.0,667614.0,615589.0,464296.0,476380.0,456389.0,435793.0,371625.0,335967.0,349326.0,320461.0,360233.0,401795.0,400516.0,397158.0,480181.0,593995.0,728192.0,791722.0,842580.0,787016.0,789116.0,720734.0,727569.0,693815.0,700541.0,675098.0,602570.0,654610.0,700733.0,754041.0,716777.0,641627.0,647768.0,720232.0,826600.0,955412.0,989776.0,1085038.0,1248893.0,1755543.0,2648495.0,3460453.0,3797630.0,3866752.0,3578963.0,3419092.0,3371077.0,3133834.0,2904028.0,2427113.0,1536951.0,1431362.0,1240912.0,1152746.0,1017037.0,879557.0,794088.0,776213.0,866713.0,994684.0,1053025.0,1082314.0,1272833.0,1740110.0,2559427.0,3543817.0,3795470.0,3786953.0,3478702.0,3325876.0,3290135.0,3147765.0,2918741.0,2411975.0,1507603.0,1415209.0,1207824.0,1049837.0,891704.0,899598.0,859797.0,852082.0,1017008.0,1110451.0,1186555.0,1212441.0,1413617.0,1954505.0,2815706.0,3879214.0,4287130.0,4297613.0,4004459.0,3668727.0,3560500.0,3427783.0,3095399.0,2544989.0,1552230.0,1468952.0,1401150.0,1246200.0,1018963.0,893128.0,836865.0,838962.0,888329.0,1035122.0,1092601.0,1138552.0,1369574.0,1856648.0,2527278.0,3220888.0,3388005.0,3576955.0,3282861.0,3294191.0,3219435.0,3105795.0,2722333.0,2231878.0,1382403.0,1330587.0],\"marker\":{},\"line\":{}}];\r\n",
              "            var layout = {\"width\":600,\"height\":600,\"template\":{\"layout\":{\"title\":{\"x\":0.05},\"font\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"paper_bgcolor\":\"rgba(255, 255, 255, 1.0)\",\"plot_bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"autotypenumbers\":\"strict\",\"colorscale\":{\"diverging\":[[0.0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1.0,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"geo\":{\"showland\":true,\"landcolor\":\"rgba(229, 236, 246, 1.0)\",\"showlakes\":true,\"lakecolor\":\"rgba(255, 255, 255, 1.0)\",\"subunitcolor\":\"rgba(255, 255, 255, 1.0)\",\"bgcolor\":\"rgba(255, 255, 255, 1.0)\"},\"mapbox\":{\"style\":\"light\"},\"polar\":{\"bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"radialaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"},\"angularaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"yaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"zaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true}},\"ternary\":{\"aaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"baxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"caxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"bgcolor\":\"rgba(229, 236, 246, 1.0)\"},\"xaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"yaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"shapedefaults\":{\"line\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}},\"colorway\":[\"rgba(99, 110, 250, 1.0)\",\"rgba(239, 85, 59, 1.0)\",\"rgba(0, 204, 150, 1.0)\",\"rgba(171, 99, 250, 1.0)\",\"rgba(255, 161, 90, 1.0)\",\"rgba(25, 211, 243, 1.0)\",\"rgba(255, 102, 146, 1.0)\",\"rgba(182, 232, 128, 1.0)\",\"rgba(255, 151, 255, 1.0)\",\"rgba(254, 203, 82, 1.0)\"]},\"data\":{\"bar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"error_x\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"error_y\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"carpet\":[{\"aaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"},\"baxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"}}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"pie\":[{\"automargin\":true}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgba(235, 240, 248, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}},\"header\":{\"fill\":{\"color\":\"rgba(200, 212, 227, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}}}]}}};\r\n",
              "            var config = {\"responsive\":true};\r\n",
              "            Plotly.newPlot('639a2709-aec4-4780-9fc6-969b6a0da29c', data, layout, config);\r\n",
              "});\r\n",
              "            };\r\n",
              "            if ((typeof(requirejs) !==  typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {\r\n",
              "                var script = document.createElement(\"script\");\r\n",
              "                script.setAttribute(\"src\", \"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\");\r\n",
              "                script.onload = function(){\r\n",
              "                    renderPlotly_639a2709aec447809fc6969b6a0da29c();\r\n",
              "                };\r\n",
              "                document.getElementsByTagName(\"head\")[0].appendChild(script);\r\n",
              "            }\r\n",
              "            else {\r\n",
              "                renderPlotly_639a2709aec447809fc6969b6a0da29c();\r\n",
              "            }\r\n",
              "</script>\r\n",
              "\n",
              "    \n",
              "</div>    \n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "var lastThreeWeek = df[\"load\"].Cast<float>().TakeLast(7 * 24 * 3);\n",
        "var x = Enumerable.Range(0, lastThreeWeek.Count());\n",
        "var line = Chart2D.Chart.Line<int, float, string>(x, lastThreeWeek, Name: \"load\");\n",
        "line.Display();"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create `ForecastInput` and `ForecastOutput` class"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "public class ForecastInput\n",
        "{\n",
        "    [ColumnName(\"load\")]\n",
        "    public float Load { get; set; }\n",
        "}\n",
        "\n",
        "public class ForecastOutnput\n",
        "{\n",
        "    [ColumnName(\"predict\")]\n",
        "    public float[] Predict { get; set; }\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Set up search space for SSA\n",
        "SSA(Singular Spectrum Analysis) is an algorithm for univariante-timeseries forecasting and can be consumed via [ForecastBySSA](https://docs.microsoft.com/dotnet/api/microsoft.ml.timeseriescatalog.forecastbyssa?view=ml-dotnet) in ML.Net. \n",
        "\n",
        "The following code shows how to create a search space on SSA's certain parameters. This is necessary if you want to set up a customized hyper-parameter optimization using `AutoMLExperiment`. \n",
        "\n",
        "In SSA, the parameters that have the most significant impact on training result are `windowSize`, `seriesLenth` and `rank`. So we want to set up a sweeping range for those parameters using `Range` attribute over Properties."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "public class ForecastBySsaSearchSpace\n",
        "{\n",
        "    [Range(2, 24 * 7 * 30)]\n",
        "    public int WindowSize { get; set; } = 2;\n",
        "\n",
        "    [Range(2, 24 * 7 * 30)]\n",
        "    public int SeriesLength { get; set; } = 2;\n",
        "\n",
        "    [Range(1, 24 * 7 * 30)]\n",
        "    public int Rank { get; set; } = 1;\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a custom `TrialRunner` for `AutoMLExperimet`.\n",
        "A `TrialRunner`, which implements `ITrialRunner`, takes in parameters and pipeline, trains the model, evaluates the model, and return the metric. `AutoMLExperiment` has built-in trial runners for binary, mulitclass classification and regression. But not for time-series forecasting, so it is necessary to provide our own trial runner.\n",
        "\n",
        "In the code below, we create `SSARunner` that implements `ITrialRunner`. The core function is `Run`. It will train SSA model first, then calculating rolling-update rmse by creating a timeseries prediction engine using a trained model, predicting next 1-hour Luna load, comparing it with actual value, then updating the model with actual value and so on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "using System.Threading;\n",
        "public class SSARunner : ITrialRunner\n",
        "{\n",
        "    private MLContext _context;\n",
        "    private IDataView _trainDataset;\n",
        "    private IDataView _evaluateDataset;\n",
        "    private SweepablePipeline _pipeline;\n",
        "\n",
        "    public SSARunner(MLContext context, IDataView trainDataset, IDataView evaluateDataset, SweepablePipeline pipeline)\n",
        "    {\n",
        "        this._context = context;\n",
        "        this._trainDataset = trainDataset;\n",
        "        this._evaluateDataset = evaluateDataset;\n",
        "        this._pipeline = pipeline;\n",
        "    }\n",
        "\n",
        "    public void Dispose()\n",
        "    {}\n",
        "\n",
        "    public Task<TrialResult> RunAsync(TrialSettings settings, CancellationToken ct)\n",
        "    {\n",
        "        try\n",
        "        {\n",
        "            return Task.Run(() => Run(settings));\n",
        "        }\n",
        "        catch (Exception ex) when (ct.IsCancellationRequested)\n",
        "        {\n",
        "            throw new OperationCanceledException(ex.Message, ex.InnerException);\n",
        "        }\n",
        "        catch (Exception)\n",
        "        {\n",
        "            throw;\n",
        "        }\n",
        "    }\n",
        "\n",
        "    public TrialResult Run(TrialSettings settings)\n",
        "    {\n",
        "        try\n",
        "        {\n",
        "            var trainDataset = this._trainDataset;\n",
        "            var testDataset = this._evaluateDataset;\n",
        "\n",
        "            var stopWatch = new Stopwatch();\n",
        "            stopWatch.Start();\n",
        "            var parameter = settings.Parameter[\"_pipeline_\"];\n",
        "            var pipeline = this._pipeline.BuildFromOption(this._context, parameter);\n",
        "            var model = pipeline.Fit(trainDataset);\n",
        "\n",
        "            var predictEngine = model.CreateTimeSeriesEngine<ForecastInput, ForecastOutnput>(this._context);\n",
        "\n",
        "            // check point\n",
        "            predictEngine.CheckPoint(this._context, \"origin\");\n",
        "\n",
        "            var predictedLoad1H = new List<float>();\n",
        "            var N = testDataset.GetRowCount();\n",
        "\n",
        "            // rolling update evaluate\n",
        "            foreach (var load in testDataset.GetColumn<Single>(\"load\"))\n",
        "            {\n",
        "                // firstly, get next n predict where n is horizon, in this case, it's always 1.\n",
        "                var predict = predictEngine.Predict();\n",
        "\n",
        "                predictedLoad1H.Add(predict.Predict[0]);\n",
        "\n",
        "                // update model with truth value\n",
        "                predictEngine.Predict(new ForecastInput()\n",
        "                {\n",
        "                    Load = load,\n",
        "                });\n",
        "            }\n",
        "\n",
        "            var rmse = Enumerable.Zip(testDataset.GetColumn<float>(\"load\"), predictedLoad1H)\n",
        "                                   .Select(x => Math.Pow(x.First - x.Second, 2))\n",
        "                                   .Average();\n",
        "            rmse = Math.Sqrt(rmse);\n",
        "\n",
        "            return new TrialResult()\n",
        "            {\n",
        "                Metric = rmse,\n",
        "                Model = model,\n",
        "                TrialSettings = settings,\n",
        "                DurationInMilliseconds = stopWatch.ElapsedMilliseconds,\n",
        "            };\n",
        "\n",
        "        }\n",
        "        catch (Exception)\n",
        "        {\n",
        "            return new TrialResult()\n",
        "            {\n",
        "                Metric = double.MaxValue,\n",
        "                Model = null,\n",
        "                TrialSettings = settings,\n",
        "                DurationInMilliseconds = 0,\n",
        "            };\n",
        "        }\n",
        "    }\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Split train-test dataset.\n",
        "The following code shows how to split train-test dataset. Unlike classification or regression, where we randomly sample a subset of dataset as test set. In forecasting, in order to avoid leakage, we will want to make sure we're not using future data to train our model. So we need to take first _N_ rows as training set and keep the rest as test set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "var rowCount = df.Rows.Count();\n",
        "var evaluateCount = 24*7;\n",
        "var trainDf = df.Head(rowCount -evaluateCount);\n",
        "var evaluateDf = df.Tail(evaluateCount);"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Construct training pipeline\n",
        "The following code shows how to construct a sweepable training pipeline. It first copies column `load` to `newLoad`, which doesn't have any actual meaning other than translating a single estimator into a pipeline, then followed by a sweepable estimator. That sweepable estimator takes a lambda function, which takes in `Parameter` and returns a trainable `IEstimator<ITransformer>`, and a search space. During hyper-parameter optimization, a `Parameter` will be sampled from that search space and fed into that lambda function from sweepable estimator, which returns a trainable `IEstimator<ITransformer>`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [],
      "source": [
        "var mlContext = new MLContext();\n",
        "var searchSpace = new SearchSpace<ForecastBySsaSearchSpace>();\n",
        "var pipeline = mlContext.Transforms.CopyColumns(\"newLoad\", \"load\")\n",
        "                .Append(mlContext.Auto().CreateSweepableEstimator((context, ss) =>\n",
        "                {\n",
        "                    return mlContext.Forecasting.ForecastBySsa(\"predict\", \"load\", ss.WindowSize, ss.SeriesLength, Convert.ToInt32(trainDf.Rows.Count), 1, rank: ss.Rank, variableHorizon: true);\n",
        "                }, searchSpace));"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Run Hyper-parameter optimization using AutoMLExperiment\n",
        "The following code shows how to config an `AutoMLExperiment` with `pipeline` and `SSARunner`. One thing to notice is that it's likely that the first few trials fail (return `Infinity`). This is because the parameters sampled from the search space do not satisfy the pre-requisite when creating SSA, which is probably why `windowsSize` is smaller than `rank`. But as training continues, there will be more successful running trials because the tuner can learn from the failure trial and try to propose parameter that has the potential to succeed the next time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div><h3>Best Trial</h3><p>Id: 21</p><p>Trainer: Unknown</p><p>Parameters: {\r\n",
              "  &quot;_pipeline_&quot;: {\r\n",
              "    &quot;_SCHEMA_&quot;: &quot;e0 * e1&quot;,\r\n",
              "    &quot;e0&quot;: {},\r\n",
              "    &quot;e1&quot;: {\r\n",
              "      &quot;Rank&quot;: 1,\r\n",
              "      &quot;SeriesLength&quot;: 3096,\r\n",
              "      &quot;WindowSize&quot;: 1494\r\n",
              "    }\r\n",
              "  }\r\n",
              "}</p><h3>Active Trial</h3><p>Id: 36</p><p>Trainer: Unknown</p><p>Parameters: {\r\n",
              "  &quot;_pipeline_&quot;: {\r\n",
              "    &quot;_SCHEMA_&quot;: &quot;e0 * e1&quot;,\r\n",
              "    &quot;e0&quot;: {},\r\n",
              "    &quot;e1&quot;: {\r\n",
              "      &quot;Rank&quot;: 1,\r\n",
              "      &quot;SeriesLength&quot;: 2853,\r\n",
              "      &quot;WindowSize&quot;: 1877\r\n",
              "    }\r\n",
              "  }\r\n",
              "}</p></div><div><h3>Plot Metrics over Trials</h3></div>\n",
              "<div>\n",
              "    <div id=\"afe26716-3bb9-4ee7-8e0f-8f37a0ff5962\"><!-- Plotly chart will be drawn inside this DIV --></div>\r\n",
              "<script type=\"text/javascript\">\r\n",
              "\r\n",
              "            var renderPlotly_afe267163bb94ee78e0f8f37a0ff5962 = function() {\r\n",
              "            var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-2.6.3.min'}}) || require;\r\n",
              "            fsharpPlotlyRequire(['plotly'], function(Plotly) {\r\n",
              "\r\n",
              "            var data = [{\"type\":\"scatter\",\"name\":\"Plot Metrics over Trials.\",\"mode\":\"markers\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36],\"y\":[1.7976931348623157E+308,1.7976931348623157E+308,670299.488449774,1.7976931348623157E+308,1.7976931348623157E+308,1.7976931348623157E+308,438225.5705499285,438225.5705499285,1.7976931348623157E+308,438225.5705499285,508133.0624702087,438225.5705499285,620434.8168877821,390837.69507758314,1.7976931348623157E+308,438225.5705499285,438225.5705499285,364010.8068450883,438225.5705499285,1.7976931348623157E+308,364718.8046745925,350317.4416053568,1.7976931348623157E+308,438225.5705499285,1.7976931348623157E+308,438225.5705499285,1.7976931348623157E+308,438225.5705499285,1.7976931348623157E+308,510679.65572385077,1.7976931348623157E+308,463837.86081169604,1.7976931348623157E+308,103583973815.4377,422962.78731453413,405504.30395369796,385559.8458050363],\"marker\":{},\"line\":{},\"showlegend\":false}];\r\n",
              "            var layout = {\"width\":600,\"height\":600,\"template\":{\"layout\":{\"title\":{\"x\":0.05},\"font\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"paper_bgcolor\":\"rgba(255, 255, 255, 1.0)\",\"plot_bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"autotypenumbers\":\"strict\",\"colorscale\":{\"diverging\":[[0.0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1.0,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"geo\":{\"showland\":true,\"landcolor\":\"rgba(229, 236, 246, 1.0)\",\"showlakes\":true,\"lakecolor\":\"rgba(255, 255, 255, 1.0)\",\"subunitcolor\":\"rgba(255, 255, 255, 1.0)\",\"bgcolor\":\"rgba(255, 255, 255, 1.0)\"},\"mapbox\":{\"style\":\"light\"},\"polar\":{\"bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"radialaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"},\"angularaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"yaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"zaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true}},\"ternary\":{\"aaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"baxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"caxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"bgcolor\":\"rgba(229, 236, 246, 1.0)\"},\"xaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"yaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"shapedefaults\":{\"line\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}},\"colorway\":[\"rgba(99, 110, 250, 1.0)\",\"rgba(239, 85, 59, 1.0)\",\"rgba(0, 204, 150, 1.0)\",\"rgba(171, 99, 250, 1.0)\",\"rgba(255, 161, 90, 1.0)\",\"rgba(25, 211, 243, 1.0)\",\"rgba(255, 102, 146, 1.0)\",\"rgba(182, 232, 128, 1.0)\",\"rgba(255, 151, 255, 1.0)\",\"rgba(254, 203, 82, 1.0)\"]},\"data\":{\"bar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"error_x\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"error_y\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"carpet\":[{\"aaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"},\"baxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"}}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"pie\":[{\"automargin\":true}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgba(235, 240, 248, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}},\"header\":{\"fill\":{\"color\":\"rgba(200, 212, 227, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}}}]}},\"xaxis\":{\"title\":{\"text\":\"Trial\"},\"showgrid\":false},\"yaxis\":{\"title\":{\"text\":\"Metric\"},\"showgrid\":false}};\r\n",
              "            var config = {\"responsive\":true};\r\n",
              "            Plotly.newPlot('afe26716-3bb9-4ee7-8e0f-8f37a0ff5962', data, layout, config);\r\n",
              "});\r\n",
              "            };\r\n",
              "            if ((typeof(requirejs) !==  typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {\r\n",
              "                var script = document.createElement(\"script\");\r\n",
              "                script.setAttribute(\"src\", \"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\");\r\n",
              "                script.onload = function(){\r\n",
              "                    renderPlotly_afe267163bb94ee78e0f8f37a0ff5962();\r\n",
              "                };\r\n",
              "                document.getElementsByTagName(\"head\")[0].appendChild(script);\r\n",
              "            }\r\n",
              "            else {\r\n",
              "                renderPlotly_afe267163bb94ee78e0f8f37a0ff5962();\r\n",
              "            }\r\n",
              "</script>\r\n",
              "\n",
              "    \n",
              "</div>    \n",
              "<div><h3>All Trials Table</h3></div><table id=\"table_637983345243183814\"><caption><h3 style=\"text-align: center;\">DataFrame - 37 rows </h3></caption><thead><tr><th><i>index</i></th><th>Trial</th><th>Metric</th><th>Trainer</th><th>Parameters</th></tr></thead><tbody><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">0</div></i></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">1</div></i></td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">2</div></i></td><td><div class=\"dni-plaintext\">2</div></td><td><div class=\"dni-plaintext\">670299.5</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:769,&quot;SeriesLength&quot;:410,&quot;WindowSize&quot;:43}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">3</div></i></td><td><div class=\"dni-plaintext\">3</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:653,&quot;SeriesLength&quot;:2,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">4</div></i></td><td><div class=\"dni-plaintext\">4</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:885,&quot;SeriesLength&quot;:1105,&quot;WindowSize&quot;:1145}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">5</div></i></td><td><div class=\"dni-plaintext\">5</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1500,&quot;SeriesLength&quot;:2,&quot;WindowSize&quot;:425}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">6</div></i></td><td><div class=\"dni-plaintext\">6</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:39,&quot;SeriesLength&quot;:1426,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">7</div></i></td><td><div class=\"dni-plaintext\">7</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2552,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">8</div></i></td><td><div class=\"dni-plaintext\">8</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:976,&quot;SeriesLength&quot;:299,&quot;WindowSize&quot;:1307}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">9</div></i></td><td><div class=\"dni-plaintext\">9</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:285,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">10</div></i></td><td><div class=\"dni-plaintext\">10</div></td><td><div class=\"dni-plaintext\">508133.06</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1212,&quot;SeriesLength&quot;:2566,&quot;WindowSize&quot;:1087}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">11</div></i></td><td><div class=\"dni-plaintext\">11</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:1951,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">12</div></i></td><td><div class=\"dni-plaintext\">12</div></td><td><div class=\"dni-plaintext\">620434.8</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1871,&quot;SeriesLength&quot;:899,&quot;WindowSize&quot;:468}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">13</div></i></td><td><div class=\"dni-plaintext\">13</div></td><td><div class=\"dni-plaintext\">390837.7</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1315,&quot;SeriesLength&quot;:1610,&quot;WindowSize&quot;:222}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">14</div></i></td><td><div class=\"dni-plaintext\">14</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1502,&quot;SeriesLength&quot;:198,&quot;WindowSize&quot;:1574}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">15</div></i></td><td><div class=\"dni-plaintext\">15</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1129,&quot;SeriesLength&quot;:3020,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">16</div></i></td><td><div class=\"dni-plaintext\">16</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1191,&quot;SeriesLength&quot;:97,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">17</div></i></td><td><div class=\"dni-plaintext\">17</div></td><td><div class=\"dni-plaintext\">364010.8</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1440,&quot;SeriesLength&quot;:3121,&quot;WindowSize&quot;:1469}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">18</div></i></td><td><div class=\"dni-plaintext\">18</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:93,&quot;SeriesLength&quot;:2022,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">19</div></i></td><td><div class=\"dni-plaintext\">19</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:2786,&quot;SeriesLength&quot;:4219,&quot;WindowSize&quot;:3847}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">20</div></i></td><td><div class=\"dni-plaintext\">20</div></td><td><div class=\"dni-plaintext\">364718.8</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:4385,&quot;SeriesLength&quot;:3145,&quot;WindowSize&quot;:1443}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">21</div></i></td><td><div class=\"dni-plaintext\">21</div></td><td><div class=\"dni-plaintext\">350317.44</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:3096,&quot;WindowSize&quot;:1494}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">22</div></i></td><td><div class=\"dni-plaintext\">22</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:710,&quot;SeriesLength&quot;:4204,&quot;WindowSize&quot;:5039}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">23</div></i></td><td><div class=\"dni-plaintext\">23</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:1987,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">24</div></i></td><td><div class=\"dni-plaintext\">24</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:3117,&quot;SeriesLength&quot;:5039,&quot;WindowSize&quot;:3165}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">25</div></i></td><td><div class=\"dni-plaintext\">25</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:447,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">26</div></i></td><td><div class=\"dni-plaintext\">26</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:2082,&quot;SeriesLength&quot;:5039,&quot;WindowSize&quot;:4550}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">27</div></i></td><td><div class=\"dni-plaintext\">27</div></td><td><div class=\"dni-plaintext\">438225.56</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:679,&quot;WindowSize&quot;:2}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">28</div></i></td><td><div class=\"dni-plaintext\">28</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1946,&quot;SeriesLength&quot;:1291,&quot;WindowSize&quot;:2772}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">29</div></i></td><td><div class=\"dni-plaintext\">29</div></td><td><div class=\"dni-plaintext\">510679.66</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:4900,&quot;WindowSize&quot;:214}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">30</div></i></td><td><div class=\"dni-plaintext\">30</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:616,&quot;SeriesLength&quot;:4725,&quot;WindowSize&quot;:2398}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">31</div></i></td><td><div class=\"dni-plaintext\">31</div></td><td><div class=\"dni-plaintext\">463837.88</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:1466,&quot;WindowSize&quot;:589}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">32</div></i></td><td><div class=\"dni-plaintext\">32</div></td><td><div class=\"dni-plaintext\">Infinity</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:686,&quot;SeriesLength&quot;:3869,&quot;WindowSize&quot;:2297}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">33</div></i></td><td><div class=\"dni-plaintext\">33</div></td><td><div class=\"dni-plaintext\">1.0358397E+11</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2322,&quot;WindowSize&quot;:689}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">34</div></i></td><td><div class=\"dni-plaintext\">34</div></td><td><div class=\"dni-plaintext\">422962.78</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:472,&quot;SeriesLength&quot;:3725,&quot;WindowSize&quot;:1115}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">35</div></i></td><td><div class=\"dni-plaintext\">35</div></td><td><div class=\"dni-plaintext\">405504.3</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2466,&quot;WindowSize&quot;:1872}}}</td></tr><tr style=\"display: none\"><td><i><div class=\"dni-plaintext\">36</div></i></td><td><div class=\"dni-plaintext\">36</div></td><td><div class=\"dni-plaintext\">385559.84</div></td><td>Unknown</td><td>{&quot;_pipeline_&quot;:{&quot;_SCHEMA_&quot;:&quot;e0 * e1&quot;,&quot;e0&quot;:{},&quot;e1&quot;:{&quot;Rank&quot;:1,&quot;SeriesLength&quot;:2853,&quot;WindowSize&quot;:1877}}}</td></tr></tbody><tfoot><tr><td colspan=\"5\" style=\"text-align: center;\"><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = 1; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">⏮</button><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; page = parseInt(page) + parseInt(-10); page = page &lt; 0 ? 0 : page; page = page > 1 ? 1 : page; document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = page + 1; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">⏪</button><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; page = parseInt(page) + parseInt(-1); page = page &lt; 0 ? 0 : page; page = page > 1 ? 1 : page; document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = page + 1; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">◀️</button><b style=\"margin: 2px;\">Page</b><b id=\"page_637983345243183814\" style=\"margin: 2px;\">1</b><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; page = parseInt(page) + parseInt(1); page = page &lt; 0 ? 0 : page; page = page > 1 ? 1 : page; document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = page + 1; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">▶️</button><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; page = parseInt(page) + parseInt(10); page = page &lt; 0 ? 0 : page; page = page > 1 ? 1 : page; document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = page + 1; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">⏩</button><button style=\"margin: 2px;\" onclick=\"var allRows = document.querySelectorAll(&#39;#table_637983345243183814 tbody tr:nth-child(n)&#39;); for (let i = 0; i &lt; allRows.length; i++) { allRows[i].style.display=&#39;none&#39;; } document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML = 2; var page = parseInt(document.querySelector(&#39;#page_637983345243183814&#39;).innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j &lt; 25; j++) { pageRows[j].style.display=&#39;table-row&#39;; } \">⏭️</button></td></tr></tfoot></table><script>var page = parseInt(document.querySelector('#page_637983345243183814').innerHTML) - 1; var pageRows = document.querySelectorAll(`#table_637983345243183814 tbody tr:nth-child(n + ${page * 25 + 1 })`); for (let j = 0; j < 25; j++) { pageRows[j].style.display='table-row'; } </script>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "// Configure AutoML\n",
        "var ssaTrialRunner = new SSARunner(mlContext, trainDf, evaluateDf, pipeline);\n",
        "// NotebookMonitor plots trials and show best run nicely in notebook output cell.\n",
        "var monitor = new NotebookMonitor(pipeline);\n",
        "\n",
        "var experiment = mlContext.Auto().CreateExperiment()\n",
        "                    .SetPipeline(pipeline)\n",
        "                    .SetTrainingTimeInSeconds(120)\n",
        "                    .SetTrialRunner(ssaTrialRunner)\n",
        "                    .SetRegressionMetric(RegressionMetric.RootMeanSquaredError, \"load\", \"Score\")\n",
        "\t\t\t\t\t.SetMonitor(monitor);\n",
        "\n",
        "// Configure Visualizer\t\t\t\n",
        "monitor.SetUpdate(monitor.Display());\n",
        "\n",
        "// Start Experiment\n",
        "var res = await experiment.RunAsync();"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Evaluate model using test dataset\n",
        "The following code shows how to use the best model produced by hyper-parameter optimization from `AutoMLExperiment` to predict the Luna `load` for next two weeks, then compare it with the actual `load` and calculate rmse metric. The evaluating way should keep the same with `SSARunner` so we need to calculate rolling-update rmse as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div class=\"dni-plaintext\">468446.7398029364</div>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "var model = res.Model;\n",
        "// evaluate\n",
        "var predictEngine = model.CreateTimeSeriesEngine<ForecastInput, ForecastOutnput>(mlContext);\n",
        "\n",
        "var predictLoads1H = new List<float>();\n",
        "foreach (var load in evaluateDf.GetColumn<Single>(\"load\"))\n",
        "{\n",
        "    // firstly, get next n predict where n is horizon\n",
        "    var predict = predictEngine.Predict();\n",
        "\n",
        "    predictLoads1H.Add(predict.Predict[0]);\n",
        "\n",
        "    // update model with truth value\n",
        "    predictEngine.Predict(new ForecastInput()\n",
        "    {\n",
        "        Load = load,\n",
        "    });\n",
        "}\n",
        "\n",
        "evaluateDf[\"predict_load_1h\"] = DataFrameColumn.Create(\"predict_load_1h\", predictLoads1H);\n",
        "\n",
        "var mse = (evaluateDf[\"load\"] - evaluateDf[\"predict_load_1h\"]).Cast<float>().Select(x => x * x).Average();\n",
        "var rmse = Math.Sqrt(mse);\n",
        "rmse"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Plot both predicted and truth sonar in test dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        },
        "vscode": {
          "languageId": "dotnet-interactive.csharp"
        }
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "<div>\n",
              "    <div id=\"0749b298-f48b-4550-9a23-bdeb578f2107\"><!-- Plotly chart will be drawn inside this DIV --></div>\r\n",
              "<script type=\"text/javascript\">\r\n",
              "\r\n",
              "            var renderPlotly_0749b298f48b45509a23bdeb578f2107 = function() {\r\n",
              "            var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-2.6.3.min'}}) || require;\r\n",
              "            fsharpPlotlyRequire(['plotly'], function(Plotly) {\r\n",
              "\r\n",
              "            var data = [{\"type\":\"scatter\",\"name\":\"predict_load_1h\",\"mode\":\"lines\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167],\"y\":[1277592.1,1258415.1,1215353.0,1057881.0,865480.6,780794.3,842908.7,941389.5,938986.44,835865.4,799732.2,1015871.9,1493781.4,2027416.1,2353968.8,2365872.5,2189283.0,2044763.9,2035892.9,2068581.5,1970479.2,1684303.4,1342524.9,1147427.5,1185751.9,1347458.0,1441807.9,1378438.9,1236780.8,1157139.1,1174565.1,1174234.8,1014675.4,697492.25,398139.7,314880.94,474969.03,691898.06,725249.5,496536.66,155780.44,-59799.438,-43289.67,101538.625,182330.81,117338.84,13079.947,47233.375,282052.9,592359.25,782583.9,769679.6,649352.6,582651.0,624263.1,674034.44,600154.7,406007.7,260703.88,348187.44,676191.56,1038860.44,1180695.5,1015585.44,693826.2,458242.28,425387.62,502948.03,510221.25,376650.16,216629.2,211465.67,422672.5,723302.7,918140.94,928253.1,857002.44,875733.56,1050384.9,1289815.5,1467551.5,1585396.8,1802244.5,2281203.2,3002691.0,3730979.5,4180153.2,4234839.0,4023696.5,3775780.0,3608397.8,3443060.5,3123210.2,2610196.5,2051091.0,1654401.0,1507774.0,1500656.2,1446612.8,1268140.1,1063668.5,994432.9,1114114.6,1320925.8,1481112.1,1593084.0,1813954.9,2306239.0,3048290.0,3801254.2,4277512.5,4357051.0,4163282.2,3919628.2,3738308.5,3538291.5,3165343.5,2586489.0,1956780.6,1492982.6,1289682.0,1240997.1,1162879.9,978371.25,784528.5,736689.9,880151.0,1105587.8,1273650.1,1381586.9,1588251.0,2059631.9,2778739.0,3511857.2,3977741.8,4063260.5,3895495.8,3697901.5,3577511.2,3447239.2,3147399.5,2639624.0,2074759.1,1664934.1,1501024.6,1475535.6,1404001.8,1211333.6,997618.75,922305.0,1034485.4,1227988.9,1369305.0,1461543.2,1668566.9,2158087.0,2910330.5,3685249.8,4193494.5,4315648.0,4175719.2,3991587.0,3867628.8,3713393.5,3370368.5,2805374.0,2177922.2,1708609.9],\"marker\":{},\"line\":{}},{\"type\":\"scatter\",\"name\":\"truth\",\"mode\":\"lines\",\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167],\"y\":[2140878.0,1895147.0,1579756.0,1402062.0,1322785.0,1165658.0,1131081.0,1244647.0,1313523.0,1327327.0,1506570.0,1799622.0,2468453.0,3322579.0,3490262.0,3537042.0,3357287.0,3054315.0,2878427.0,2817847.0,2560142.0,2086403.0,1311481.0,1131604.0,835595.0,706414.0,564374.0,501562.0,460359.0,407497.0,524727.0,513550.0,498451.0,471818.0,568307.0,736493.0,880628.0,980110.0,1056655.0,960999.0,944985.0,792694.0,730271.0,704140.0,667614.0,615589.0,464296.0,476380.0,456389.0,435793.0,371625.0,335967.0,349326.0,320461.0,360233.0,401795.0,400516.0,397158.0,480181.0,593995.0,728192.0,791722.0,842580.0,787016.0,789116.0,720734.0,727569.0,693815.0,700541.0,675098.0,602570.0,654610.0,700733.0,754041.0,716777.0,641627.0,647768.0,720232.0,826600.0,955412.0,989776.0,1085038.0,1248893.0,1755543.0,2648495.0,3460453.0,3797630.0,3866752.0,3578963.0,3419092.0,3371077.0,3133834.0,2904028.0,2427113.0,1536951.0,1431362.0,1240912.0,1152746.0,1017037.0,879557.0,794088.0,776213.0,866713.0,994684.0,1053025.0,1082314.0,1272833.0,1740110.0,2559427.0,3543817.0,3795470.0,3786953.0,3478702.0,3325876.0,3290135.0,3147765.0,2918741.0,2411975.0,1507603.0,1415209.0,1207824.0,1049837.0,891704.0,899598.0,859797.0,852082.0,1017008.0,1110451.0,1186555.0,1212441.0,1413617.0,1954505.0,2815706.0,3879214.0,4287130.0,4297613.0,4004459.0,3668727.0,3560500.0,3427783.0,3095399.0,2544989.0,1552230.0,1468952.0,1401150.0,1246200.0,1018963.0,893128.0,836865.0,838962.0,888329.0,1035122.0,1092601.0,1138552.0,1369574.0,1856648.0,2527278.0,3220888.0,3388005.0,3576955.0,3282861.0,3294191.0,3219435.0,3105795.0,2722333.0,2231878.0,1382403.0,1330587.0],\"marker\":{},\"line\":{}}];\r\n",
              "            var layout = {\"width\":600,\"height\":600,\"template\":{\"layout\":{\"title\":{\"x\":0.05},\"font\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"paper_bgcolor\":\"rgba(255, 255, 255, 1.0)\",\"plot_bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"autotypenumbers\":\"strict\",\"colorscale\":{\"diverging\":[[0.0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1.0,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"geo\":{\"showland\":true,\"landcolor\":\"rgba(229, 236, 246, 1.0)\",\"showlakes\":true,\"lakecolor\":\"rgba(255, 255, 255, 1.0)\",\"subunitcolor\":\"rgba(255, 255, 255, 1.0)\",\"bgcolor\":\"rgba(255, 255, 255, 1.0)\"},\"mapbox\":{\"style\":\"light\"},\"polar\":{\"bgcolor\":\"rgba(229, 236, 246, 1.0)\",\"radialaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"},\"angularaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"yaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true},\"zaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"gridwidth\":2.0,\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"backgroundcolor\":\"rgba(229, 236, 246, 1.0)\",\"showbackground\":true}},\"ternary\":{\"aaxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"baxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"caxis\":{\"ticks\":\"\",\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\"},\"bgcolor\":\"rgba(229, 236, 246, 1.0)\"},\"xaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"yaxis\":{\"title\":{\"standoff\":15},\"ticks\":\"\",\"automargin\":true,\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinecolor\":\"rgba(255, 255, 255, 1.0)\",\"zerolinewidth\":2.0},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"shapedefaults\":{\"line\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}},\"colorway\":[\"rgba(99, 110, 250, 1.0)\",\"rgba(239, 85, 59, 1.0)\",\"rgba(0, 204, 150, 1.0)\",\"rgba(171, 99, 250, 1.0)\",\"rgba(255, 161, 90, 1.0)\",\"rgba(25, 211, 243, 1.0)\",\"rgba(255, 102, 146, 1.0)\",\"rgba(182, 232, 128, 1.0)\",\"rgba(255, 151, 255, 1.0)\",\"rgba(254, 203, 82, 1.0)\"]},\"data\":{\"bar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"error_x\":{\"color\":\"rgba(42, 63, 95, 1.0)\"},\"error_y\":{\"color\":\"rgba(42, 63, 95, 1.0)\"}}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"rgba(229, 236, 246, 1.0)\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"carpet\":[{\"aaxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"},\"baxis\":{\"linecolor\":\"rgba(255, 255, 255, 1.0)\",\"gridcolor\":\"rgba(255, 255, 255, 1.0)\",\"endlinecolor\":\"rgba(42, 63, 95, 1.0)\",\"minorgridcolor\":\"rgba(255, 255, 255, 1.0)\",\"startlinecolor\":\"rgba(42, 63, 95, 1.0)\"}}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}}}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"pie\":[{\"automargin\":true}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}},\"line\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"}}}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0.0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgba(235, 240, 248, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}},\"header\":{\"fill\":{\"color\":\"rgba(200, 212, 227, 1.0)\"},\"line\":{\"color\":\"rgba(255, 255, 255, 1.0)\"}}}]}}};\r\n",
              "            var config = {\"responsive\":true};\r\n",
              "            Plotly.newPlot('0749b298-f48b-4550-9a23-bdeb578f2107', data, layout, config);\r\n",
              "});\r\n",
              "            };\r\n",
              "            if ((typeof(requirejs) !==  typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {\r\n",
              "                var script = document.createElement(\"script\");\r\n",
              "                script.setAttribute(\"src\", \"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\");\r\n",
              "                script.onload = function(){\r\n",
              "                    renderPlotly_0749b298f48b45509a23bdeb578f2107();\r\n",
              "                };\r\n",
              "                document.getElementsByTagName(\"head\")[0].appendChild(script);\r\n",
              "            }\r\n",
              "            else {\r\n",
              "                renderPlotly_0749b298f48b45509a23bdeb578f2107();\r\n",
              "            }\r\n",
              "</script>\r\n",
              "\n",
              "    \n",
              "</div>    \n"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "var predicted = evaluateDf[\"predict_load_1h\"].Cast<float>();\n",
        "var truth = evaluateDf[\"load\"].Cast<float>();\n",
        "var X = Enumerable.Range(0, truth.Count());\n",
        "var predictedChart = Chart2D.Chart.Line<int, float, string>(X, predicted, Name: \"predict_load_1h\");\n",
        "var truthChart = Chart2D.Chart.Line<int, float, string>(X, truth, Name: \"truth\");\n",
        "var combineChart = Chart.Combine(new[]{ predictedChart, truthChart});\n",
        "combineChart.Display()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": ".NET (C#)",
      "language": "C#",
      "name": ".net-csharp"
    },
    "language_info": {
      "file_extension": ".cs",
      "mimetype": "text/x-csharp",
      "name": "C#",
      "pygments_lexer": "csharp",
      "version": "8.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}