1. [Design Phase - 1]  Algorithm Selection.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c8bb1e6d",
   "metadata": {},
   "source": [
    "## ** The Improved Power Disaggregation (Non-intrusive Load Monitoring) **\n",
    "\n",
    "- Author : Gyubaek Kim (database.kim@yonsei.ac.kr)\n",
    "- Description\n",
    "  * Various algorithms have been utilized by disaggregation systems. The existing methods adopted neural network-based algorithms. The neural \n",
    "  network-based algorithms performed better than combinational optimization and hidden Markov models. Thus, we compared the \n",
    "  performance of the neural network-based algorithms and decision tree-based ensemble algorithms, such as random forest and gradient \n",
    "  boosting machine (GBM).\n",
    "- License\n",
    "   * All rights reserved. No part of this code may be reproduced or transmitted in any form or by any means, or utilized by any information storage and retrieval system without written permission from the copyright owner. You can use this source code for free projects only. I will not allow commercial use of this code. I am not allowing anyone to modify any or all parts of the source code for commercial purposes. By using my source code, you agree to the following: \n",
    "       * 1. You will not distribute any or all parts of this source code for commercial use \n",
    "       * 2. You will cite me as the original creator of this source code. \n",
    "       * 3. You will inform me of its use before you use any or all parts of this source code. \n",
    "   For use of any or all parts of this source code, email me. This source code is provided free to use for everybody provided this is used non-commercially. Violators of this agreement will be subject to legal action by the author. If you see commercial applications or software using any or all parts of this source code, inform me.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4395c65b",
   "metadata": {},
   "source": [
    "#### import packages of benchmark project (contrib. project) to be compared\n",
    "\n",
    "- https://github.com/OdysseasKr/neural-disaggregator\n",
    "- The project implemented 5 neural network-based NILM models\n",
    "  * RNN (Recurrent Neural Network)\n",
    "  * GRU (Gated Recurrent Unit)\n",
    "  * Window GRU\n",
    "  * DAE (Denosing Autoencoder)\n",
    "  * S2P (Short Sequence to Point)\n",
    "  \n",
    "- https://github.com/rgtzths/ICMLC_ICWAPR_code_base\n",
    "- The project implemented ResNet-based NILM model\n",
    "  * ResNet (Residual Network)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e9853e49",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/kgbmanse/anaconda3/envs/nilm/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "#### NIMLTK as common framework ####\n",
    "from nilmtk.utils import find_nearest\n",
    "from nilmtk.feature_detectors import cluster\n",
    "from nilmtk.legacy.disaggregate import Disaggregator\n",
    "from nilmtk.datastore import HDFDataStore\n",
    "from nilmtk import DataSet\n",
    "\n",
    "#### Benchmark Resource ####\n",
    "from benchmark.rnndisaggregator import RNNDisaggregator\n",
    "from benchmark.grudisaggregator import GRUDisaggregator\n",
    "from benchmark.windowgrudisaggregator import WindowGRUDisaggregator\n",
    "from benchmark.daedisaggregator import DAEDisaggregator\n",
    "from benchmark.shortseq2pointdisaggregator import ShortSeq2PointDisaggregator\n",
    "from benchmark.metrics import recall_precision_accuracy_f1, relative_error_total_energy, mean_absolute_error\n",
    "\n",
    "#### Additional Benchmark Resource ####\n",
    "from benchmark.resnet import RESNETDisaggregator\n",
    "\n",
    "from work.benchmark.rnndisaggregator_fe import RNNDisaggregatorFE\n",
    "from work.benchmark.grudisaggregator_fe import GRUDisaggregatorFE\n",
    "from work.benchmark.windowgrudisaggregator_fe import WindowGRUDisaggregatorFE\n",
    "from work.benchmark.daedisaggregator_fe import DAEDisaggregatorFE\n",
    "from work.benchmark.shortseq2pointdisaggregator_fe import ShortSeq2PointDisaggregatorFE\n",
    "\n",
    "#### Our Model ####\n",
    "from work.dtdisaggregator import DTDisaggregator\n",
    "from work.utils import checkSimilarity\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import h5py\n",
    "import numpy.random\n",
    "import time\n",
    "import datetime\n",
    "import math\n",
    "import itertools\n",
    "import warnings\n",
    "\n",
    "from scipy.signal import find_peaks\n",
    "\n",
    "from matplotlib import rcParams\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "numpy.random.seed(42)\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "%matplotlib inline\n",
    "rcParams['figure.figsize'] = (23, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1ddf2ff",
   "metadata": {},
   "source": [
    "#### Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0cab9f30",
   "metadata": {},
   "outputs": [],
   "source": [
    "###############################################################\n",
    "# Common\n",
    "###############################################################\n",
    "TRAIN_SAMPLE_PERIOD = 1\n",
    "TEST_SAMPLE_PERIOD = 1\n",
    "\n",
    "EPOCHS = 1\n",
    "\n",
    "DATA_FILE = 'data/enertalk.h5'\n",
    "\n",
    "T_S = '2017-01-01'\n",
    "T_E = '2017-01-20'\n",
    "P_S = '2017-01-21'\n",
    "P_E = '2017-01-31'\n",
    "\n",
    "HOME_ID = 1\n",
    "appliances = ['television', 'washing machine', 'rice cooker', 'water filter', 'microwave']\n",
    "#appliances = ['rice cooker', 'water filter', 'microwave']\n",
    "#appliances = ['rice cooker']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "68d68c46",
   "metadata": {},
   "outputs": [],
   "source": [
    "# start_time, end_time : YYYY-MM-DD HH:MI:SS\n",
    "def disaggregate(dataset, model, appliance, start_time, end_time, evaluate=False):\n",
    "    \n",
    "    dataset.set_window(start=start_time, end=end_time)\n",
    "    \n",
    "    test_elec = dataset.buildings[HOME_ID].elec    \n",
    "    test_mains = test_elec.all_meters()[0]   \n",
    "    test_meter = test_elec.submeters()[appliance]\n",
    "    \n",
    "    tempfile = 'work/temp/' + str(time.time())\n",
    "    \n",
    "    output = HDFDataStore(tempfile, 'w')    \n",
    "    model.disaggregate(test_mains, output, test_meter, sample_period=TEST_SAMPLE_PERIOD)\n",
    "    output.close()\n",
    "    \n",
    "    result = DataSet(tempfile)    \n",
    "    res_elec = result.buildings[HOME_ID].elec\n",
    "\n",
    "    predicted = res_elec[appliance] \n",
    "    \n",
    "    ground_truth = None\n",
    "   \n",
    "    if evaluate == True:\n",
    "        \n",
    "        ground_truth = test_elec[appliance]\n",
    "\n",
    "        # Performance Metric\n",
    "        print(\"============ Relative error in total energy: {}\".format(relative_error_total_energy(predicted, ground_truth)))\n",
    "        print(\"============ Mean absolute error(in Watts): {}\".format(mean_absolute_error(predicted, ground_truth)))\n",
    "        \n",
    "        # Performance Metric\n",
    "        rpaf = recall_precision_accuracy_f1(predicted, ground_truth)\n",
    "        print(\"============ Recall: {}\".format(rpaf[0]))\n",
    "        print(\"============ Precision: {}\".format(rpaf[1]))\n",
    "        print(\"============ Accuracy: {}\".format(rpaf[2]))\n",
    "        print(\"============ F1 Score: {}\".format(rpaf[3]))\n",
    "        \n",
    "    return predicted"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f4dda1e",
   "metadata": {},
   "source": [
    "## Alogorithm Comparison\n",
    "\n",
    "- Description\n",
    "  In the first experiemnt, only total electricity consumption was used as training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "011adeab",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-07-01 13:25:14.617487: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory\n",
      "2022-07-01 13:25:14.619178: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory\n",
      "2022-07-01 13:25:14.619232: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1835] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n",
      "Skipping registering GPU devices...\n",
      "2022-07-01 13:25:14.620473: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_3 (Conv1D)            (None, 50, 16)            80        \n",
      "_________________________________________________________________\n",
      "bidirectional_4 (Bidirection (None, 50, 128)           31488     \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 50, 128)           0         \n",
      "_________________________________________________________________\n",
      "bidirectional_5 (Bidirection (None, 256)               198144    \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 262,737\n",
      "Trainable params: 262,737\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Model: \"sequential_4\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_6 (Conv1D)            (None, 100, 30)           330       \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_7 (Conv1D)            (None, 100, 30)           7230      \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_8 (Conv1D)            (None, 100, 40)           7240      \n",
      "_________________________________________________________________\n",
      "dropout_9 (Dropout)          (None, 100, 40)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_9 (Conv1D)            (None, 100, 50)           10050     \n",
      "_________________________________________________________________\n",
      "dropout_10 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_10 (Conv1D)           (None, 100, 50)           12550     \n",
      "_________________________________________________________________\n",
      "dropout_11 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 5000)              0         \n",
      "_________________________________________________________________\n",
      "dense_9 (Dense)              (None, 1024)              5121024   \n",
      "_________________________________________________________________\n",
      "dropout_12 (Dropout)         (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 1)                 1025      \n",
      "=================================================================\n",
      "Total params: 5,159,449\n",
      "Trainable params: 5,159,449\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "********************\n",
      "[GRU-television]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-07-01 13:26:10.049959: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12571/12571 [==============================] - 161s 12ms/step - loss: 8.3385e-05\n",
      "elapsed time (secs) = 213.24771451950073\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.1760326745768233\n",
      "============ Mean absolute error(in Watts): 31.43260061551896\n",
      "============ Recall: 0.946935706318179\n",
      "============ Precision: 0.42402476681374623\n",
      "============ Accuracy: 0.5647593867149364\n",
      "============ F1 Score: 0.585756044653722\n",
      "********************\n",
      "[RNN-television]\n",
      "********************\n",
      "6750/6750 [==============================] - 112s 16ms/step - loss: 1.1177e-04\n",
      "elapsed time (secs) = 134.64802622795105\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.04027350796761409\n",
      "============ Mean absolute error(in Watts): 32.16830288464859\n",
      "============ Recall: 0.9770316069322138\n",
      "============ Precision: 0.38387371392406483\n",
      "============ Accuracy: 0.4829359302636653\n",
      "============ F1 Score: 0.5511871337798593\n",
      "********************\n",
      "[WGRU-television]\n",
      "********************\n",
      "6750/6750 [==============================] - 1307s 193ms/step - loss: 1.0927e-04\n",
      "elapsed time (secs) = 1330.5954525470734\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.08209278881793423\n",
      "============ Mean absolute error(in Watts): 31.990118239361134\n",
      "============ Recall: 0.9892986364851617\n",
      "============ Precision: 0.3608830042743212\n",
      "============ Accuracy: 0.42723210252347293\n",
      "============ F1 Score: 0.5288489389596926\n",
      "********************\n",
      "[DAE-television]\n",
      "********************\n",
      "211/211 [==============================] - 4s 19ms/step - loss: 1.3928e-04\n",
      "elapsed time (secs) = 27.21939778327942\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.19855818933661376\n",
      "============ Mean absolute error(in Watts): 39.107744070195785\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.3249671967304434\n",
      "============ Accuracy: 0.3249671967304434\n",
      "============ F1 Score: 0.49052866747546503\n",
      "********************\n",
      "[S2P-television]\n",
      "********************\n",
      "6750/6750 [==============================] - 323s 48ms/step - loss: 1.1632e-04\n",
      "elapsed time (secs) = 347.23751950263977\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.09940473031771728\n",
      "============ Mean absolute error(in Watts): 36.04638439244428\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.3249307216074441\n",
      "============ Accuracy: 0.3249307216074441\n",
      "============ F1 Score: 0.49048711198005696\n",
      "********************\n",
      "[DRF-television]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100\n",
      "building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100\n",
      "building tree 5 of 100\n",
      "building tree 6 of 100\n",
      "building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "building tree 9 of 100\n",
      "building tree 10 of 100\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100\n",
      "building tree 13 of 100\n",
      "building tree 14 of 100\n",
      "building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100\n",
      "building tree 19 of 100\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100\n",
      "building tree 24 of 100\n",
      "building tree 25 of 100\n",
      "building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "building tree 28 of 100\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100\n",
      "building tree 31 of 100\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100\n",
      "building tree 34 of 100\n",
      "building tree 35 of 100\n",
      "building tree 36 of 100\n",
      "building tree 37 of 100\n",
      "building tree 38 of 100\n",
      "building tree 39 of 100\n",
      "building tree 40 of 100\n",
      "building tree 41 of 100\n",
      "building tree 42 of 100\n",
      "building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "building tree 45 of 100\n",
      "building tree 46 of 100\n",
      "building tree 47 of 100\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100\n",
      "building tree 50 of 100\n",
      "building tree 51 of 100\n",
      "building tree 52 of 100\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100\n",
      "building tree 59 of 100\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100\n",
      "building tree 78 of 100\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:   13.0s remaining:    1.8s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:   14.8s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train on chunk is done..\n",
      "elapsed time (secs) = 37.885735273361206\n",
      "New sensible chunk: 357621\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.9s remaining:    0.3s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    2.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 173735\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    0.7s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    0.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 332127\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.1s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.01036903614583705\n",
      "============ Mean absolute error(in Watts): 15.854856724882369\n",
      "============ Recall: 0.9844762885642705\n",
      "============ Precision: 0.5787822916884913\n",
      "============ Accuracy: 0.7621267868082795\n",
      "============ F1 Score: 0.7289868094836607\n",
      "********************\n",
      "[GBM-television]\n",
      "********************\n",
      "train on chunk is done..\n",
      "elapsed time (secs) = 70.82628417015076\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.009987066062628988\n",
      "============ Mean absolute error(in Watts): 30.75948593107578\n",
      "============ Recall: 0.9557773794293005\n",
      "============ Precision: 0.4160033008010027\n",
      "============ Accuracy: 0.549605607998323\n",
      "============ F1 Score: 0.5796940435212554\n",
      "********************\n",
      "[RESNET-television]\n",
      "********************\n",
      "16/16 [==============================] - 39s 2s/step - loss: 1.3174 - mean_absolute_error: 0.5637 - root_mean_squared_error: 1.1478\n",
      "elapsed time (secs) = 62.54013395309448\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.9931492302377326\n",
      "============ Mean absolute error(in Watts): 29.27451280889788\n",
      "============ Recall: 0.013627794428427351\n",
      "============ Precision: 0.8533809417540728\n",
      "============ Accuracy: 0.6787005157032986\n",
      "============ F1 Score: 0.026827180760758233\n",
      "Model: \"sequential_7\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_14 (Conv1D)           (None, 50, 16)            80        \n",
      "_________________________________________________________________\n",
      "bidirectional_10 (Bidirectio (None, 50, 128)           31488     \n",
      "_________________________________________________________________\n",
      "dropout_13 (Dropout)         (None, 50, 128)           0         \n",
      "_________________________________________________________________\n",
      "bidirectional_11 (Bidirectio (None, 256)               198144    \n",
      "_________________________________________________________________\n",
      "dropout_14 (Dropout)         (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_16 (Dense)             (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_15 (Dropout)         (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_17 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 262,737\n",
      "Trainable params: 262,737\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Model: \"sequential_9\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_17 (Conv1D)           (None, 100, 30)           330       \n",
      "_________________________________________________________________\n",
      "dropout_20 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_18 (Conv1D)           (None, 100, 30)           7230      \n",
      "_________________________________________________________________\n",
      "dropout_21 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_19 (Conv1D)           (None, 100, 40)           7240      \n",
      "_________________________________________________________________\n",
      "dropout_22 (Dropout)         (None, 100, 40)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_20 (Conv1D)           (None, 100, 50)           10050     \n",
      "_________________________________________________________________\n",
      "dropout_23 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_21 (Conv1D)           (None, 100, 50)           12550     \n",
      "_________________________________________________________________\n",
      "dropout_24 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 5000)              0         \n",
      "_________________________________________________________________\n",
      "dense_21 (Dense)             (None, 1024)              5121024   \n",
      "_________________________________________________________________\n",
      "dropout_25 (Dropout)         (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_22 (Dense)             (None, 1)                 1025      \n",
      "=================================================================\n",
      "Total params: 5,159,449\n",
      "Trainable params: 5,159,449\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "********************\n",
      "[GRU-washing machine]\n",
      "********************\n",
      "6750/6750 [==============================] - 71s 10ms/step - loss: 6.4628e-04\n",
      "elapsed time (secs) = 97.75429320335388\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.07168307668483481\n",
      "============ Mean absolute error(in Watts): 9.84999038614883\n",
      "============ Recall: 0.22204272848431\n",
      "============ Precision: 0.7731139703297476\n",
      "============ Accuracy: 0.9516457223725826\n",
      "============ F1 Score: 0.3449996078123774\n",
      "********************\n",
      "[RNN-washing machine]\n",
      "********************\n",
      "6750/6750 [==============================] - 110s 16ms/step - loss: 7.6636e-04\n",
      "elapsed time (secs) = 133.50603437423706\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.07501496095126531\n",
      "============ Mean absolute error(in Watts): 14.693533844502223\n",
      "============ Recall: 0.3245022414280522\n",
      "============ Precision: 0.309890660855815\n",
      "============ Accuracy: 0.9198141012946435\n",
      "============ F1 Score: 0.31702818137878647\n",
      "********************\n",
      "[WGRU-washing machine]\n",
      "********************\n",
      "6750/6750 [==============================] - 1293s 191ms/step - loss: 0.0012\n",
      "elapsed time (secs) = 1316.7574932575226\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.18348767184489903\n",
      "============ Mean absolute error(in Watts): 23.646718493183105\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.05735902906638682\n",
      "============ Accuracy: 0.05735902906638682\n",
      "============ F1 Score: 0.10849489622655978\n",
      "********************\n",
      "[DAE-washing machine]\n",
      "********************\n",
      "211/211 [==============================] - 4s 17ms/step - loss: 0.0029\n",
      "elapsed time (secs) = 26.16872811317444\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.2526411538673509\n",
      "============ Mean absolute error(in Watts): 40.17433698704768\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.05735158040535924\n",
      "============ Accuracy: 0.05735158040535924\n",
      "============ F1 Score: 0.10848157125442084\n",
      "********************\n",
      "[S2P-washing machine]\n",
      "********************\n",
      "6750/6750 [==============================] - 260s 38ms/step - loss: 8.1226e-04\n",
      "elapsed time (secs) = 284.2447361946106\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.9986292945935001\n",
      "============ Mean absolute error(in Watts): 23.991351090854288\n",
      "============ Recall: 0.0005250403877221325\n",
      "============ Precision: 1.0\n",
      "============ Accuracy: 0.9426611243952622\n",
      "============ F1 Score: 0.0010495297299479272\n",
      "********************\n",
      "[DRF-washing machine]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100\n",
      "building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100\n",
      "building tree 5 of 100\n",
      "building tree 6 of 100\n",
      "building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "building tree 9 of 100\n",
      "building tree 10 of 100\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100\n",
      "building tree 13 of 100\n",
      "building tree 14 of 100\n",
      "building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100\n",
      "building tree 19 of 100\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100\n",
      "building tree 24 of 100\n",
      "building tree 25 of 100\n",
      "building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "building tree 28 of 100\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100building tree 31 of 100\n",
      "\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100\n",
      "building tree 34 of 100\n",
      "building tree 35 of 100\n",
      "building tree 36 of 100\n",
      "building tree 37 of 100\n",
      "building tree 38 of 100\n",
      "building tree 39 of 100\n",
      "building tree 40 of 100\n",
      "building tree 41 of 100\n",
      "building tree 42 of 100\n",
      "building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "building tree 45 of 100\n",
      "building tree 46 of 100\n",
      "building tree 47 of 100\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100\n",
      "building tree 50 of 100\n",
      "building tree 51 of 100building tree 52 of 100\n",
      "\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100\n",
      "building tree 59 of 100\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100building tree 78 of 100\n",
      "\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:   14.3s remaining:    1.9s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:   15.9s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train on chunk is done..\n",
      "elapsed time (secs) = 38.418766498565674\n",
      "New sensible chunk: 357621\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    2.1s remaining:    0.3s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    2.3s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 173735\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    0.6s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    0.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 332127\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.1s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.021793342880014454\n",
      "============ Mean absolute error(in Watts): 4.792342843278862\n",
      "============ Recall: 0.7483542667905173\n",
      "============ Precision: 0.5904846881871196\n",
      "============ Accuracy: 0.9558021543033373\n",
      "============ F1 Score: 0.6601118592141356\n",
      "********************\n",
      "[GBM-washing machine]\n",
      "********************\n",
      "train on chunk is done..\n",
      "elapsed time (secs) = 76.67428827285767\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.020196308622214866\n",
      "============ Mean absolute error(in Watts): 11.163201657398272\n",
      "============ Recall: 0.2715762691329106\n",
      "============ Precision: 0.45124815461011947\n",
      "============ Accuracy: 0.9392829720630795\n",
      "============ F1 Score: 0.3390817638605249\n",
      "********************\n",
      "[RESNET-washing machine]\n",
      "********************\n",
      "16/16 [==============================] - 38s 2s/step - loss: 1.1714 - mean_absolute_error: 0.5082 - root_mean_squared_error: 1.0823\n",
      "elapsed time (secs) = 61.79400610923767\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.8257616746390778\n",
      "============ Mean absolute error(in Watts): 157.76343504639325\n",
      "============ Recall: 0.900327127337345\n",
      "============ Precision: 0.051966609710783776\n",
      "============ Accuracy: 0.05229646048957649\n",
      "============ F1 Score: 0.0982615901337306\n",
      "Model: \"sequential_12\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_25 (Conv1D)           (None, 50, 16)            80        \n",
      "_________________________________________________________________\n",
      "bidirectional_16 (Bidirectio (None, 50, 128)           31488     \n",
      "_________________________________________________________________\n",
      "dropout_26 (Dropout)         (None, 50, 128)           0         \n",
      "_________________________________________________________________\n",
      "bidirectional_17 (Bidirectio (None, 256)               198144    \n",
      "_________________________________________________________________\n",
      "dropout_27 (Dropout)         (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_28 (Dense)             (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_28 (Dropout)         (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_29 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 262,737\n",
      "Trainable params: 262,737\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Model: \"sequential_14\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_28 (Conv1D)           (None, 100, 30)           330       \n",
      "_________________________________________________________________\n",
      "dropout_33 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_29 (Conv1D)           (None, 100, 30)           7230      \n",
      "_________________________________________________________________\n",
      "dropout_34 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_30 (Conv1D)           (None, 100, 40)           7240      \n",
      "_________________________________________________________________\n",
      "dropout_35 (Dropout)         (None, 100, 40)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_31 (Conv1D)           (None, 100, 50)           10050     \n",
      "_________________________________________________________________\n",
      "dropout_36 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_32 (Conv1D)           (None, 100, 50)           12550     \n",
      "_________________________________________________________________\n",
      "dropout_37 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "flatten_5 (Flatten)          (None, 5000)              0         \n",
      "_________________________________________________________________\n",
      "dense_33 (Dense)             (None, 1024)              5121024   \n",
      "_________________________________________________________________\n",
      "dropout_38 (Dropout)         (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_34 (Dense)             (None, 1)                 1025      \n",
      "=================================================================\n",
      "Total params: 5,159,449\n",
      "Trainable params: 5,159,449\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "********************\n",
      "[GRU-rice cooker]\n",
      "********************\n",
      "6750/6750 [==============================] - 71s 10ms/step - loss: 3.4083e-04\n",
      "elapsed time (secs) = 95.42207193374634\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.10945205758438888\n",
      "============ Mean absolute error(in Watts): 9.64520756505563\n",
      "============ Recall: 0.31238250231487974\n",
      "============ Precision: 0.44598005047470257\n",
      "============ Accuracy: 0.9556029605746971\n",
      "============ F1 Score: 0.36741361671231976\n",
      "********************\n",
      "[RNN-rice cooker]\n",
      "********************\n",
      "6750/6750 [==============================] - 111s 16ms/step - loss: 6.3389e-04\n",
      "elapsed time (secs) = 135.74306392669678\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.24196489969351384\n",
      "============ Mean absolute error(in Watts): 22.907414074624764\n",
      "============ Recall: 0.7823171244984427\n",
      "============ Precision: 0.09673110549834855\n",
      "============ Accuracy: 0.6895021430697374\n",
      "============ F1 Score: 0.17217348882734176\n",
      "********************\n",
      "[WGRU-rice cooker]\n",
      "********************\n",
      "6750/6750 [==============================] - 1272s 188ms/step - loss: 3.0156e-04\n",
      "elapsed time (secs) = 1295.9462139606476\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.009686104549920855\n",
      "============ Mean absolute error(in Watts): 6.868398529284696\n",
      "============ Recall: 0.32596313027862733\n",
      "============ Precision: 0.5105924753867792\n",
      "============ Accuracy: 0.9592776376234459\n",
      "============ F1 Score: 0.3979037865424466\n",
      "********************\n",
      "[DAE-rice cooker]\n",
      "********************\n",
      "211/211 [==============================] - 4s 17ms/step - loss: 7.3027e-04\n",
      "elapsed time (secs) = 26.81532573699951\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.4359062204852021\n",
      "============ Mean absolute error(in Watts): 30.735400303815805\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.04127363543610108\n",
      "============ Accuracy: 0.04127363543610108\n",
      "============ F1 Score: 0.07927529139603166\n",
      "********************\n",
      "[S2P-rice cooker]\n",
      "********************\n",
      "6750/6750 [==============================] - 266s 39ms/step - loss: 4.2741e-04\n",
      "elapsed time (secs) = 290.44653058052063\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.40330225387987656\n",
      "============ Mean absolute error(in Watts): 12.962365333134702\n",
      "============ Recall: 0.36824826734756866\n",
      "============ Precision: 0.42177657796631957\n",
      "============ Accuracy: 0.9530725778049639\n",
      "============ F1 Score: 0.39319901131001417\n",
      "********************\n",
      "[DRF-rice cooker]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100\n",
      "building tree 5 of 100\n",
      "building tree 6 of 100\n",
      "building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "building tree 9 of 100\n",
      "building tree 10 of 100\n",
      "\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100\n",
      "building tree 13 of 100\n",
      "building tree 14 of 100building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100\n",
      "\n",
      "building tree 19 of 100\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100\n",
      "building tree 24 of 100\n",
      "building tree 25 of 100\n",
      "building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "building tree 28 of 100\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100\n",
      "building tree 31 of 100\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100building tree 34 of 100\n",
      "\n",
      "building tree 35 of 100\n",
      "building tree 36 of 100\n",
      "building tree 37 of 100\n",
      "building tree 38 of 100building tree 39 of 100\n",
      "building tree 40 of 100building tree 41 of 100\n",
      "building tree 42 of 100building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "\n",
      "\n",
      "building tree 45 of 100\n",
      "building tree 46 of 100\n",
      "building tree 47 of 100\n",
      "\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100building tree 50 of 100\n",
      "\n",
      "building tree 51 of 100\n",
      "building tree 52 of 100\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100building tree 59 of 100\n",
      "\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100\n",
      "building tree 78 of 100\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:   16.2s remaining:    2.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:   18.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train on chunk is done..\n",
      "elapsed time (secs) = 41.270031213760376\n",
      "New sensible chunk: 357621\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    2.1s remaining:    0.3s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    2.3s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 173735\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    0.6s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    0.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 332127\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.1s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.3s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.008576108206751636\n",
      "============ Mean absolute error(in Watts): 3.35701965002714\n",
      "============ Recall: 0.6996548724711692\n",
      "============ Precision: 0.6668895426584648\n",
      "============ Accuracy: 0.9731794909210509\n",
      "============ F1 Score: 0.6828794040723548\n",
      "********************\n",
      "[GBM-rice cooker]\n",
      "********************\n",
      "train on chunk is done..\n",
      "elapsed time (secs) = 73.22079467773438\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.008240669047552714\n",
      "============ Mean absolute error(in Watts): 8.984373265245116\n",
      "============ Recall: 0.3573332585089368\n",
      "============ Precision: 0.3783197670964292\n",
      "============ Accuracy: 0.9492391841858709\n",
      "============ F1 Score: 0.36752716411018604\n",
      "********************\n",
      "[RESNET-rice cooker]\n",
      "********************\n",
      "16/16 [==============================] - 38s 2s/step - loss: 1.4180 - mean_absolute_error: 0.5337 - root_mean_squared_error: 1.1908\n",
      "elapsed time (secs) = 61.104241371154785\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.8579537142211139\n",
      "============ Mean absolute error(in Watts): 92.38145931952602\n",
      "============ Recall: 0.7312494738909622\n",
      "============ Precision: 0.03105690727497417\n",
      "============ Accuracy: 0.04728187418136589\n",
      "============ F1 Score: 0.05958325331284809\n",
      "Model: \"sequential_17\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_36 (Conv1D)           (None, 50, 16)            80        \n",
      "_________________________________________________________________\n",
      "bidirectional_22 (Bidirectio (None, 50, 128)           31488     \n",
      "_________________________________________________________________\n",
      "dropout_39 (Dropout)         (None, 50, 128)           0         \n",
      "_________________________________________________________________\n",
      "bidirectional_23 (Bidirectio (None, 256)               198144    \n",
      "_________________________________________________________________\n",
      "dropout_40 (Dropout)         (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_40 (Dense)             (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_41 (Dropout)         (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_41 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 262,737\n",
      "Trainable params: 262,737\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Model: \"sequential_19\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_39 (Conv1D)           (None, 100, 30)           330       \n",
      "_________________________________________________________________\n",
      "dropout_46 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_40 (Conv1D)           (None, 100, 30)           7230      \n",
      "_________________________________________________________________\n",
      "dropout_47 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_41 (Conv1D)           (None, 100, 40)           7240      \n",
      "_________________________________________________________________\n",
      "dropout_48 (Dropout)         (None, 100, 40)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_42 (Conv1D)           (None, 100, 50)           10050     \n",
      "_________________________________________________________________\n",
      "dropout_49 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_43 (Conv1D)           (None, 100, 50)           12550     \n",
      "_________________________________________________________________\n",
      "dropout_50 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "flatten_7 (Flatten)          (None, 5000)              0         \n",
      "_________________________________________________________________\n",
      "dense_45 (Dense)             (None, 1024)              5121024   \n",
      "_________________________________________________________________\n",
      "dropout_51 (Dropout)         (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_46 (Dense)             (None, 1)                 1025      \n",
      "=================================================================\n",
      "Total params: 5,159,449\n",
      "Trainable params: 5,159,449\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "********************\n",
      "[GRU-water filter]\n",
      "********************\n",
      "6750/6750 [==============================] - 71s 10ms/step - loss: 4.3568e-04\n",
      "elapsed time (secs) = 97.24639344215393\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.04267060060111629\n",
      "============ Mean absolute error(in Watts): 49.60084022496204\n",
      "============ Recall: 0.9927704629057581\n",
      "============ Precision: 0.42779004391385267\n",
      "============ Accuracy: 0.45439448001751054\n",
      "============ F1 Score: 0.5979292228440927\n",
      "********************\n",
      "[RNN-water filter]\n",
      "********************\n",
      "6750/6750 [==============================] - 111s 16ms/step - loss: 4.5372e-04\n",
      "elapsed time (secs) = 134.3831925392151\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.12031431741670849\n",
      "============ Mean absolute error(in Watts): 53.11009540978933\n",
      "============ Recall: 0.9998696359442042\n",
      "============ Precision: 0.41155967412301575\n",
      "============ Accuracy: 0.41574858045515767\n",
      "============ F1 Score: 0.5831053933761612\n",
      "********************\n",
      "[WGRU-water filter]\n",
      "********************\n",
      "6750/6750 [==============================] - 1276s 188ms/step - loss: 4.0225e-04\n",
      "elapsed time (secs) = 1299.5861399173737\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.09746232071801042\n",
      "============ Mean absolute error(in Watts): 48.05628487636619\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.40860200107953587\n",
      "============ Accuracy: 0.40860200107953587\n",
      "============ F1 Score: 0.5801525211044541\n",
      "********************\n",
      "[DAE-water filter]\n",
      "********************\n",
      "211/211 [==============================] - 4s 18ms/step - loss: 5.5596e-04\n",
      "elapsed time (secs) = 26.16006875038147\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.06041740234598536\n",
      "============ Mean absolute error(in Watts): 56.92958465759904\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.40864593430544505\n",
      "============ Accuracy: 0.40864593430544505\n",
      "============ F1 Score: 0.58019680368713\n",
      "********************\n",
      "[S2P-water filter]\n",
      "********************\n",
      "6750/6750 [==============================] - 266s 39ms/step - loss: 4.2034e-04\n",
      "elapsed time (secs) = 289.9271790981293\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.25049403602648734\n",
      "============ Mean absolute error(in Watts): 49.85744603038819\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.4085571558323602\n",
      "============ Accuracy: 0.4085571558323602\n",
      "============ F1 Score: 0.5801073163991433\n",
      "********************\n",
      "[DRF-water filter]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100\n",
      "building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100building tree 5 of 100\n",
      "building tree 6 of 100building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "\n",
      "building tree 9 of 100\n",
      "\n",
      "building tree 10 of 100\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100\n",
      "building tree 13 of 100\n",
      "building tree 14 of 100\n",
      "building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100\n",
      "building tree 19 of 100\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100building tree 24 of 100\n",
      "building tree 25 of 100building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "\n",
      "building tree 28 of 100\n",
      "\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100\n",
      "building tree 31 of 100\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100\n",
      "building tree 34 of 100\n",
      "building tree 35 of 100\n",
      "building tree 36 of 100\n",
      "building tree 37 of 100\n",
      "building tree 38 of 100\n",
      "building tree 39 of 100\n",
      "building tree 40 of 100\n",
      "building tree 41 of 100\n",
      "building tree 42 of 100\n",
      "building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "building tree 45 of 100\n",
      "building tree 46 of 100\n",
      "building tree 47 of 100\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100\n",
      "building tree 50 of 100\n",
      "building tree 51 of 100\n",
      "building tree 52 of 100\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100\n",
      "building tree 59 of 100\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100\n",
      "building tree 78 of 100\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:   12.3s remaining:    1.7s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:   13.8s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train on chunk is done..\n",
      "elapsed time (secs) = 38.803539514541626\n",
      "New sensible chunk: 357621\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.8s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    2.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 173735\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    0.7s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    0.8s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 332127\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.3s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.009142204033222532\n",
      "============ Mean absolute error(in Watts): 25.87383364541551\n",
      "============ Recall: 0.9909028561064224\n",
      "============ Precision: 0.5720727427416782\n",
      "============ Accuracy: 0.6933841045720751\n",
      "============ F1 Score: 0.7253709080309899\n",
      "********************\n",
      "[GBM-water filter]\n",
      "********************\n",
      "train on chunk is done..\n",
      "elapsed time (secs) = 70.71453404426575\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.009212037034873746\n",
      "============ Mean absolute error(in Watts): 49.80143318235509\n",
      "============ Recall: 0.9964036524607632\n",
      "============ Precision: 0.42617261743144175\n",
      "============ Accuracy: 0.45028089790047493\n",
      "============ F1 Score: 0.5970013159570404\n",
      "********************\n",
      "[RESNET-water filter]\n",
      "********************\n",
      "16/16 [==============================] - 38s 2s/step - loss: 2.1849 - mean_absolute_error: 0.6864 - root_mean_squared_error: 1.4781\n",
      "elapsed time (secs) = 60.66292595863342\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.5581008791380434\n",
      "============ Mean absolute error(in Watts): 92.48892877542737\n",
      "============ Recall: 1.0\n",
      "============ Precision: 0.40864593430544505\n",
      "============ Accuracy: 0.40864593430544505\n",
      "============ F1 Score: 0.58019680368713\n",
      "Model: \"sequential_22\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_47 (Conv1D)           (None, 50, 16)            80        \n",
      "_________________________________________________________________\n",
      "bidirectional_28 (Bidirectio (None, 50, 128)           31488     \n",
      "_________________________________________________________________\n",
      "dropout_52 (Dropout)         (None, 50, 128)           0         \n",
      "_________________________________________________________________\n",
      "bidirectional_29 (Bidirectio (None, 256)               198144    \n",
      "_________________________________________________________________\n",
      "dropout_53 (Dropout)         (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_52 (Dense)             (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_54 (Dropout)         (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_53 (Dense)             (None, 1)                 129       \n",
      "=================================================================\n",
      "Total params: 262,737\n",
      "Trainable params: 262,737\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Model: \"sequential_24\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_50 (Conv1D)           (None, 100, 30)           330       \n",
      "_________________________________________________________________\n",
      "dropout_59 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_51 (Conv1D)           (None, 100, 30)           7230      \n",
      "_________________________________________________________________\n",
      "dropout_60 (Dropout)         (None, 100, 30)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_52 (Conv1D)           (None, 100, 40)           7240      \n",
      "_________________________________________________________________\n",
      "dropout_61 (Dropout)         (None, 100, 40)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_53 (Conv1D)           (None, 100, 50)           10050     \n",
      "_________________________________________________________________\n",
      "dropout_62 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "conv1d_54 (Conv1D)           (None, 100, 50)           12550     \n",
      "_________________________________________________________________\n",
      "dropout_63 (Dropout)         (None, 100, 50)           0         \n",
      "_________________________________________________________________\n",
      "flatten_9 (Flatten)          (None, 5000)              0         \n",
      "_________________________________________________________________\n",
      "dense_57 (Dense)             (None, 1024)              5121024   \n",
      "_________________________________________________________________\n",
      "dropout_64 (Dropout)         (None, 1024)              0         \n",
      "_________________________________________________________________\n",
      "dense_58 (Dense)             (None, 1)                 1025      \n",
      "=================================================================\n",
      "Total params: 5,159,449\n",
      "Trainable params: 5,159,449\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "********************\n",
      "[GRU-microwave]\n",
      "********************\n",
      "6750/6750 [==============================] - 71s 10ms/step - loss: 3.6021e-04\n",
      "elapsed time (secs) = 95.97215342521667\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.2061021042363955\n",
      "============ Mean absolute error(in Watts): 10.156294408654459\n",
      "============ Recall: 0.12507764826250595\n",
      "============ Precision: 0.26143740930268083\n",
      "============ Accuracy: 0.9221430465754313\n",
      "============ F1 Score: 0.16920415224913493\n",
      "********************\n",
      "[RNN-microwave]\n",
      "********************\n",
      "6750/6750 [==============================] - 112s 16ms/step - loss: 4.0711e-04\n",
      "elapsed time (secs) = 134.50029301643372\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.08623299461847028\n",
      "============ Mean absolute error(in Watts): 12.171733435190724\n",
      "============ Recall: 0.4624547813059524\n",
      "============ Precision: 0.13139739510062967\n",
      "============ Accuracy: 0.7721466946001128\n",
      "============ F1 Score: 0.2046480792008764\n",
      "********************\n",
      "[WGRU-microwave]\n",
      "********************\n",
      "6750/6750 [==============================] - 1276s 188ms/step - loss: 3.3453e-04\n",
      "elapsed time (secs) = 1298.9913957118988\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.4045930917404653\n",
      "============ Mean absolute error(in Watts): 13.208395408556918\n",
      "============ Recall: 0.12599576116348754\n",
      "============ Precision: 0.25967766229853895\n",
      "============ Accuracy: 0.9218193653904514\n",
      "============ F1 Score: 0.1696683397303415\n",
      "********************\n",
      "[DAE-microwave]\n",
      "********************\n",
      "211/211 [==============================] - 4s 17ms/step - loss: 4.4634e-04\n",
      "elapsed time (secs) = 25.81242799758911\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.25320142266206974\n",
      "============ Mean absolute error(in Watts): 10.451015217024871\n",
      "============ Recall: 0.0\n",
      "============ Precision: nan\n",
      "============ Accuracy: 0.9366123863756122\n",
      "============ F1 Score: nan\n",
      "********************\n",
      "[S2P-microwave]\n",
      "********************\n",
      "6750/6750 [==============================] - 264s 39ms/step - loss: 4.7210e-04\n",
      "elapsed time (secs) = 290.63641357421875\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.5631694360275026\n",
      "============ Mean absolute error(in Watts): 8.623785241499128\n",
      "============ Recall: 0.0\n",
      "============ Precision: nan\n",
      "============ Accuracy: 0.9365928932881055\n",
      "============ F1 Score: nan\n",
      "********************\n",
      "[DRF-microwave]\n",
      "********************\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "building tree 1 of 100\n",
      "building tree 2 of 100\n",
      "building tree 3 of 100\n",
      "building tree 4 of 100\n",
      "building tree 5 of 100\n",
      "building tree 6 of 100\n",
      "building tree 7 of 100\n",
      "building tree 8 of 100\n",
      "building tree 9 of 100\n",
      "building tree 10 of 100\n",
      "building tree 11 of 100\n",
      "building tree 12 of 100building tree 13 of 100\n",
      "building tree 14 of 100\n",
      "building tree 15 of 100\n",
      "building tree 16 of 100\n",
      "\n",
      "building tree 17 of 100\n",
      "building tree 18 of 100building tree 19 of 100\n",
      "\n",
      "building tree 20 of 100\n",
      "building tree 21 of 100\n",
      "building tree 22 of 100\n",
      "building tree 23 of 100\n",
      "building tree 24 of 100\n",
      "building tree 25 of 100\n",
      "building tree 26 of 100\n",
      "building tree 27 of 100\n",
      "building tree 28 of 100\n",
      "building tree 29 of 100\n",
      "building tree 30 of 100\n",
      "building tree 31 of 100\n",
      "building tree 32 of 100\n",
      "building tree 33 of 100\n",
      "building tree 34 of 100\n",
      "building tree 35 of 100building tree 36 of 100building tree 37 of 100building tree 38 of 100\n",
      "\n",
      "building tree 39 of 100building tree 40 of 100\n",
      "building tree 41 of 100\n",
      "\n",
      "\n",
      "\n",
      "building tree 42 of 100\n",
      "building tree 43 of 100\n",
      "building tree 44 of 100\n",
      "building tree 45 of 100building tree 46 of 100\n",
      "\n",
      "building tree 47 of 100\n",
      "building tree 48 of 100\n",
      "building tree 49 of 100\n",
      "building tree 50 of 100\n",
      "building tree 51 of 100\n",
      "building tree 52 of 100\n",
      "building tree 53 of 100\n",
      "building tree 54 of 100\n",
      "building tree 55 of 100\n",
      "building tree 56 of 100\n",
      "building tree 57 of 100\n",
      "building tree 58 of 100\n",
      "building tree 59 of 100\n",
      "building tree 60 of 100\n",
      "building tree 61 of 100\n",
      "building tree 62 of 100\n",
      "building tree 63 of 100\n",
      "building tree 64 of 100\n",
      "building tree 65 of 100\n",
      "building tree 66 of 100\n",
      "building tree 67 of 100\n",
      "building tree 68 of 100\n",
      "building tree 69 of 100\n",
      "building tree 70 of 100\n",
      "building tree 71 of 100\n",
      "building tree 72 of 100\n",
      "building tree 73 of 100\n",
      "building tree 74 of 100\n",
      "building tree 75 of 100\n",
      "building tree 76 of 100\n",
      "building tree 77 of 100\n",
      "building tree 78 of 100\n",
      "building tree 79 of 100\n",
      "building tree 80 of 100\n",
      "building tree 81 of 100\n",
      "building tree 82 of 100\n",
      "building tree 83 of 100\n",
      "building tree 84 of 100\n",
      "building tree 85 of 100\n",
      "building tree 86 of 100\n",
      "building tree 87 of 100\n",
      "building tree 88 of 100\n",
      "building tree 89 of 100\n",
      "building tree 90 of 100\n",
      "building tree 91 of 100\n",
      "building tree 92 of 100\n",
      "building tree 93 of 100\n",
      "building tree 94 of 100\n",
      "building tree 95 of 100\n",
      "building tree 96 of 100\n",
      "building tree 97 of 100\n",
      "building tree 98 of 100\n",
      "building tree 99 of 100\n",
      "building tree 100 of 100\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:   13.4s remaining:    1.8s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:   15.2s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train on chunk is done..\n",
      "elapsed time (secs) = 37.61514234542847\n",
      "New sensible chunk: 357621\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.7s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.9s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 173735\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    0.6s remaining:    0.1s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    0.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New sensible chunk: 332127\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=32)]: Using backend ThreadingBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=32)]: Done  88 out of 100 | elapsed:    1.3s remaining:    0.2s\n",
      "[Parallel(n_jobs=32)]: Done 100 out of 100 | elapsed:    1.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.009994332164012748\n",
      "============ Mean absolute error(in Watts): 3.714192880446433\n",
      "============ Recall: 0.7119340811926773\n",
      "============ Precision: 0.688852354688163\n",
      "============ Accuracy: 0.9613564166437941\n",
      "============ F1 Score: 0.7002030511581103\n",
      "********************\n",
      "[GBM-microwave]\n",
      "********************\n",
      "train on chunk is done..\n",
      "elapsed time (secs) = 77.44414496421814\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "Disaggregation is done..\n",
      "============ Relative error in total energy: 0.011029575936066158\n",
      "============ Mean absolute error(in Watts): 9.04897982028675\n",
      "============ Recall: 0.1237073848065188\n",
      "============ Precision: 0.26535251009131167\n",
      "============ Accuracy: 0.9227441020705724\n",
      "============ F1 Score: 0.16874556080298067\n",
      "********************\n",
      "[RESNET-microwave]\n",
      "********************\n",
      "16/16 [==============================] - 39s 2s/step - loss: 1.1665 - mean_absolute_error: 0.4989 - root_mean_squared_error: 1.0800\n",
      "elapsed time (secs) = 61.9014093875885\n",
      "New sensible chunk: 357621\n",
      "New sensible chunk: 173735\n",
      "New sensible chunk: 332127\n",
      "============ Relative error in total energy: 0.5633021459106192\n",
      "============ Mean absolute error(in Watts): 9.30041559146915\n",
      "============ Recall: 0.12569883436255344\n",
      "============ Precision: 0.2594855548012371\n",
      "============ Accuracy: 0.921841939776324\n",
      "============ F1 Score: 0.16935801496652225\n"
     ]
    }
   ],
   "source": [
    "dataset = DataSet(DATA_FILE)\n",
    "dataset.set_window(start=T_S, end=T_E)\n",
    "\n",
    "train_elec = dataset.buildings[HOME_ID].elec\n",
    "train_mains = train_elec.all_meters()[0] # The aggregated meter that provides the input\n",
    "\n",
    "for aname in appliances:\n",
    "    \n",
    "    train_meter = train_elec.submeters()[aname]\n",
    "    \n",
    "    models = {}\n",
    "    \n",
    "    models['GRU-' + aname] = GRUDisaggregator()\n",
    "    models['RNN-' + aname] = RNNDisaggregator()\n",
    "    models['WGRU-' + aname] = WindowGRUDisaggregator(window_size=50)\n",
    "    models['DAE-' + aname] = DAEDisaggregator(256)\n",
    "    models['S2P-' + aname] = ShortSeq2PointDisaggregator(window_size=100)\n",
    "    models['DRF-' + aname] = DTDisaggregator(algorithm='DRF', use_feature=False, reactive=False)\n",
    "    models['GBM-' + aname] = DTDisaggregator(algorithm='GBM', use_feature=False, reactive=False)\n",
    "    models['RESNET-' + aname] = RESNETDisaggregator()\n",
    "    \n",
    "    for mname, model in models.items():\n",
    "        \n",
    "        try:\n",
    "            \n",
    "            print(\"*\"*20)\n",
    "            print(\"[\" + mname + \"]\") \n",
    "            print(\"*\" *20)\n",
    "\n",
    "            start = time.time()\n",
    "            model.train(train_mains, train_meter, epochs=EPOCHS, sample_period=TRAIN_SAMPLE_PERIOD)\n",
    "            end = time.time()            \n",
    "            print(\"elapsed time (secs) = \" + str(end-start))\n",
    "            \n",
    "            disaggregate(dataset, model, aname, P_S, P_E, True)\n",
    "        \n",
    "        except Exception as e:\n",
    "            print('train & test error', e)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0dd1f1b9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ac704bc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}