examples/05_example_input_formats.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0eb609b3",
   "metadata": {},
   "source": [
    "# Example 5: input file formats\n",
    "\n",
    "This example shows the 3 types of file formats that *sportran* can read.\n",
    "\n",
    "- **`table`**: a plain-text file where data is organized in columns.\n",
    "- **`dict`**: a Numpy binary file that contains a dictionary.\n",
    "- **`lammps`**: a LAMMPS output log file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7723b049",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# import scipy as sp\n",
    "# import matplotlib.pyplot as plt\n",
    "try:\n",
    "    import sportran as st\n",
    "except ImportError:\n",
    "    from sys import path\n",
    "    path.append('..')\n",
    "    import sportran as st\n",
    "\n",
    "# c = plt.rcParams['axes.prop_cycle'].by_key()['color']\n",
    "\n",
    "data_path = './data/Silica/'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a09ab3e3",
   "metadata": {},
   "source": [
    "## 1. `table` format\n",
    "A `table`-type file is a plain-text file containing multiple time series. \n",
    "\n",
    "- Each row represents a data point (a specific time), and each row represents a different time series (i.e. a variable, e.g. temperature, energy, heat flux, ...). \n",
    "- The first line of each column contains the name of the variable. Cartesian component of vector quantities can be expressed with square brackets (e.g. `vector[1]`). \n",
    "- Similarly to LAMMPS conventions, `c_` or `v_` prefixes of column names are ignored, so e.g. `c_flux` will be saved as `flux`.\n",
    "- Comment lines start with `#`.\n",
    "\n",
    "For further information, see the documentation of [TableFile](https://sportran.readthedocs.io/en/latest/_autosummary/sportran.i_o.read_tablefile.TableFile.html#sportran.i_o.read_tablefile.TableFile)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "79708ff9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Solid Silica - BKS potential, melted and quenched\r\n",
      "# 216 atoms, T~1000K, dens~2.295g/cm^3\r\n",
      "# NVE, dt = 1.0 fs, 100 ps, print_step = 1.0 fs\r\n",
      "# Temperature = 983.172635 K, Volume = 3130.431110818 A^3\r\n",
      "# LAMMPS metal units\r\n",
      "Temp c_flux1[1] c_flux1[2] c_flux1[3]\r\n",
      "998.48171 -265.30586 1520.6107 67.461829\r\n",
      "1003.699 -168.68352 1377.4459 101.82146\r\n",
      "1003.8906 -93.688306 1180.375 117.20939\r\n",
      "998.1473 -42.571972 932.96168 111.11515\r\n",
      "986.48517 -15.323416 642.52765 85.389352\r\n",
      "969.86291 -10.876607 319.90865 45.695167\r\n",
      "950.03861 -27.873411 -21.428315 -0.1944876\r\n",
      "929.29852 -64.46361 -366.51677 -44.776231\r\n",
      "910.08762 -117.84517 -700.11875 -82.966928\r\n"
     ]
    }
   ],
   "source": [
    "# Example\n",
    "!head -n 15 ./data/Silica/Silica.dat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b458c7f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Solid Silica - BKS potential, melted and quenched\n",
      "# 216 atoms, T~1000K, dens~2.295g/cm^3\n",
      "# NVE, dt = 1.0 fs, 100 ps, print_step = 1.0 fs\n",
      "# Temperature = 983.172635 K, Volume = 3130.431110818 A^3\n",
      "# LAMMPS metal units\n",
      "Temp c_flux1[1] c_flux1[2] c_flux1[3]\n",
      " #####################################\n",
      "  all_ckeys =  [('Temp', [0]), ('flux1', array([1, 2, 3]))]\n",
      " #####################################\n",
      "Data length =  100001\n"
     ]
    }
   ],
   "source": [
    "# Load table file\n",
    "f = st.i_o.TableFile(data_path + '/Silica.dat', group_vectors=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "adde30fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Temp': [0], 'flux1': array([1, 2, 3])}\n"
     ]
    }
   ],
   "source": [
    "# list of available keys (column names) and their column indexes\n",
    "print(f.all_ckeys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f12501de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  ckey =  [('Temp', [0]), ('flux1', array([1, 2, 3]))]\n",
      "    step =    100000 - 100.00% completed\n",
      "  ( 100000 ) steps read.\n",
      "DONE.  Elapsed time:  0.7381811141967773 seconds\n"
     ]
    }
   ],
   "source": [
    "# read the file loading the following columns\n",
    "data = f.read_datalines(\n",
    "    NSTEPS = 0,  # read all the steps\n",
    "    select_ckeys = ['Temp', 'flux1'],  # read only these columns\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "86299727",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Temp': array([[ 998.48171],\n",
       "        [1003.699  ],\n",
       "        [1003.8906 ],\n",
       "        ...,\n",
       "        [ 967.21723],\n",
       "        [ 978.47566],\n",
       "        [ 985.41455]]),\n",
       " 'flux1': array([[ -265.30586 ,  1520.6107  ,    67.461829],\n",
       "        [ -168.68352 ,  1377.4459  ,   101.82146 ],\n",
       "        [  -93.688306,  1180.375   ,   117.20939 ],\n",
       "        ...,\n",
       "        [ 1226.9778  ,   212.0939  , -1126.4643  ],\n",
       "        [ 1223.8753  ,   186.93836 ,  -881.39541 ],\n",
       "        [ 1232.7723  ,   141.30647 ,  -620.41895 ]])}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# data can be also retrieved from the f.data dictionary\n",
    "f.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "870ab4a7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100000, 3)\n"
     ]
    }
   ],
   "source": [
    "TEMPERATURE = np.mean(data['Temp'])\n",
    "flux = data['flux1']\n",
    "print(flux.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5b1b3b56",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell and volume information can be set manually or retrieved from\n",
    "# a LAMMPS data file (written using the `write_data` command)\n",
    "box, VOLUME = st.i_o.read_lammps_datafile.get_box(data_path + '/lammps/silica_216_1000K.init')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9a4206f3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using single component code.\n"
     ]
    }
   ],
   "source": [
    "# we can finally create a HeatCurrent:\n",
    "j = st.HeatCurrent(flux, units='metal', TEMPERATURE=TEMPERATURE, VOLUME=VOLUME, DT_FS=1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "529c1ce4",
   "metadata": {},
   "source": [
    "Data contained in `TableFile` can be converted to a numpy binary file, that can be reloaded faster at a later time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6b5aea43",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.save('new_data.npy', f.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a43504f0",
   "metadata": {},
   "source": [
    "Side note: File is read sequentially, so you can actually read the first and second 1000 lines like this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e4099d55",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  ckey =  [('Temp', [0]), ('flux1', array([1, 2, 3]))]\n",
      "  ( 1000 ) steps read.\n",
      "DONE.  Elapsed time:  0.023930072784423828 seconds\n",
      "  ckey =  [('Temp', [0]), ('flux1', array([1, 2, 3]))]\n",
      "  ( 1000 ) steps read.\n",
      "DONE.  Elapsed time:  0.01228475570678711 seconds\n"
     ]
    }
   ],
   "source": [
    "f.gotostep(0)  # go back to the step number 0\n",
    "\n",
    "f.read_datalines(NSTEPS=1000)  # read first 1000 steps\n",
    "first_block = f.data\n",
    "\n",
    "f.read_datalines(NSTEPS=1000)  # read next 1000 steps\n",
    "second_block = f.data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02566e47",
   "metadata": {},
   "source": [
    "## 2. `dict` format\n",
    "A `dict`-type file is a Numpy binary file that contains a dictionary.\n",
    "\n",
    "It can be read simply like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "22b4f2a6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'flux1': array([[ -265.30586 ,  1520.6107  ,    67.461829],\n",
       "        [ -168.68352 ,  1377.4459  ,   101.82146 ],\n",
       "        [  -93.688306,  1180.375   ,   117.20939 ],\n",
       "        ...,\n",
       "        [ 1226.9778  ,   212.0939  , -1126.4643  ],\n",
       "        [ 1223.8753  ,   186.93836 ,  -881.39541 ],\n",
       "        [ 1232.7723  ,   141.30647 ,  -620.41895 ]]),\n",
       " 'Temperature': 983.1726353043,\n",
       " 'Volume': 3130.431110818276,\n",
       " 'DT_FS': 1.0,\n",
       " 'units': 'lammps-metal'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = np.load(data_path + '/Silica.npy', allow_pickle=True).tolist()\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a174f917",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using single component code.\n"
     ]
    }
   ],
   "source": [
    "# we can finally create a HeatCurrent:\n",
    "j = st.HeatCurrent(\n",
    "    data['flux1'], units='metal', TEMPERATURE=data['Temperature'], VOLUME=data['Volume'], DT_FS=data['DT_FS'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9805c32",
   "metadata": {},
   "source": [
    "## 3. `lammps` format\n",
    "A `lammps`-type file is a LAMMPS log file, i.e. the output generated by lammps (see [`log` command](https://docs.lammps.org/log.html), by default it is called `log.lammps`.\n",
    "\n",
    "The [LammpsLogFile](https://sportran.readthedocs.io/en/latest/_autosummary/sportran.i_o.read_lammps_log.LAMMPSLogFile.html#sportran.i_o.read_lammps_log.LAMMPSLogFile) class can parse a lammps log file and convert data into a dictionary, and eventually save it as Numpy binary files.\n",
    "\n",
    "As in a LAMMPS script there might be multiple `run` commands, we need to tell the parser which `run` it should read.\n",
    "We can do so by indicating a keyword string (called `run_keyword`) that it should look for. It skip all the lines until it finds this string, so it should be something evident.\n",
    "\n",
    "For example, it can be an uppercase COMMENT LINE that you have placed _just before_ the `run` command that launches the production run of your simulation. This is an example of a LAMMPS Log file where the `NVE RUN` keyword has been inserted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "569b599e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# NVE RUN\r\n",
      "fix          NVE_RUN all nve\r\n",
      "run          100000\r\n",
      "Generated 0 of 1 mixed pair_coeff terms from geometric mixing rule\r\n",
      "Per MPI rank memory allocation (min/avg/max) = 4.015 | 4.018 | 4.022 Mbytes\r\n",
      "   Step          Time           Temp          PotEng         KinEng         TotEng         Press        c_flux1[1]     c_flux1[2]     c_flux1[3]  \r\n",
      "         0   0              998.48171     -3245.1241      27.748737     -3217.3753     -4193.4348     -265.30586      1520.6107      67.461829    \r\n",
      "         1   0.001          1003.699      -3245.2719      27.893731     -3217.3782     -5450.2034     -168.68352      1377.4459      101.82146    \r\n",
      "         2   0.002          1003.8906     -3245.2786      27.899055     -3217.3795     -6701.621      -93.688306      1180.375       117.20939    \r\n",
      "         3   0.003          998.1473      -3245.1188      27.739443     -3217.3794     -7882.8782     -42.571972      932.96168      111.11515    \r\n",
      "         4   0.004          986.48517     -3244.7932      27.415341     -3217.3778     -8949.7826     -15.323416      642.52765      85.389352    \r\n",
      "         5   0.005          969.86291     -3244.328       26.953393     -3217.3746     -9878.8464     -10.876607      319.90865      45.695167    \r\n",
      "         6   0.006          950.03861     -3243.7731      26.402458     -3217.3707     -10669.51      -27.873411     -21.428315     -0.1944876    \r\n",
      "         7   0.007          929.29852     -3243.1925      25.826071     -3217.3664     -11337.595     -64.46361      -366.51677     -44.776231    \r\n",
      "         8   0.008          910.08762     -3242.6549      25.292183     -3217.3627     -11909.803     -117.84517     -700.11875     -82.966928    \r\n",
      "tail: error writing 'standard output': Broken pipe\r\n"
     ]
    }
   ],
   "source": [
    "!tail -n +174 ./data/Silica/lammps/silica.out | head -n 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0242fdcf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  run_keyword found at line 174.\n",
      "  column headers found at line 179. Reading data...\n",
      " #####################################\n",
      "  all_ckeys =  [('KinEng', array([4])), ('PotEng', array([3])), ('Press', array([6])), ('Step', array([0])), ('Temp', array([2])), ('Time', array([1])), ('TotEng', array([5])), ('flux1', array([7, 8, 9]))]\n",
      " #####################################\n"
     ]
    }
   ],
   "source": [
    "f = st.i_o.LAMMPSLogFile(data_path + '/lammps/silica.out', run_keyword='NVE RUN')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "e9021abf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  ckey =  [('KinEng', array([4])), ('PotEng', array([3])), ('Press', array([6])), ('Step', array([0])), ('Temp', array([2])), ('Time', array([1])), ('TotEng', array([5])), ('flux1', array([7, 8, 9]))]\n",
      "    step =    100000 -  99.96% completed\n",
      "  endrun_keyword found.\n",
      "  Retaining an even number of steps (even_NSTEPS=True).\n",
      "  ( 100000 ) steps read.\n",
      "DONE.  Elapsed time:  1.8878519535064697 seconds\n"
     ]
    }
   ],
   "source": [
    "# read the file loading the following columns\n",
    "data = f.read_datalines(\n",
    "    NSTEPS = 0,  # read all the steps\n",
    "    select_ckeys = None,  # columns to be read, if None read them all!\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "96ff3733",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['Step', 'Time', 'Temp', 'PotEng', 'KinEng', 'TotEng', 'Press', 'flux1'])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "be7816c6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100000, 3)\n"
     ]
    }
   ],
   "source": [
    "TEMPERATURE = np.mean(data['Temp'])\n",
    "flux = data['flux1']\n",
    "print(flux.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4ee7a0d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell and volume information can be set manually or retrieved from\n",
    "# a LAMMPS data file (written using the `write_data` command)\n",
    "box, VOLUME = st.i_o.read_lammps_datafile.get_box(data_path + '/lammps/silica_216_1000K.init')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "39dad1d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using single component code.\n"
     ]
    }
   ],
   "source": [
    "# we can finally create a HeatCurrent:\n",
    "j = st.HeatCurrent(flux, units='metal', TEMPERATURE=TEMPERATURE, VOLUME=VOLUME, DT_FS=1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8177382d",
   "metadata": {},
   "source": [
    "Finally, we can write the content of the `LAMMPSLogFile` into a numpy binary file, that can be reloaded faster at a later time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3d8a1f5c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "These keys will be saved in file \"new_data.npy\" :\n",
      "  ['Temp_ave', 'Temp_std', 'Time', 'Temp', 'flux1', 'box', 'Volume', 'DT', 'DT_TIMEUNITS']\n"
     ]
    }
   ],
   "source": [
    "f.save_numpy_dict('new_data.npy',\n",
    "                  select_ckeys=['Time', 'Temp', 'flux1'],\n",
    "                  lammps_data_file=data_path + '/lammps/silica_216_1000K.init')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "553b7b85",
   "metadata": {},
   "source": [
    "For further information, see the documentation of [LAMMPSLogFile](https://sportran.readthedocs.io/en/latest/_autosummary/sportran.i_o.read_lammps_log.LAMMPSLogFile.html#sportran.i_o.read_lammps_log.LAMMPSLogFile)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}