docs/notebooks/first_steps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "901feabe",
   "metadata": {},
   "source": [
    "# First steps\n",
    "\n",
    "ConFlowGen has been developed with Jupyter Notebooks in mind.\n",
    "While it is also possible to write your code as pure Python Scripts (see e.g. the\n",
    "[Python Script examples](https://github.com/1kastner/conflowgen/tree/main/examples/Python_Script)),\n",
    "Jupyter Notebooks allow to mix the code with explanatory texts, LaTeX formulas, etc. in a single document.\n",
    "In addition, the generated visualisations neatly blend in.\n",
    "\n",
    "For newcomers it is suggested to download the\n",
    "[Anaconda distribution](https://www.anaconda.com/)\n",
    "and first to familiarize yourself with the newly installed tools.\n",
    "Then, it is suggested to create a separate environment as described for the\n",
    "[Jupyter Notebook examples](https://github.com/1kastner/conflowgen/tree/main/examples/Jupyter_Notebook).\n",
    "During the environment set up, ConFlowGen is automatically installed.\n",
    "Then, you can start JupyterLab\n",
    "[from Anaconda Navigator](https://docs.anaconda.com/free/navigator/)\n",
    "or\n",
    "[from the CLI](https://jupyterlab.readthedocs.io/en/stable/getting_started/starting.html).\n",
    "Please ensure that you are in the correct conda environment (it should be `conflowgen`) when you work."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "224b56dd",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1ceed50",
   "metadata": {},
   "source": [
    "To start, we first import ConFlowGen as a module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "07e1c5f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import datetime\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "import conflowgen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "189dbaab-c9c6-4072-a134-f41ec4d3c272",
   "metadata": {},
   "source": [
    "Then, we set up the logger"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6254cb12-dd67-41c1-a741-b419197fb40f",
   "metadata": {},
   "outputs": [],
   "source": [
    "logger = conflowgen.setup_logger(\n",
    "    logging_directory=\"./data/logger\",  # use subdirectory relative to Jupyter Notebook\n",
    "    format_string=\"%(message)s\"  # only show log messages, discard timestamp etc.\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "654d3a06",
   "metadata": {},
   "source": [
    "## Database selection"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39c749b5",
   "metadata": {},
   "source": [
    "Now, we select a database to work in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0831553",
   "metadata": {},
   "outputs": [],
   "source": [
    "database_chooser = conflowgen.DatabaseChooser(\n",
    "    sqlite_databases_directory=\"./data/db\"  # use subdirectory relative to Jupyter Notebook\n",
    ")\n",
    "demo_file_name = \"my_demo.sqlite\"\n",
    "\n",
    "database_chooser.create_new_sqlite_database(demo_file_name, overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9485720c",
   "metadata": {},
   "source": [
    "## General settings"
   ]
  },
  {
   "cell_type": "raw",
   "id": "86e26446",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "We use\n",
    ":meth:`.ContainerFlowGenerationManager.set_properties`\n",
    "to set a ``name``, the ``start_date`` and the ``end_date`` for the\n",
    "generation process.\n",
    "The ``start_date`` and ``end_date`` parameters must be provided as a :class:`datetime.date`.\n",
    "In our example, the ``end_time`` is set as a 21 day interval on top of the ``start_date``\n",
    "which is set to the time of execution.\n",
    "Other general settings can be set here, see the :class:`.ContainerFlowGenerationManager` class for more information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2981777",
   "metadata": {},
   "outputs": [],
   "source": [
    "today = datetime.datetime.now().date()\n",
    "container_flow_generation_manager = conflowgen.ContainerFlowGenerationManager()\n",
    "container_flow_generation_manager.set_properties(\n",
    "    name=\"Demo file\",\n",
    "    start_date=today,\n",
    "    end_date=today + datetime.timedelta(days=21)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3443083f",
   "metadata": {},
   "source": [
    "## Creating schedules"
   ]
  },
  {
   "cell_type": "raw",
   "id": "f10e7873",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "In the next step, we add a feeder liner service to the database.\n",
    "We do this by using\n",
    ":meth:`.PortCallManager.add_vehicle`.\n",
    "The concrete vehicle instances are only generated when\n",
    ":meth:`.ContainerFlowGenerationManager.generate`\n",
    "is invoked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "583d62ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "port_call_manager = conflowgen.PortCallManager()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80014840",
   "metadata": {},
   "source": [
    "At first we define a name for our new feeder liner service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1e05d99",
   "metadata": {},
   "outputs": [],
   "source": [
    "feeder_service_name = \"LX050\""
   ]
  },
  {
   "cell_type": "raw",
   "id": "9bef51c3",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "By using\n",
    ":meth:`.PortCallManager.add_vehicle`,\n",
    "we can define the attributes for our feeder service.\n",
    "\n",
    "- ``vehicle_type`` defines, that we deal with a feeder as the mode of transport.\n",
    "  Other valid modes of transport are deep_sea_vessel, barge, or train.\n",
    "- ``service_name`` defines a fictional name for that service.\n",
    "- ``vehicle_arrives_at`` specifies the date where the first port call of this particular line is usually happening.\n",
    "  This parameter must be a :py:obj:`datetime.date`.\n",
    "- ``vehicle_arrives_at_time`` sets the scheduled timeslot of the port call.\n",
    "  This parameter must be a :py:obj:`datetime.time`.\n",
    "- ``average_vehicle_capacity`` defines the average capacity of the vessels utilized on this line.\n",
    "  Parameter must be :py:obj:`int` or :py:obj:`float`.\n",
    "- ``average_moved_capacity`` sets the capacity which is in average moved between the feeder and the terminal at each call.\n",
    "  Parameter must be :py:obj:`int` or :py:obj:`float`.\n",
    "- ``next_destinations`` can be set, consisting of name and frequency which can, e.g., be used as implication for storage and stacking problems.\n",
    "  A list of name-frequency pairs is expected here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82ccee1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "port_call_manager.add_vehicle(\n",
    "    vehicle_type=conflowgen.ModeOfTransport.feeder,\n",
    "    service_name=feeder_service_name,\n",
    "    vehicle_arrives_at=datetime.date(2021, 7, 9),\n",
    "    vehicle_arrives_at_time=datetime.time(11),\n",
    "    average_vehicle_capacity=800,\n",
    "    average_moved_capacity=100,\n",
    "    next_destinations=[\n",
    "        (\"DEBRV\", 0.4),  # 40% of the containers go here...\n",
    "        (\"RULED\", 0.6)   # and the other 60% of the containers go here.\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dfbc7a0",
   "metadata": {},
   "source": [
    "Following the same principle and structure we can also add schedules for trains and deep sea vessels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dc13855",
   "metadata": {},
   "outputs": [],
   "source": [
    "port_call_manager.add_vehicle(\n",
    "    vehicle_type=conflowgen.ModeOfTransport.train,\n",
    "    service_name=\"JR03A\",\n",
    "    vehicle_arrives_at=datetime.date(2021, 7, 12),\n",
    "    vehicle_arrives_at_time=datetime.time(17),\n",
    "    average_vehicle_capacity=90,\n",
    "    average_moved_capacity=90,\n",
    "    next_destinations=None  # Here we don't have containers that need to be grouped by destination\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b82cd354",
   "metadata": {},
   "outputs": [],
   "source": [
    "port_call_manager.add_vehicle(\n",
    "    vehicle_type=conflowgen.ModeOfTransport.deep_sea_vessel,\n",
    "    service_name=\"LX050\",\n",
    "    vehicle_arrives_at=datetime.date(2021, 7, 10),\n",
    "    vehicle_arrives_at_time=datetime.time(19),\n",
    "    average_vehicle_capacity=16000,\n",
    "    average_moved_capacity=150,  # for faster demo\n",
    "    next_destinations=[\n",
    "        (\"ZADUR\", 0.3),  # 30% of the containers go to ZADUR...\n",
    "        (\"CNSHG\", 0.7)   # and the other 70% of the containers go to CNSHG.\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "469bf87b",
   "metadata": {},
   "source": [
    "## Generate the data"
   ]
  },
  {
   "cell_type": "raw",
   "id": "428e6a81",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "After we have set some exemplary schedules, we can now come to the actual generation.\n",
    "The generation process of the synthetic container flow data is started with\n",
    ":meth:`.ContainerFlowGenerationManager.generate`,\n",
    "based on the information you set earlier.\n",
    "With the optional parameter ``overwrite`` you determine the behavior if already some data exists - with the default behavior ``overwrite=True``, any previous container flow data is overwritten, with ``overwrite=False`` the generation step is skipped if any data is detected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0a1314b",
   "metadata": {},
   "outputs": [],
   "source": [
    "container_flow_generation_manager.generate()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e04cf852-374a-4023-a388-038f75664930",
   "metadata": {},
   "source": [
    "## Export the data\n",
    "\n",
    "The data is exported with two lines of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4065218d-259c-4d8e-8606-50c8e211af58",
   "metadata": {},
   "outputs": [],
   "source": [
    "export_container_flow_manager = conflowgen.ExportContainerFlowManager()\n",
    "path_to_exported_data = export_container_flow_manager.export(\"first_steps\", \"./data/export\", overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "472e848d-9fcd-439b-be33-f652e407a78b",
   "metadata": {},
   "source": [
    "## Examining the exported data\n",
    "\n",
    "The CSV files we can open e.g. with pandas.\n",
    "Here, the tabular data is presented."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b43c1b98-fbce-4b96-878f-c05698e59cea",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_containers = pd.read_csv(\n",
    "    os.path.join(path_to_exported_data, \"containers.csv\"),\n",
    "    index_col=\"id\",\n",
    "    dtype={\n",
    "        \"delivered_by_truck\": \"Int64\",\n",
    "        \"picked_up_by_truck\": \"Int64\",\n",
    "        \"delivered_by_vehicle\": \"Int64\",\n",
    "        \"picked_up_by_vehicle\": \"Int64\",\n",
    "        \"destination_sequence_id\": \"Int64\"\n",
    "    }\n",
    ")\n",
    "df_containers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0012b465-7059-4f50-8978-d125e74c7200",
   "metadata": {},
   "source": [
    "The column `delivered_by` contains the vehicle type a container is delivered by.\n",
    "For trucks, the column `delivered_by_truck` mentions the corresponding ID of the vehicle that is stored in the separate table `trucks.csv`.\n",
    "The same scheme applies to `picked_up_by` and `picked_up_by_truck`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87701627-f9ae-452a-b3b2-bf60fbf92096",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_trucks = pd.read_csv(\n",
    "    os.path.join(path_to_exported_data, \"trucks.csv\"),\n",
    "    index_col=\"id\"\n",
    ")\n",
    "df_trucks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2852be6d-5b16-47f3-a6b4-88db5acec37f",
   "metadata": {},
   "source": [
    "Here, we can see which trucks deliver a container, pick up a container, and for which time their arrivals are realized."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b11de63-a8a7-4b0b-a77f-45afa574e7d9",
   "metadata": {},
   "source": [
    "All vehicle types except trucks are considered vehicles that typically arrive according to a schedule and which move larger amounts of containers at once.\n",
    "Thus, the table has a slightly different shape.\n",
    "They all have ids from the same ID range and for each vehicle type the ids might be non-consecutive.\n",
    "So e.g. for trains, in the column `delivered_by_vehicle` in the container table the ID is mentioned that we can look up in the table of `trains.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cfaf5fe1-973d-43c8-82f0-24cdff693bdc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_trains = pd.read_csv(\n",
    "    os.path.join(path_to_exported_data, \"trains.csv\"),\n",
    "    index_col=\"id\"\n",
    ")\n",
    "df_trains"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0949540f-f1c4-4a8e-a86c-21fb1dddcd16",
   "metadata": {},
   "source": [
    "Corresponding CSV files exist for the other vehicles as well."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}