docs/source/client/notebooks/python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "## End-user python API\n",
    "This section explains python API in panda-client for end-users.\n",
    "All functions are available through the API object."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "from pandaclient import panda_api\n",
    "c = panda_api.get_api()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "----\n",
    "\n",
    "<br>\n",
    "\n",
    "### Task submission API\n",
    "\n",
    "*  **Submit a task (low-level)**\n",
    "\n",
    "You need to prepare a dictionary of task parameters following [an example](https://github.com/PanDAWMS/panda-client/blob/master/pandaclient/example_task.py).\n",
    "This function gives you a taskID, which is the unique identifier in the system, once it successfully submits the task.\n",
    "Note that it is highly recommended using another high-level API, such as execute_prun, execute_pathena,\n",
    "and execute_phpo, and only developers should this method since the task parameter dictionary is\n",
    "quite cryptic. Note that although all task submission functions have similar interface to\n",
    "subporcess functions they are executed in the same python interpreter, i.e., they don't spawn\n",
    "child processes."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "from pandaclient.example_task import taskParamMap\n",
    "communication_status, o = c.submit_task(taskParamMap)\n",
    "if communication_status:\n",
    "    server_return_code, dialog_message, task_id = o\n",
    "    if o == 0:\n",
    "        print (\"taskID={}\".format(task_id))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "submit_task(self, task_params, verbose=False)\n",
    "    Description: Submit a task to PanDA\n",
    "\n",
    "    args:\n",
    "        task_params: a dictionary of task parameters\n",
    "        verbose: True to see debug messages\n",
    "    returns:\n",
    "        status code\n",
    "              0: communication succeeded to the panda server\n",
    "              255: communication failure\n",
    "        tuple of return code, message from the server, and task ID if successful\n",
    "              0: request is processed\n",
    "              1: duplication in DEFT\n",
    "              2: duplication in JEDI\n",
    "              3: accepted for incremental execution\n",
    "              4: server error\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Submit a prun task**"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "import uuid\n",
    "com = \"--exec ls --outDS user.hoge.{} --vo sphenix\".format(str(uuid.uuid4()))\n",
    "status, task_dict = c.execute_prun(com.split(), console_log=False)\n",
    "if status:\n",
    "    print (\"taskID={}\".format(task_dict['jediTaskID']))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "execute_prun(args, console_log=True)\n",
    "    Description: Execute prun command\n",
    "\n",
    "    args:\n",
    "        args: The arguments used to execute prun. This is a list of strings.\n",
    "        console_log: False to disable console logging\n",
    "\n",
    "    returns:\n",
    "        status: True if succeeded. Otherwise, False\n",
    "        a dictionary: Task submission attributes including jediTaskID\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Submit a pathena task**\n",
    "```\n",
    "execute_pathena(args, console_log=True)\n",
    "    Description: execute pathena command\n",
    "\n",
    "    args:\n",
    "        args: The arguments used to execute prun. This is a list of strings.\n",
    "        console_log: False to disable console logging\n",
    "\n",
    "    returns:\n",
    "        status: True if succeeded. Otherwise, False\n",
    "        a dictionary: Task submission attributes including jediTaskID\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Submit a phpo task**\n",
    "```\n",
    "execute_phpo(args, console_log=True)\n",
    "    Description: execute phpo command\n",
    "\n",
    "    args:\n",
    "        args: The arguments used to execute prun. This is a list of strings.\n",
    "        console_log: False to disable console logging\n",
    "\n",
    "    returns:\n",
    "        status: True if succeeded. Otherwise, False\n",
    "        a dictionary: Task submission attributes including jediTaskID\n",
    "```\n",
    "\n",
    "----\n",
    "\n",
    "<br>\n",
    "\n",
    "### Task management API\n",
    "\n",
    "* **Kill a task**"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "communication_status, o = c.kill_task(task_id)\n",
    "if communication_status:\n",
    "    server_return_code, dialog_message = o\n",
    "    if o == 0:\n",
    "        print('OK')\n",
    "    else:\n",
    "        print (\"Not good with {} : {}\".format(server_return_code, dialog_message))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "kill_task(taskID, verbose=False)\n",
    "    Description: kill a task\n",
    "\n",
    "    args:\n",
    "        jediTaskID: jediTaskID of the task to be killed\n",
    "    returns:\n",
    "        status code\n",
    "              0: communication succeeded to the panda server\n",
    "              255: communication failure\n",
    "        tuple of return code and diagnostic message\n",
    "              0: request is registered\n",
    "              1: server error\n",
    "              2: task not found\n",
    "              3: permission denied\n",
    "              4: irrelevant task status\n",
    "            100: non SSL connection\n",
    "            101: irrelevant taskID\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Finish a task**"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "communication_status, o = c.finish_task(task_id)\n",
    "if communication_status:\n",
    "    server_return_code, dialog_message = o\n",
    "    if o == 0:\n",
    "        print('OK')\n",
    "    else:\n",
    "        print (\"Not good with {} : {}\".format(server_return_code, dialog_message))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "finish_task(task_id, wait_running=False, verbose=False)\n",
    "    Description: finish a task\n",
    "\n",
    "    args:\n",
    "        task_id: jediTaskID of the task to finish\n",
    "        wait_running: True to wait until running jobs are done\n",
    "        verbose: True to see debug messages\n",
    "    returns:\n",
    "        status code\n",
    "              0: communication succeeded to the panda server\n",
    "              255: communication failure\n",
    "        tuple of return code and diagnostic message\n",
    "              0: request is registered\n",
    "              1: server error\n",
    "              2: task not found\n",
    "              3: permission denied\n",
    "              4: irrelevant task status\n",
    "            100: non SSL connection\n",
    "            101: irrelevant taskID\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Retry a task**"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "communication_status, o = c.retry_task(task_id)\n",
    "if communication_status:\n",
    "    server_return_code, dialog_message = o\n",
    "    if o == 0:\n",
    "        print('OK')\n",
    "    else:\n",
    "        print (\"Not good with {} : {}\".format(server_return_code, dialog_message))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "retry_task(task_id, new_parameters=Non, verbose=False)\n",
    "    Description: finish a task\n",
    "\n",
    "    args:\n",
    "        task_id: jediTaskID of the task to finish\n",
    "        new_parameters: a dictionary of task parameters to overwrite\n",
    "        verbose: True to see debug messages\n",
    "    returns:\n",
    "        status code\n",
    "              0: communication succeeded to the panda server\n",
    "              255: communication failure\n",
    "        tuple of return code and diagnostic message\n",
    "              0: request is registered\n",
    "              1: server error\n",
    "              2: task not found\n",
    "              3: permission denied\n",
    "              4: irrelevant task status\n",
    "            100: non SSL connection\n",
    "            101: irrelevant taskID\n",
    "```\n",
    "\n",
    "-----\n",
    "\n",
    "<br>\n",
    "\n",
    "### Bookkeeping API\n",
    "\n",
    "* **Get tasks**"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "tasks = c.get_tasks()\n",
    "for task in tasks:\n",
    "    print ('taskID={} status={}'.format(task['jeditaskid'], task['status']))"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "```\n",
    "get_tasks(self, task_ids, limit=1000, days=14, status=None, username=None)\n",
    "    Description: get a list of task dictionaries\n",
    "\n",
    "    args:\n",
    "       task_ids: a list of task IDs, or None to get recent tasks\n",
    "       limit: the max number of tasks to fetch from the server\n",
    "       days: tasks for last N days to fetch\n",
    "       status: filtering with task status\n",
    "       username: user name of the tasks, or None to get own tasks\n",
    "    returns:\n",
    "       a list of task dictionaries\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Show tasks**\n",
    "```\n",
    "show_tasks(self, task_ids, limit=1000, days=14, status=None, username=None)\n",
    "    Description: show tasks on the console\n",
    "\n",
    "    args:\n",
    "       task_ids: a list of task IDs, or None to get recent tasks\n",
    "       limit: the max number of tasks to fetch from the server\n",
    "       days: tasks for last N days to fetch\n",
    "       status: filtering with task status\n",
    "       username: user name of the tasks, or None to get own tasks\n",
    "    returns:\n",
    "       None\n",
    "```\n",
    "<br>\n",
    "\n",
    "* **Get metadata of all jobs in a task**\n",
    "```\n",
    "get_job_metadata(self, task_id, output_json_filename)\n",
    "    Description: get metadata of all jobs in a task\n",
    "\n",
    "    args:\n",
    "       task_id: task ID\n",
    "       output_json_filename: output json filename\n",
    "```"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}