Skip to content

Commit

Permalink
feat: add limit lk and llm and cron as variables
Browse files Browse the repository at this point in the history
  • Loading branch information
FlorentLvr committed May 3, 2024
1 parent be7110d commit aa782e0
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions __pipeline__.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@
"- `abi_spreadsheet`: Google Sheets spreadsheet URL\n",
"- `sheet_entity`: Entity sheet name that stores all your personal data\n",
"- `long_lived_token`: Long lived token to connect to Naas API.\n",
"- `limit_linkedin`: Max call on a specific LinkedIn endpoint (profile top card, company info).\n",
"- `limit_llm`: Max call on LLM (Naas API) to enrich specific data from Growth OBT (PEOPLE, ORGANIZATIONS, CONTACTS).\n",
"- `cron`: Represents the scheduling pattern of your notebook (https://crontab.guru/)\n",
"\n",
"**Outputs**\n",
"- `datalake_dir`: Datalake directory"
Expand All @@ -125,6 +128,9 @@
"long_lived_token = naas.secret.get('NAAS_API_TOKEN')\n",
"entity_start = 0\n",
"entity_end = None\n",
"limit_linkedin = 5\n",
"limit_llm = 5\n",
"cron = \"0 12 * * *\"\n",
"\n",
"# Outputs\n",
"datalake_dir = naas.secret.get(\"ABI_DATALAKE_DIR\")"
Expand Down Expand Up @@ -281,7 +287,9 @@
" \"linkedin_url\": linkedin_url,\n",
" \"li_at\": li_at,\n",
" \"JSESSIONID\": JSESSIONID,\n",
" \"entity_dir\": entity_dir\n",
" \"entity_dir\": entity_dir,\n",
" \"limit_linkedin\": limit_linkedin,\n",
" \"limit_llm\": limit_llm\n",
" }\n",
" )\n",
" sales = NotebookStep(\n",
Expand Down Expand Up @@ -361,7 +369,6 @@
"outputs": [],
"source": [
"# Schedule pipeline\n",
"cron = \"0 8 * * *\"\n",
"print(\"⏰ Scheduler:\", cron)\n",
"naas.scheduler.add(cron=cron)\n",
"# naas.scheduler.delete()"
Expand Down

0 comments on commit aa782e0

Please sign in to comment.