From 8833f5b085c5c49485d20e89514031f4b3d1a513 Mon Sep 17 00:00:00 2001 From: Liam Huber Date: Mon, 6 May 2024 16:08:06 -0700 Subject: [PATCH] [minor] Introduce for-loop (#309) * Change paradigm to whether or not the node uses __reduced__ and a constructor Instead of "Meta" nodes * Allow direct use of Constructed children * Move and update constructed stuff * Add new singleton behaviour so factory-produced classes can pass is-tests * PEP8 newline * Remove unnecessary __getstate__ The object isn't holding instance level state and older versions of python bork here. * Add constructed __*state__ compatibility for older versions * :bug: add missing `return` * Format black * Revert singleton * Remove constructed It's superceded by the snippets.factory stuff * Format black * Let the factory clear method take specific names * Don't override __module__ to the factory function If it was explicitly set downstream, leave that. But if the user left it empty, still default it back to the factory function's module * Clean up storage if job tests fail * Make tinybase the default storage backend * Switch Function and Macro over to using classfactory With this, everything is pickleable (unless you slap something unpickleable on top, or define it in a place that can't be reached by pickle like inside a local function scope). The big downside is that `h5io` storage is now basically useless, since all our nodes come from custom reconstructors. Similarly, for the node job `DataContainer` can no longer store the input node. The `tinybase` backend is still working ok, so I made it the default, and I got the node job working again by forcing it to cloudpickle the input node on saving. These are some ugly hacks, but since storage is an alpha feature right now anyhow, I'd prefer to push ahead with pickleability. * Remove unused decorator And reformat tests in the vein of usage in Function and Macro * Format black * Expose concurrent.futures executors on the creator * Only expose the base Executor from pympipool Doesn't hurt us now and prepares for the version bump * Extend `Runnable` to use a non-static method This is significant. `on_run` is no longer a property returning a staticmethod that will be shipped off, but we directly ship off `self.on_run` so `self` goes with it to remote processes. Similarly, `run_args` gets extended to be `tuple[tuple, dict]` so positional arguments can be sent too. Stacked on top of pickleability, this means we can now use standard `concurrent.futures.ProcessPoolExecutor` -- as long as the nodes are all defined somewhere importable, i.e. not in `__main__`. Since working in notebooks is pretty common, the more flexible `pympipool.Executor` is left as the default `Workflow.create.Executor`. This simplifies some stuff under the hood too, e.g. `Function` and `Composite` now just directly do their thing in `on_run` instead of needing the misdirection of returning their own static methods. * Format black * Expose concurrent.futures executors on the creator * Only expose the base Executor from pympipool Doesn't hurt us now and prepares for the version bump * Extend `Runnable` to use a non-static method This is significant. `on_run` is no longer a property returning a staticmethod that will be shipped off, but we directly ship off `self.on_run` so `self` goes with it to remote processes. Similarly, `run_args` gets extended to be `tuple[tuple, dict]` so positional arguments can be sent too. Stacked on top of pickleability, this means we can now use standard `concurrent.futures.ProcessPoolExecutor` -- as long as the nodes are all defined somewhere importable, i.e. not in `__main__`. Since working in notebooks is pretty common, the more flexible `pympipool.Executor` is left as the default `Workflow.create.Executor`. This simplifies some stuff under the hood too, e.g. `Function` and `Composite` now just directly do their thing in `on_run` instead of needing the misdirection of returning their own static methods. * Format black * Compute qualname if not provided * Fail early if there is a function in the factory made hierarchy * Skip the factory fanciness if you see This enables _FactoryMade objects to be cloudpickled, even when they can't be pickled, while still not letting the mere fact that they are dynamic classes stand in the way of pickling. Nicely lifts our constraint on the node job interaction with pyiron base, which was leveraging cloudpickle * Format black * Test ClassFactory this way too * Test existing list nodes * Rename length base class * Refactor transformers to use on_run and run_args more directly * Introduce an inputs-to-dict transformer * Preview IO as a separate step To guarantee IO construction happens as early as possible in case it fails * Add dataframe transformer * Remove prints :facepalm: * Add dataframe transformer tests * Add transformers to the create menu * Format black * :broom: be more consistent in caching/shortcuts Instead of always defining private holders by hand * Introduce a dataclass node * Give the dataclass node a simpler name Since we can inject attribute access, I don't anticipate ever needing the reverse dataclass-to-outputs node, so let's simplify the naming here. * Remove unused import * Set the output type hint automatically * Add docs * Add tests * Format black * PEP8 newline * Introduce for-loop * Refactor: break _build_body into smaller functions * Resolve dataframe column name conflicts When a body node has the same labels for looped input as for output * Update docstrings * Refactor: rename file * Don't break when one of iter or zipped is empty * :bug: pass body hint, not hint and default, to row collector input hint * Silence disconnection warning Since(?) disconnection is reciprocal it was firing left right and centre * Update deepdive * Remove old for loop * Remove unused import * Add tests * Remove unused attributes * Add a shortcut for assigning an executor to all body nodes * Format black * Format black --------- Co-authored-by: pyiron-runner --- notebooks/deepdive.ipynb | 760 +++++++++++++++++++++-------- pyiron_workflow/channels.py | 6 - pyiron_workflow/create.py | 10 +- pyiron_workflow/for_loop.py | 479 ++++++++++++++++++ pyiron_workflow/loops.py | 113 ----- tests/integration/test_workflow.py | 25 +- tests/unit/test_for_loop.py | 320 ++++++++++++ 7 files changed, 1364 insertions(+), 349 deletions(-) create mode 100644 pyiron_workflow/for_loop.py create mode 100644 tests/unit/test_for_loop.py diff --git a/notebooks/deepdive.ipynb b/notebooks/deepdive.ipynb index f1190821c..6484ad357 100644 --- a/notebooks/deepdive.ipynb +++ b/notebooks/deepdive.ipynb @@ -702,16 +702,6 @@ "tags": [] }, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel ran was not connected to run, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel run was not connected to ran, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "data": { "text/plain": [ @@ -1034,17 +1024,9 @@ "id": "6569014a-815b-46dd-8b47-4e1cd4584b3b", "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel run was not connected to ran, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGdCAYAAAA8F1jjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAlX0lEQVR4nO3df1Bc1f3/8dcCgSUW1pI0sPlRirGmQVpTyBBJTJ2qwURLjdOOtGkSY2OnSW3zqzpNmlYk0xlGa62/AvVH0NrEmDbqNJlSWmZslRhbKsGOiDY2oYXoIgOpC1YhZjmfP/Jlv24XEu4Cuwd4PmbuH3s4l/t+u0n25f1x1mWMMQIAALBQXKwLAAAAGAxBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgrYRYFzAUfX19evvtt5WSkiKXyxXrcgAAwBAYY9Td3a3p06crLi6ycyNjIqi8/fbbmjVrVqzLAAAAEWhtbdXMmTMj2ndMBJWUlBRJZxpNTU2NcTUAAGAourq6NGvWrODneCTGRFDpv9yTmppKUAEAYIwZzm0b3EwLAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFhrTCz4BkxkgT6juuaTau/u0bQUt/Kz0hQfx3deAZgYCCqAxaobfSo92CSfvyc45vW4VVKUraU53hhWBgDRwaUfwFLVjT6t330kJKRIUpu/R+t3H1F1oy9GlQFA9BBUAAsF+oxKDzbJDPCz/rHSg00K9A00AwDGD4IKYKG65pNhZ1I+ykjy+XtU13wyekUBQAwQVAALtXcPHlIimQcAYxVBBbDQtBT3iM4DgLGKoAJYKD8rTV6PW4M9hOzSmad/8rPSolkWAEQdQQWwUHycSyVF2ZIUFlb6X5cUZbOeCoBxj6ACWGppjlcVK3OV4Qm9vJPhcatiZS7rqACYEFjwDbDY0hyvlmRnsDItgAmLoAJYLj7OpYLZU2JdBgDEBJd+AACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKyVEOsCAACIhkCfUV3zSbV392hailv5WWmKj3PFuiycA0EFADDuVTf6VHqwST5/T3DM63GrpChbS3O8MawM5zJhL/0E+oxeOtap377yll461qlAn4l1SQCAUVDd6NP63UdCQooktfl7tH73EVU3+mJUGYZiQp5RIVkDwMQQ6DMqPdikgf5X1EhySSo92KQl2RlcBrLUhDujQrIGgImjrvlk2L/3H2Uk+fw9qms+Gb2i4MiECirnStbSmWTNZSAAGB/auwcPKZHMQ/RNqKBCsgaAiWVaintE5yH6JlRQIVkDwMSSn5Umr8etwe4+cenMPYr5WWnRLAsORBRUysvLlZWVJbfbrby8PNXW1p51/p49e3TJJZdo8uTJ8nq9uummm9TZ2RlRwcNBsgaAiSU+zqWSomxJCgsr/a9LirK5kdZijoPKvn37tGnTJm3fvl0NDQ1avHixli1bppaWlgHnHzp0SKtXr9batWv12muv6Te/+Y3+9re/6eabbx528U6RrAFg4lma41XFylxleEL/JzTD41bFylye9rScyxjj6M7RBQsWKDc3VxUVFcGxuXPnavny5SorKwubf/fdd6uiokLHjh0Ljj3wwAO666671NraOqRjdnV1yePxyO/3KzU11Um5Yfqf+pEUclNtf3jhDy0AjE+sTBt9I/H57eiMyqlTp1RfX6/CwsKQ8cLCQh0+fHjAfRYuXKgTJ06oqqpKxhi988472r9/v6699tpBj9Pb26uurq6QbaSQrAFgYoqPc6lg9hRdN2+GCmZPCQspLARqJ0cLvnV0dCgQCCg9PT1kPD09XW1tbQPus3DhQu3Zs0fFxcXq6enR6dOn9eUvf1kPPPDAoMcpKytTaWmpk9IcWZrj1ZLsDJI1AEASC4HaLKKbaV2u0A90Y0zYWL+mpiZt2LBBt99+u+rr61VdXa3m5matW7du0N+/bds2+f3+4DbUS0ROnCtZAwAmBhYCtZujMypTp05VfHx82NmT9vb2sLMs/crKyrRo0SLddtttkqTPfe5zOu+887R48WL95Cc/kdcbnlSTkpKUlJTkpDQAABxjiX37OTqjkpiYqLy8PNXU1ISM19TUaOHChQPu8/777ysuLvQw8fHxks6ciQEAIFZYCNR+ji/9bNmyRY8++qgqKyv1+uuva/PmzWppaQleytm2bZtWr14dnF9UVKRnnnlGFRUVOn78uF588UVt2LBB+fn5mj59+sh1AgCAQywEaj/H355cXFyszs5O7dixQz6fTzk5OaqqqlJmZqYkyefzhaypsmbNGnV3d+vBBx/U97//fZ1//vm64oordOedd45cFwAARICFQO3neB2VWBjJdVQAAOgX6DO67M7n1ObvGfA+FZfOLF9x6AdXcI9KBKK+jgoAAOMJS+zbj6ACAJjQWAjUbo7vUQEAYLxhIVB7EVQAAND/XwgUduHSDwAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUSYl0AAADjRaDPqK75pNq7ezQtxa38rDTFx7liXdaYRlABAGAEVDf6VHqwST5/T3DM63GrpChbS3O8MaxsbOPSDwAAw1Td6NP63UdCQooktfl7tH73EVU3+mJU2dhHUAEAYBgCfUalB5tkBvhZ/1jpwSYF+gaagXMhqAAAMAx1zSfDzqR8lJHk8/eorvlk9IoaRwgqAAAMQ3v34CElknkIRVABAGAYpqW4R3QeQhFUAAAYhvysNHk9bg32ELJLZ57+yc9Ki2ZZ4wZBBQCAYYiPc6mkKFuSwsJK/+uSomzWU4kQQQUAgGFamuNVxcpcZXhCL+9keNyqWJnLOirDwIJvAACMgKU5Xi3JzmBl2hFGUAEAYITEx7lUMHtKrMsYV7j0AwAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgrYiCSnl5ubKysuR2u5WXl6fa2tqzzu/t7dX27duVmZmppKQkzZ49W5WVlREVDAAAJg7H66js27dPmzZtUnl5uRYtWqSHHnpIy5YtU1NTkz75yU8OuM8NN9ygd955R7t27dKFF16o9vZ2nT59etjFAwCA8c1ljDFOdliwYIFyc3NVUVERHJs7d66WL1+usrKysPnV1dX62te+puPHjystLbIvZOrq6pLH45Hf71dqampEvwMAAETXSHx+O7r0c+rUKdXX16uwsDBkvLCwUIcPHx5wnwMHDmj+/Pm66667NGPGDF100UW69dZb9cEHHwx6nN7eXnV1dYVsAABg4nF06aejo0OBQEDp6ekh4+np6Wpraxtwn+PHj+vQoUNyu9169tln1dHRoe985zs6efLkoPeplJWVqbS01ElpAABgHIroZlqXK/QLlowxYWP9+vr65HK5tGfPHuXn5+uaa67RPffco8cff3zQsyrbtm2T3+8Pbq2trZGUCQAAxjhHZ1SmTp2q+Pj4sLMn7e3tYWdZ+nm9Xs2YMUMejyc4NnfuXBljdOLECX36058O2ycpKUlJSUlOSgMAAOOQozMqiYmJysvLU01NTch4TU2NFi5cOOA+ixYt0ttvv6333nsvOHb06FHFxcVp5syZEZQMAAAmCseXfrZs2aJHH31UlZWVev3117V582a1tLRo3bp1ks5ctlm9enVw/ooVKzRlyhTddNNNampq0gsvvKDbbrtN3/zmN5WcnDxynQAAgHHH8ToqxcXF6uzs1I4dO+Tz+ZSTk6OqqiplZmZKknw+n1paWoLzP/axj6mmpkbf+973NH/+fE2ZMkU33HCDfvKTn4xcFwAAYFxyvI5KLLCOCgAAY0/U11EBAACIJoIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1HC/4hokp0GdU13xS7d09mpbiVn5WmuLjBv4iSgAARgpBBedU3ehT6cEm+fw9wTGvx62SomwtzfHGsDIAwHjHpR+cVXWjT+t3HwkJKZLU5u/R+t1HVN3oi1FlAICJgKCCQQX6jEoPNmmg71joHys92KRAn/XfwgAAGKMIKhhUXfPJsDMpH2Uk+fw9qms+Gb2iAAATCkEFg2rvHjykRDIPAACnCCoY1LQU94jOAwDAKYIKBpWflSavx63BHkJ26czTP/lZadEsCwAwgRBUMKj4OJdKirIlKSys9L8uKcpmPRUAwKghqOCsluZ4VbEyVxme0Ms7GR63Klbmso4KAGBUseAbzmlpjldLsjNYmRYAEHUEFQxJfJxLBbOnxLoMAMAEw6UfAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgmxLgAAAMRGoM+orvmk2rt7NC3FrfysNMXHuWJdVoiIzqiUl5crKytLbrdbeXl5qq2tHdJ+L774ohISEjRv3rxIDgsAAEZIdaNPl935nL7+yF+08alX9PVH/qLL7nxO1Y2+WJcWwnFQ2bdvnzZt2qTt27eroaFBixcv1rJly9TS0nLW/fx+v1avXq0rr7wy4mIBAMDwVTf6tH73Efn8PSHjbf4erd99xKqw4jio3HPPPVq7dq1uvvlmzZ07V/fee69mzZqlioqKs+737W9/WytWrFBBQUHExQIAgOEJ9BmVHmySGeBn/WOlB5sU6BtoRvQ5CiqnTp1SfX29CgsLQ8YLCwt1+PDhQfd77LHHdOzYMZWUlAzpOL29verq6grZAADA8NU1nww7k/JRRpLP36O65pPRK+osHAWVjo4OBQIBpaenh4ynp6erra1twH3efPNNbd26VXv27FFCwtDu3S0rK5PH4wlus2bNclImAAAYRHv34CElknmjLaKbaV2u0DuCjTFhY5IUCAS0YsUKlZaW6qKLLhry79+2bZv8fn9wa21tjaRMAADwP6aluEd03mhz9Hjy1KlTFR8fH3b2pL29PewsiyR1d3fr5ZdfVkNDg7773e9Kkvr6+mSMUUJCgv74xz/qiiuuCNsvKSlJSUlJTkoDAOCsxsKjuNGQn5Umr8etNn/PgPepuCRleM7897GBo6CSmJiovLw81dTU6Prrrw+O19TU6Lrrrgubn5qaqldffTVkrLy8XM8995z279+vrKysCMsGAGDoqht9Kj3YFHJvhtfjVklRtpbmeGNYWfTFx7lUUpSt9buPyCWFhJX+2FZSlG1NiHO84NuWLVu0atUqzZ8/XwUFBXr44YfV0tKidevWSTpz2eatt97SE088obi4OOXk5ITsP23aNLnd7rBxAABGQ/+juP979qD/UdyKlbkTLqwszfGqYmVuWHjLsDC8OQ4qxcXF6uzs1I4dO+Tz+ZSTk6OqqiplZmZKknw+3znXVAEAIBrO9SiuS2cexV2SnWHNGYRoWZrj1ZLsDOsvh7mMMXY8KH0WXV1d8ng88vv9Sk1NjXU5AIAx4qVjnfr6I38557y937pUBbOnRKGiiWUkPr/5UkIAwLg11h7FRTiCCgBg3Bprj+IiHEEFADBu9T+KO9hdFy6defrHlkdxEY6gAgAYt/ofxZUUFlZsfBQX4QgqAIBxrf9R3AxP6OWdDI97Qj6aPNY4fjwZAICxZqw8iotwBBUAwIQQH+fiEeQxiEs/AADAWgQVAABgLYIKAACwFkEFAABYi5tpMaBAn+HueABAzBFUEKa60Rf21d9eC7/6GwAw/nHpByGqG31av/tISEiRpDZ/j9bvPqLqRl+MKgMATEQEFQQF+oxKDzbJDPCz/rHSg00K9A00AwCAkUdQQVBd88mwMykfZST5/D2qaz4ZvaIAABMaQQVB7d2Dh5RI5gEAMFwEFQRNS3Gfe5KDeQAADBdBBUH5WWnyetxhX4Xez6UzT//kZ6VFsywAwARGUEFQfJxLJUXZkhQWVvpflxRls54KACBqCCoIsTTHq4qVucrwhF7eyfC4VbEyl3VUAABRxYJvCLM0x6sl2RmsTAsAiDmCCgYUH+dSwewpsS4DADDBcekHAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtSIKKuXl5crKypLb7VZeXp5qa2sHnfvMM89oyZIl+sQnPqHU1FQVFBToD3/4Q8QFAwCAicNxUNm3b582bdqk7du3q6GhQYsXL9ayZcvU0tIy4PwXXnhBS5YsUVVVlerr6/XFL35RRUVFamhoGHbxAABgfHMZY4yTHRYsWKDc3FxVVFQEx+bOnavly5errKxsSL/j4osvVnFxsW6//fYhze/q6pLH45Hf71dqaqqTcgEAQIyMxOe3ozMqp06dUn19vQoLC0PGCwsLdfjw4SH9jr6+PnV3dystLc3JoQEAwASU4GRyR0eHAoGA0tPTQ8bT09PV1tY2pN/xs5/9TP/97391ww03DDqnt7dXvb29wdddXV1OygQAAONERDfTulyukNfGmLCxgezdu1d33HGH9u3bp2nTpg06r6ysTB6PJ7jNmjUrkjIBAMAY5yioTJ06VfHx8WFnT9rb28POsvyvffv2ae3atfr1r3+tq6666qxzt23bJr/fH9xaW1udlAkAAMYJR0ElMTFReXl5qqmpCRmvqanRwoULB91v7969WrNmjZ588klde+215zxOUlKSUlNTQzYAADDxOLpHRZK2bNmiVatWaf78+SooKNDDDz+slpYWrVu3TtKZsyFvvfWWnnjiCUlnQsrq1at133336dJLLw2ejUlOTpbH4xnBVgAAwHjjOKgUFxers7NTO3bskM/nU05OjqqqqpSZmSlJ8vl8IWuqPPTQQzp9+rRuueUW3XLLLcHxG2+8UY8//vjwOwAAAOOW43VUYoF1VAAAGHuivo4KAABANBFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYi6ACAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLUIKgAAwFoEFQAAYC2CCgAAsBZBBQAAWIugAgAArEVQAQAA1iKoAAAAaxFUAACAtQgqAADAWgQVAABgLYIKAACwFkEFAABYKyHWBQC2C/QZ1TWfVHt3j6aluJWflab4OFesywKACYGgApxFdaNPpQeb5PP3BMe8HrdKirK1NMcbw8oAYGLg0g8wiOpGn9bvPhISUiSpzd+j9buPqLrRF6PKAGDiIKgAAwj0GZUebJIZ4Gf9Y6UHmxToG2gGAGCkEFSAAdQ1nww7k/JRRpLP36O65pPRKwoAJiCCCjCA9u7BQ0ok8wAAkSGoAAOYluIe0XkAgMgQVIAB5Gelyetxa7CHkF068/RPflZaNMsCgAmHoAIMID7OpZKibEkKCyv9r0uKsllPBQBGGUEFGMTSHK8qVuYqwxN6eSfD41bFylzWUQGAKGDBN+AsluZ4tSQ7g5VpASBGCCrAOcTHuVQwe0qsywCACYlLPwAAwFoEFQAAYK2Igkp5ebmysrLkdruVl5en2tras85//vnnlZeXJ7fbrQsuuEC/+MUvIioWAABMLI6Dyr59+7Rp0yZt375dDQ0NWrx4sZYtW6aWlpYB5zc3N+uaa67R4sWL1dDQoB/+8IfasGGDnn766WEXDwAAxjeXMcbRt6otWLBAubm5qqioCI7NnTtXy5cvV1lZWdj8H/zgBzpw4IBef/314Ni6dev097//XS+99NKQjtnV1SWPxyO/36/U1FQn5QIAgBgZic9vR2dUTp06pfr6ehUWFoaMFxYW6vDhwwPu89JLL4XNv/rqq/Xyyy/rww8/dFguAACYSBw9ntzR0aFAIKD09PSQ8fT0dLW1tQ24T1tb24DzT58+rY6ODnm94Ytm9fb2qre3N/i6q6vLSZkAAGCciOhmWpcrdLErY0zY2LnmDzTer6ysTB6PJ7jNmjUrkjIBAMAY5yioTJ06VfHx8WFnT9rb28POmvTLyMgYcH5CQoKmTBl4Ea1t27bJ7/cHt9bWVidlAgCAccLRpZ/ExETl5eWppqZG119/fXC8pqZG11133YD7FBQU6ODBgyFjf/zjHzV//nxNmjRpwH2SkpKUlJQUfN1/BoZLQAAAjB39n9sOn9sJZRx66qmnzKRJk8yuXbtMU1OT2bRpkznvvPPMv/71L2OMMVu3bjWrVq0Kzj9+/LiZPHmy2bx5s2lqajK7du0ykyZNMvv37x/yMVtbW40kNjY2NjY2tjG4tba2Oo0bQY6/66e4uFidnZ3asWOHfD6fcnJyVFVVpczMTEmSz+cLWVMlKytLVVVV2rx5s3bu3Knp06fr/vvv11e+8pUhH3P69OlqbW1VSkrKWe+FsUFXV5dmzZql1tbWcfkoNf2NbfQ3ttHf2DYR+zPGqLu7W9OnT4/49zpeRwVnN97XfKG/sY3+xjb6G9voLzJ81w8AALAWQQUAAFiLoDLCkpKSVFJSEvLU0nhCf2Mb/Y1t9De20V9kuEcFAABYizMqAADAWgQVAABgLYIKAACwFkEFAABYi6ASgfLycmVlZcntdisvL0+1tbWDzvX5fFqxYoXmzJmjuLg4bdq0KXqFRshJf88884yWLFmiT3ziE0pNTVVBQYH+8Ic/RLFa55z0d+jQIS1atEhTpkxRcnKyPvOZz+jnP/95FKt1zkl/H/Xiiy8qISFB8+bNG90Ch8lJf3/+85/lcrnCtjfeeCOKFTvj9P3r7e3V9u3blZmZqaSkJM2ePVuVlZVRqtY5J/2tWbNmwPfv4osvjmLFzjh9//bs2aNLLrlEkydPltfr1U033aTOzs4oVeuc0/527typuXPnKjk5WXPmzNETTzzh/KARL74/QfV/19EjjzximpqazMaNG815551n/v3vfw84v7m52WzYsMH88pe/NPPmzTMbN26MbsEOOe1v48aN5s477zR1dXXm6NGjZtu2bWbSpEnmyJEjUa58aJz2d+TIEfPkk0+axsZG09zcbH71q1+ZyZMnm4ceeijKlQ+N0/76vfvuu+aCCy4whYWF5pJLLolOsRFw2t+f/vQnI8n84x//MD6fL7idPn06ypUPTSTv35e//GWzYMECU1NTY5qbm81f//pX8+KLL0ax6qFz2t+7774b8r61traatLQ0U1JSEt3Ch8hpf7W1tSYuLs7cd9995vjx46a2ttZcfPHFZvny5VGufGic9ldeXm5SUlLMU089ZY4dO2b27t1rPvaxj5kDBw44Oi5BxaH8/Hyzbt26kLHPfOYzZuvWrefc9/LLL7c+qAynv37Z2dmmtLR0pEsbESPR3/XXX29Wrlw50qWNiEj7Ky4uNj/60Y9MSUmJ1UHFaX/9QeU///lPFKobPqf9/f73vzcej8d0dnZGo7xhG+7fv2effda4XK7gl+Daxml/P/3pT80FF1wQMnb//febmTNnjlqNw+G0v4KCAnPrrbeGjG3cuNEsWrTI0XG59OPAqVOnVF9fr8LCwpDxwsJCHT58OEZVjZyR6K+vr0/d3d1KS0sbjRKHZST6a2ho0OHDh3X55ZePRonDEml/jz32mI4dO6aSkpLRLnFYhvP+ff7zn5fX69WVV16pP/3pT6NZZsQi6e/AgQOaP3++7rrrLs2YMUMXXXSRbr31Vn3wwQfRKNmRkfj7t2vXLl111VXBL8G1SST9LVy4UCdOnFBVVZWMMXrnnXe0f/9+XXvttdEo2ZFI+uvt7ZXb7Q4ZS05OVl1dnT788MMhH5ug4kBHR4cCgYDS09NDxtPT09XW1hajqkbOSPT3s5/9TP/97391ww03jEaJwzKc/mbOnKmkpCTNnz9ft9xyi26++ebRLDUikfT35ptvauvWrdqzZ48SEhx/mXpURdKf1+vVww8/rKefflrPPPOM5syZoyuvvFIvvPBCNEp2JJL+jh8/rkOHDqmxsVHPPvus7r33Xu3fv1+33HJLNEp2ZLj/vvh8Pv3+97+38u+eFFl/Cxcu1J49e1RcXKzExERlZGTo/PPP1wMPPBCNkh2JpL+rr75ajz76qOrr62WM0csvv6zKykp9+OGH6ujoGPKx7f6XyVIulyvktTEmbGwsi7S/vXv36o477tBvf/tbTZs2bbTKG7ZI+qutrdV7772nv/zlL9q6dasuvPBCff3rXx/NMiM21P4CgYBWrFih0tJSXXTRRdEqb9icvH9z5szRnDlzgq8LCgrU2tqqu+++W1/4whdGtc5IOemvr69PLpdLe/bskcfjkSTdc889+upXv6qdO3cqOTl51Ot1KtJ/Xx5//HGdf/75Wr58+ShVNjKc9NfU1KQNGzbo9ttv19VXXy2fz6fbbrtN69at065du6JRrmNO+vvxj3+strY2XXrppTLGKD09XWvWrNFdd92l+Pj4IR+TMyoOTJ06VfHx8WHpsb29PSxljkXD6W/fvn1au3atfv3rX+uqq64azTIjNpz+srKy9NnPflbf+ta3tHnzZt1xxx2jWGlknPbX3d2tl19+Wd/97neVkJCghIQE7dixQ3//+9+VkJCg5557LlqlD8lI/f279NJL9eabb450ecMWSX9er1czZswIhhRJmjt3rowxOnHixKjW69Rw3j9jjCorK7Vq1SolJiaOZpkRi6S/srIyLVq0SLfddps+97nP6eqrr1Z5ebkqKyvl8/miUfaQRdJfcnKyKisr9f777+tf//qXWlpa9KlPfUopKSmaOnXqkI9NUHEgMTFReXl5qqmpCRmvqanRwoULY1TVyIm0v71792rNmjV68sknrby22m+k3j9jjHp7e0e6vGFz2l9qaqpeffVVvfLKK8Ft3bp1mjNnjl555RUtWLAgWqUPyUi9fw0NDfJ6vSNd3rBF0t+iRYv09ttv67333guOHT16VHFxcZo5c+ao1uvUcN6/559/Xv/85z+1du3a0SxxWCLp7/3331dcXOjHcP+ZBmPZ1/AN5/2bNGmSZs6cqfj4eD311FP60pe+FNb3WTm69RbBx7N27dplmpqazKZNm8x5550XvAt969atZtWqVSH7NDQ0mIaGBpOXl2dWrFhhGhoazGuvvRaL8s/JaX9PPvmkSUhIMDt37gx5jPDdd9+NVQtn5bS/Bx980Bw4cMAcPXrUHD161FRWVprU1FSzffv2WLVwVpH8+fwo25/6cdrfz3/+c/Pss8+ao0ePmsbGRrN161YjyTz99NOxauGsnPbX3d1tZs6cab761a+a1157zTz//PPm05/+tLn55ptj1cJZRfrnc+XKlWbBggXRLtcxp/099thjJiEhwZSXl5tjx46ZQ4cOmfnz55v8/PxYtXBWTvv7xz/+YX71q1+Zo0ePmr/+9a+muLjYpKWlmebmZkfHJahEYOfOnSYzM9MkJiaa3Nxc8/zzzwd/duONN5rLL788ZL6ksC0zMzO6RTvgpL/LL798wP5uvPHG6Bc+RE76u//++83FF19sJk+ebFJTU83nP/95U15ebgKBQAwqHxqnfz4/yvagYoyz/u68804ze/Zs43a7zcc//nFz2WWXmd/97ncxqHronL5/r7/+urnqqqtMcnKymTlzptmyZYt5//33o1z10Dnt79133zXJycnm4YcfjnKlkXHa3/3332+ys7NNcnKy8Xq95hvf+IY5ceJElKseOif9NTU1mXnz5pnk5GSTmppqrrvuOvPGG284PqbLGMvOLwEAAPw/3KMCAACsRVABAADWIqgAAABrEVQAAIC1CCoAAMBaBBUAAGAtggoAALAWQQUAAFiLoAIAAKxFUAEAANYiqAAAAGsRVAAAgLX+D1GbPy8dq/V2AAAAAElFTkSuQmCC", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAlLElEQVR4nO3df1Bc1R338c8CwqLCWhKBVShiVAzBVoEhIal12hhMtLRppyM1TfzRpCNpbJNmdCaZWJFMZ6jWxtQ2UKNGGxNjOo1OzTRimakaYh5lTEiniBobacFkkUlSF6yF6HKeP/LA4xZIuMuyZ2Hfr5n7xx7O3f1+Z0P4zP1xrssYYwQAAGBJnO0CAABAbCOMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALAqwXYBo9Hf369jx44pJSVFLpfLdjkAAGAUjDHq6enRRRddpLi4kY9/TIgwcuzYMWVnZ9suAwAAhKCjo0NZWVkj/nxChJGUlBRJp5tJTU21XA0AABiN7u5uZWdnD/4dH8mECCMDp2ZSU1MJIwAATDBnu8SCC1gBAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVk2IRc8AAIgVgX6jpraT6urpVXqKWyW5aYqPm9zPZSOMAAAQJepbfKre3Sqfv3dwzOtxq6o8X/MLvBYrG1+cpgEAIArUt/i0fNvBoCAiSZ3+Xi3fdlD1LT5LlY0/wggAAJYF+o2qd7fKDPOzgbHq3a0K9A83Y+IjjAAAYFlT28khR0Q+z0jy+XvV1HYyckVFEGEEAADLunpGDiKhzJtoCCMAAFiWnuIO67yJhjACAIBlJblp8nrcGukGXpdO31VTkpsWybIihjACAIBl8XEuVZXnS9KQQDLwuqo8f9KuN0IYAQAgCswv8KpucaEyPcGnYjI9btUtLpzU64yw6BkAAFFifoFX8/IzWYEVAADYEx/nUum0KbbLiChO0wAAAKs4MgIAiCmx+CC6aEcYAQDEjFh9EF204zQNACAmxPKD6KIdYQQAMOnF+oPooh1hBAAw6cX6g+iiHWEEADDpxfqD6KIdYQQAMOnF+oPooh1hBAAw6cX6g+iiHWEEADDpxfqD6KIdYQQAEBNi+UF00Y5FzwAAMSNWH0QX7QgjAICYEosPoot2nKYBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFZxNw3GTaDfcPscAOCsCCMYF/UtPlXvbg16SqbX41ZVeT4LCwEAgnCaBmFX3+LT8m0Hhzyuu9Pfq+XbDqq+xWepMgBANCKMIKwC/UbVu1tlhvnZwFj17lYF+oebAQCIRYQRhFVT28khR0Q+z0jy+XvV1HYyckUBAKIaYQRh1dUzchAJZR4AYPIjjCCs0lPcZ5/kYB4AYPIjjCCsSnLT5PW4NdINvC6dvqumJDctkmUBAKIYYQRhFR/nUlV5viQNCSQDr6vK81lvBAAwiDCCsJtf4FXd4kJleoJPxWR63KpbXMg6IwCAICx6hnExv8CrefmZrMAKADgrwgjGTXycS6XTptguAwAQ5ThNAwAArCKMAAAAq0IKI7W1tcrNzZXb7VZRUZEaGxvPOL+vr0/r1q1TTk6OkpKSNG3aNG3ZsiWkggEAwOTi+JqRnTt3atWqVaqtrdWcOXP06KOPasGCBWptbdUXv/jFYfe5+eab9eGHH+qJJ57QZZddpq6uLn322WdjLh4AAEx8LmOMoyeWzZw5U4WFhaqrqxscmz59uhYuXKiampoh8+vr6/W9731P77//vtLSQlvoqru7Wx6PR36/X6mpqSG9BwAAiKzR/v12dJrm1KlTOnDggMrKyoLGy8rKtH///mH3eeGFF1RcXKwHH3xQF198sa644grdfffd+u9//zvi5/T19am7uztoAwAAk5Oj0zTHjx9XIBBQRkZG0HhGRoY6OzuH3ef999/Xvn375Ha79fzzz+v48eP60Y9+pJMnT4543UhNTY2qq6udlAYAACaokC5gdbmCF64yxgwZG9Df3y+Xy6Xt27erpKREN954ozZs2KCnnnpqxKMja9euld/vH9w6OjpCKRMAAEwAjo6MTJ06VfHx8UOOgnR1dQ05WjLA6/Xq4osvlsfjGRybPn26jDH64IMPdPnllw/ZJykpSUlJSU5KAwAAE5SjIyOJiYkqKipSQ0ND0HhDQ4Nmz5497D5z5szRsWPH9PHHHw+OHT58WHFxccrKygqhZAAAMJk4Pk2zevVqPf7449qyZYvefvtt/fSnP1V7e7sqKyslnT7Fcuuttw7OX7RokaZMmaI77rhDra2t2rt3r+655x794Ac/UHJycvg6AQAAE5LjdUYqKip04sQJrV+/Xj6fTwUFBdqzZ49ycnIkST6fT+3t7YPzzz//fDU0NOjHP/6xiouLNWXKFN188836+c9/Hr4uAADAhOV4nREbWGcEAICJZ1zWGQEAAAg3wggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALAqpDBSW1ur3Nxcud1uFRUVqbGxccS5r7zyilwu15DtnXfeCbloAAAweTgOIzt37tSqVau0bt06NTc369prr9WCBQvU3t5+xv3effdd+Xy+we3yyy8PuWgAADB5OA4jGzZs0NKlS7Vs2TJNnz5dGzduVHZ2turq6s64X3p6ujIzMwe3+Pj4kIsGAACTh6MwcurUKR04cEBlZWVB42VlZdq/f/8Z973mmmvk9Xo1d+5cvfzyy2ec29fXp+7u7qANAABMTo7CyPHjxxUIBJSRkRE0npGRoc7OzmH38Xq92rx5s3bt2qXnnntOeXl5mjt3rvbu3Tvi59TU1Mjj8Qxu2dnZTsoEAAATSEIoO7lcrqDXxpghYwPy8vKUl5c3+Lq0tFQdHR166KGH9NWvfnXYfdauXavVq1cPvu7u7iaQAAAwSTk6MjJ16lTFx8cPOQrS1dU15GjJmcyaNUvvvffeiD9PSkpSampq0AYAACYnR2EkMTFRRUVFamhoCBpvaGjQ7NmzR/0+zc3N8nq9Tj4aAABMUo5P06xevVpLlixRcXGxSktLtXnzZrW3t6uyslLS6VMsR48e1datWyVJGzdu1CWXXKIZM2bo1KlT2rZtm3bt2qVdu3aFtxMAADAhOQ4jFRUVOnHihNavXy+fz6eCggLt2bNHOTk5kiSfzxe05sipU6d099136+jRo0pOTtaMGTP05z//WTfeeGP4ugAAABOWyxhjbBdxNt3d3fJ4PPL7/Vw/AgDABDHav988mwYAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABY5Xg5eAAAEBmBfqOmtpPq6ulVeopbJblpio9z2S4r7AgjAABEofoWn6p3t8rn7x0c83rcqirP1/yCyfXke07TAAAQZepbfFq+7WBQEJGkTn+vlm87qPoWn6XKxgdhBACAKBLoN6re3arhnmI7MFa9u1WB/qh/zu2oEUYAAIgiTW0nhxwR+TwjyefvVVPbycgVNc4IIwAARJGunpGDSCjzJgLCCAAAUSQ9xR3WeRMBYQQAgChSkpsmr8etkW7gden0XTUluWmRLGtcEUYAAIgi8XEuVZXnS9KQQDLwuqo8f1KtN0IYAQAgyswv8KpucaEyPcGnYjI9btUtLpx064yw6BkAAFFofoFX8/IzWYEVAADYEx/nUum0KbbLGHecpgEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAViXYLgAAANgR6Ddqajuprp5epae4VZKbpvg4V8TrCOnISG1trXJzc+V2u1VUVKTGxsZR7ffaa68pISFBV199dSgfCwAAwqS+xaevPPBX3fLY61r57CHd8tjr+soDf1V9iy/itTgOIzt37tSqVau0bt06NTc369prr9WCBQvU3t5+xv38fr9uvfVWzZ07N+RiAQDA2NW3+LR820H5/L1B453+Xi3fdjDigcRxGNmwYYOWLl2qZcuWafr06dq4caOys7NVV1d3xv3uvPNOLVq0SKWlpSEXCwAAxibQb1S9u1VmmJ8NjFXvblWgf7gZ48NRGDl16pQOHDigsrKyoPGysjLt379/xP2efPJJHTlyRFVVVaP6nL6+PnV3dwdtAABg7JraTg45IvJ5RpLP36umtpMRq8lRGDl+/LgCgYAyMjKCxjMyMtTZ2TnsPu+9957WrFmj7du3KyFhdNfL1tTUyOPxDG7Z2dlOygQAACPo6hk5iIQyLxxCuoDV5Qq+0tYYM2RMkgKBgBYtWqTq6mpdccUVo37/tWvXyu/3D24dHR2hlAkAAP5Heoo7rPPCwdGtvVOnTlV8fPyQoyBdXV1DjpZIUk9Pj9588001NzfrrrvukiT19/fLGKOEhAT95S9/0de//vUh+yUlJSkpKclJaQAAYBRKctPk9bjV6e8d9roRl6RMz+nbfCPF0ZGRxMREFRUVqaGhIWi8oaFBs2fPHjI/NTVVf//733Xo0KHBrbKyUnl5eTp06JBmzpw5tuoBAIAj8XEuVZXnSzodPD5v4HVVeX5E1xtxvOjZ6tWrtWTJEhUXF6u0tFSbN29We3u7KisrJZ0+xXL06FFt3bpVcXFxKigoCNo/PT1dbrd7yDgAAIiM+QVe1S0uVPXu1qCLWTM9blWV52t+gTei9TgOIxUVFTpx4oTWr18vn8+ngoIC7dmzRzk5OZIkn8931jVHEL2iZTU+AMD4ml/g1bz8zKj4P99ljIncjcQh6u7ulsfjkd/vV2pqqu1yJq36Ft+QlOy1lJIBABPfaP9+86A8SIq+1fgAALGDMIKoXI0PABA7CCOIytX4AACxgzCCqFyNDwAQOwgjiMrV+AAAsYMwgsHV+Ea6mcul03fVRHI1PgBA7CCMICpX4wMAxA7CCCT9/9X4Mj3Bp2IyPW7VLS5knREAwLhxvAIrJq9oWo0PABA7CCMIEh/nUum0KbbLAADEEE7TAAAAqwgjAADAKsIIAACwijACAACs4gJWAEDMCfQb7hyMIoQRAEBMqW/xqXp3a9ADQr0et6rK81lTyRJO0wAAYkZ9i0/Ltx0c8qTyTn+vlm87qPoWn6XKYhthBAAQEwL9RtW7W2WG+dnAWPXuVgX6h5uB8UQYAQDEhKa2k0OOiHyekeTz96qp7WTkioIkwggAIEZ09YwcREKZh/AhjAAAYkJ6ivvskxzMQ/gQRgAAMaEkN01ej1sj3cDr0um7akpy0yJZFkQYAQDEiPg4l6rK8yVpSCAZeF1Vns96IxYQRgAAMWN+gVd1iwuV6Qk+FZPpcatucSHrjFjComcAgJgyv8CrefmZrMAaRQgjAICYEx/nUum0KbbLwP/DaRoAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWEUYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFUJtguwJdBv1NR2Ul09vUpPcaskN03xcS7bZQEAEHNiMozUt/hUvbtVPn/v4JjX41ZVeb7mF3gtVgYAQOyJudM09S0+Ld92MCiISFKnv1fLtx1UfYvPUmUAAMSmmAojgX6j6t2tMsP8bGCsenerAv3DzQAAAOMhpsJIU9vJIUdEPs9I8vl71dR2MnJFAQAQ42IqjHT1jBxEQpkHAADGLqbCSHqKO6zzAADA2MVUGCnJTZPX49ZIN/C6dPqumpLctEiWBQBATAspjNTW1io3N1dut1tFRUVqbGwcce6+ffs0Z84cTZkyRcnJybryyiv18MMPh1zwWMTHuVRVni9JQwLJwOuq8nzWGwEAIIIch5GdO3dq1apVWrdunZqbm3XttddqwYIFam9vH3b+eeedp7vuukt79+7V22+/rXvvvVf33nuvNm/ePObiQzG/wKu6xYXK9ASfisn0uFW3uJB1RgAAiDCXMcbRfawzZ85UYWGh6urqBsemT5+uhQsXqqamZlTv8Z3vfEfnnXeenn766VHN7+7ulsfjkd/vV2pqqpNyR8QKrAAAjK/R/v12tALrqVOndODAAa1ZsyZovKysTPv37x/VezQ3N2v//v36+c9/PuKcvr4+9fX1Db7u7u52UuaoxMe5VDptStjfFwAAOOPoNM3x48cVCASUkZERNJ6RkaHOzs4z7puVlaWkpCQVFxdrxYoVWrZs2Yhza2pq5PF4Brfs7GwnZQIAgAkkpAtYXa7g0xnGmCFj/6uxsVFvvvmmfve732njxo3asWPHiHPXrl0rv98/uHV0dIRSJgAAmAAcnaaZOnWq4uPjhxwF6erqGnK05H/l5uZKkq666ip9+OGHuv/++3XLLbcMOzcpKUlJSUlOSgMAABOUoyMjiYmJKioqUkNDQ9B4Q0ODZs+ePer3McYEXRMCAABil6MjI5K0evVqLVmyRMXFxSotLdXmzZvV3t6uyspKSadPsRw9elRbt26VJG3atElf/OIXdeWVV0o6ve7IQw89pB//+MdhbAMAAExUjsNIRUWFTpw4ofXr18vn86mgoEB79uxRTk6OJMnn8wWtOdLf36+1a9eqra1NCQkJmjZtmn7xi1/ozjvvDF8XAABgwnK8zogN47HOCAAAGF+j/fsdU8+mAQAA0YcwAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCKMAIAAKwijAAAAKsIIwAAwCrCCAAAsIowAgAArCKMAAAAqxJsFwAAZxPoN2pqO6munl6lp7hVkpum+DiX7bIAhAlhBEBUq2/xqXp3q3z+3sExr8etqvJ8zS/wWqwMQLhwmgZA1Kpv8Wn5toNBQUSSOv29Wr7toOpbfJYqAxBOhBEAUSnQb1S9u1VmmJ8NjFXvblWgf7gZACYSwgiAqNTUdnLIEZHPM5J8/l41tZ2MXFEAxgVhBEBU6uoZOYiEMg9A9CKMAIhK6SnusM4DEL0IIwCiUklumrwet0a6gdel03fVlOSmRbIsAOOAMAIgKsXHuVRVni9JQwLJwOuq8nzWGwEmAcIIgKg1v8CrusWFyvQEn4rJ9LhVt7iQdUaASYJFzwBEtfkFXs3Lz2QFVmASI4wAiHrxcS6VTptiuwwA44TTNAAAwCrCCAAAsIowAgAArCKMAAAAqwgjAADAKsIIAACwijACAACsIowAAACrCCMAAMAqwggAALCK5eABxJxAv+FZN0AUIYwAiCn1LT5V726Vz987OOb1uFVVns9TgAFLQjpNU1tbq9zcXLndbhUVFamxsXHEuc8995zmzZunCy+8UKmpqSotLdVLL70UcsEAEKr6Fp+WbzsYFEQkqdPfq+XbDqq+xWepMiC2OQ4jO3fu1KpVq7Ru3To1Nzfr2muv1YIFC9Te3j7s/L1792revHnas2ePDhw4oK997WsqLy9Xc3PzmIsHgNEK9BtV726VGeZnA2PVu1sV6B9uBoDx5DLGOPrNmzlzpgoLC1VXVzc4Nn36dC1cuFA1NTWjeo8ZM2aooqJC991336jmd3d3y+PxyO/3KzU11Um5ACBJ+j9HTuiWx14/67wdP5yl0mlTIlARMPmN9u+3oyMjp06d0oEDB1RWVhY0XlZWpv3794/qPfr7+9XT06O0tLQR5/T19am7uztoA4Cx6OrpPfskB/MAhI+jMHL8+HEFAgFlZGQEjWdkZKizs3NU7/GrX/1K//nPf3TzzTePOKempkYej2dwy87OdlImAAyRnuIO6zwA4RPSBawuV/AtcMaYIWPD2bFjh+6//37t3LlT6enpI85bu3at/H7/4NbR0RFKmQAwqCQ3TV6PWyP9T+XS6btqSnJHPmoLYHw4CiNTp05VfHz8kKMgXV1dQ46W/K+dO3dq6dKl+sMf/qDrr7/+jHOTkpKUmpoatAHAWMTHuVRVni9JQwLJwOuq8nzWGwEscBRGEhMTVVRUpIaGhqDxhoYGzZ49e8T9duzYodtvv13PPPOMbrrpptAqBYAxml/gVd3iQmV6gk/FZHrcqltcyDojgCWOFz1bvXq1lixZouLiYpWWlmrz5s1qb29XZWWlpNOnWI4ePaqtW7dKOh1Ebr31Vv3617/WrFmzBo+qJCcny+PxhLEVADi7+QVezcvPZAVWIIo4DiMVFRU6ceKE1q9fL5/Pp4KCAu3Zs0c5OTmSJJ/PF7TmyKOPPqrPPvtMK1as0IoVKwbHb7vtNj311FNj7wAAHIqPc3H7LhBFHK8zYgPrjAAAMPGMyzojAAAA4UYYAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGCV4xVYbRhYl627u9tyJQAAYLQG/m6fbX3VCRFGenp6JEnZ2dmWKwEAAE719PSc8Xl0E2I5+P7+fh07dkwpKSlyuaLrYVbd3d3Kzs5WR0fHpFuqnt4mrsnc32TuTZrc/dHbxBVqf8YY9fT06KKLLlJc3MhXhkyIIyNxcXHKysqyXcYZpaamTsp/gBK9TWSTub/J3Js0ufujt4krlP7OdERkABewAgAAqwgjAADAKsLIGCUlJamqqkpJSUm2Swk7epu4JnN/k7k3aXL3R28T13j3NyEuYAUAAJMXR0YAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRs6itrZWubm5crvdKioqUmNj44hzfT6fFi1apLy8PMXFxWnVqlWRKzRETvp77rnnNG/ePF144YVKTU1VaWmpXnrppQhW64yT3vbt26c5c+ZoypQpSk5O1pVXXqmHH344gtU656S/z3vttdeUkJCgq6++enwLHAMnvb3yyityuVxDtnfeeSeCFY+e0++tr69P69atU05OjpKSkjRt2jRt2bIlQtU656S/22+/fdjvbsaMGRGsePScfnfbt2/Xl7/8ZZ177rnyer264447dOLEiQhV65zT/jZt2qTp06crOTlZeXl52rp1a+gfbjCiZ5991pxzzjnmscceM62trWblypXmvPPOM//617+Gnd/W1mZ+8pOfmN///vfm6quvNitXroxswQ457W/lypXmgQceME1NTebw4cNm7dq15pxzzjEHDx6McOVn57S3gwcPmmeeeca0tLSYtrY28/TTT5tzzz3XPProoxGufHSc9jfgo48+MpdeeqkpKyszX/7ylyNTrENOe3v55ZeNJPPuu+8an883uH322WcRrvzsQvnevvnNb5qZM2eahoYG09bWZt544w3z2muvRbDq0XPa30cffRT0nXV0dJi0tDRTVVUV2cJHwWlvjY2NJi4uzvz6178277//vmlsbDQzZswwCxcujHDlo+O0v9raWpOSkmKeffZZc+TIEbNjxw5z/vnnmxdeeCGkzyeMnEFJSYmprKwMGrvyyivNmjVrzrrvddddF/VhZCz9DcjPzzfV1dXhLm3MwtHbt7/9bbN48eJwlxYWofZXUVFh7r33XlNVVRW1YcRpbwNh5N///ncEqhsbp729+OKLxuPxmBMnTkSivDEb6+/d888/b1wul/nnP/85HuWNidPefvnLX5pLL700aOyRRx4xWVlZ41bjWDjtr7S01Nx9991BYytXrjRz5swJ6fM5TTOCU6dO6cCBAyorKwsaLysr0/79+y1VFT7h6K+/v189PT1KS0sbjxJDFo7empubtX//fl133XXjUeKYhNrfk08+qSNHjqiqqmq8SwzZWL67a665Rl6vV3PnztXLL788nmWGJJTeXnjhBRUXF+vBBx/UxRdfrCuuuEJ33323/vvf/0aiZEfC8Xv3xBNP6Prrr1dOTs54lBiyUHqbPXu2PvjgA+3Zs0fGGH344Yf64x//qJtuuikSJTsSSn99fX1yu91BY8nJyWpqatKnn37quAbCyAiOHz+uQCCgjIyMoPGMjAx1dnZaqip8wtHfr371K/3nP//RzTffPB4lhmwsvWVlZSkpKUnFxcVasWKFli1bNp6lhiSU/t577z2tWbNG27dvV0JC9D4fM5TevF6vNm/erF27dum5555TXl6e5s6dq71790ai5FELpbf3339f+/btU0tLi55//nlt3LhRf/zjH7VixYpIlOzIWP9P8fl8evHFFyfN79zs2bO1fft2VVRUKDExUZmZmbrgggv0m9/8JhIlOxJKfzfccIMef/xxHThwQMYYvfnmm9qyZYs+/fRTHT9+3HEN0fu/UpRwuVxBr40xQ8YmslD727Fjh+6//3796U9/Unp6+niVNyah9NbY2KiPP/5Yr7/+utasWaPLLrtMt9xyy3iWGbLR9hcIBLRo0SJVV1friiuuiFR5Y+Lku8vLy1NeXt7g69LSUnV0dOihhx7SV7/61XGtMxROeuvv75fL5dL27dsHn3y6YcMGffe739WmTZuUnJw87vU6Fer/KU899ZQuuOACLVy4cJwqGzsnvbW2tuonP/mJ7rvvPt1www3y+Xy65557VFlZqSeeeCIS5TrmpL+f/exn6uzs1KxZs2SMUUZGhm6//XY9+OCDio+Pd/zZHBkZwdSpUxUfHz8kFXZ1dQ1JjxPRWPrbuXOnli5dqj/84Q+6/vrrx7PMkIylt9zcXF111VX64Q9/qJ/+9Ke6//77x7HS0Djtr6enR2+++abuuusuJSQkKCEhQevXr9ff/vY3JSQk6K9//WukSj+rcP3ezZo1S++99164yxuTUHrzer26+OKLgx7BPn36dBlj9MEHH4xrvU6N5bszxmjLli1asmSJEhMTx7PMkITSW01NjebMmaN77rlHX/rSl3TDDTeotrZWW7Zskc/ni0TZoxZKf8nJydqyZYs++eQT/fOf/1R7e7suueQSpaSkaOrUqY5rIIyMIDExUUVFRWpoaAgab2ho0OzZsy1VFT6h9rdjxw7dfvvteuaZZ6Ly3KcUvu/OGKO+vr5wlzdmTvtLTU3V3//+dx06dGhwq6ysVF5eng4dOqSZM2dGqvSzCtd319zcLK/XG+7yxiSU3ubMmaNjx47p448/Hhw7fPiw4uLilJWVNa71OjWW7+7VV1/VP/7xDy1dunQ8SwxZKL198skniosL/hM7cMTARNkj4cby3Z1zzjnKyspSfHy8nn32WX3jG98Y0veohHTZa4wYuNXpiSeeMK2trWbVqlXmvPPOG7zSe82aNWbJkiVB+zQ3N5vm5mZTVFRkFi1aZJqbm81bb71lo/yzctrfM888YxISEsymTZuCbsf76KOPbLUwIqe9/fa3vzUvvPCCOXz4sDl8+LDZsmWLSU1NNevWrbPVwhmF8m/z86L5bhqnvT388MPm+eefN4cPHzYtLS1mzZo1RpLZtWuXrRZG5LS3np4ek5WVZb773e+at956y7z66qvm8ssvN8uWLbPVwhmF+u9y8eLFZubMmZEu1xGnvT355JMmISHB1NbWmiNHjph9+/aZ4uJiU1JSYquFM3La37vvvmuefvppc/jwYfPGG2+YiooKk5aWZtra2kL6fMLIWWzatMnk5OSYxMREU1hYaF599dXBn912223muuuuC5ovaciWk5MT2aIdcNLfddddN2x/t912W+QLHwUnvT3yyCNmxowZ5txzzzWpqanmmmuuMbW1tSYQCFiofHSc/tv8vGgOI8Y46+2BBx4w06ZNM26323zhC18wX/nKV8yf//xnC1WPjtPv7e233zbXX3+9SU5ONllZWWb16tXmk08+iXDVo+e0v48++sgkJyebzZs3R7hS55z29sgjj5j8/HyTnJxsvF6v+f73v28++OCDCFc9ek76a21tNVdffbVJTk42qamp5lvf+pZ55513Qv5slzFRdrwIAADEFK4ZAQAAVhFGAACAVYQRAABgFWEEAABYRRgBAABWEUYAAIBVhBEAAGAVYQQAAFhFGAEAAFYRRgAAgFWEEQAAYBVhBAAAWPV/AV7pxheBMQ3AAAAAAElFTkSuQmCC", "text/plain": [ "
" ] @@ -1719,7 +1701,7 @@ "\n" ], "text/plain": [ - "" + "" ] }, "execution_count": 41, @@ -1756,7 +1738,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "d43198bb314e4436a40aa9add084fed6", + "model_id": "a15ca91c407b4d2fadbae54e51621a30", "version_major": 2, "version_minor": 0 }, @@ -1775,7 +1757,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "009ad311f3c548f8ae2786dfd6a935e6", + "model_id": "bfc8738882d54fc791ee117bb509481f", "version_major": 2, "version_minor": 0 }, @@ -1789,7 +1771,7 @@ { "data": { "text/plain": [ - "" + "" ] }, "execution_count": 42, @@ -2049,7 +2031,7 @@ "\n" ], "text/plain": [ - "" + "" ] }, "execution_count": 43, @@ -2103,16 +2085,6 @@ "id": "2b9bb21a-73cd-444e-84a9-100e202aa422", "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to x, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel run was not connected to ran, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "data": { "text/plain": [ @@ -2285,20 +2257,7 @@ "execution_count": 49, "id": "a832e552-b3cc-411a-a258-ef21574fc439", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to name, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to crystalstructure, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to a, andthus could not disconnect from it.\n", - " warn(\n" - ] - } - ], + "outputs": [], "source": [ "wf = Workflow(\"phase_preference\")\n", "wf.element = wf.create.standard.UserInput()\n", @@ -3296,7 +3255,7 @@ "\n" ], "text/plain": [ - "" + "" ] }, "execution_count": 50, @@ -3324,7 +3283,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "d42f0d45153c43d08c00b1be09a5d371", + "model_id": "c2d3d353f02f469cbb9bdf4fd8249962", "version_major": 2, "version_minor": 0 }, @@ -3345,7 +3304,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "9d3a8ad5fad943ca84a31ef0447ed921", + "model_id": "e3cb6c8a60ba4488976c563b03615dbd", "version_major": 2, "version_minor": 0 }, @@ -3375,14 +3334,6 @@ "id": "091e2386-0081-436c-a736-23d019bd9b91", "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel ran was not connected to accumulate_and_run, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "name": "stdout", "output_type": "stream", @@ -3393,7 +3344,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "1022ff0dc68a42649273a16d4783738c", + "model_id": "f1f215c871884c07a807522324b5d468", "version_major": 2, "version_minor": 0 }, @@ -3414,7 +3365,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "d702dbfc432d4f58847d9d13dcc3caf0", + "model_id": "484cb214c9aa42e7bf7e0c15a71dc7bf", "version_major": 2, "version_minor": 0 }, @@ -3455,18 +3406,7 @@ "execution_count": 53, "id": "4cdffdca-48d3-4486-9045-48102c7e5f31", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel job was not connected to job, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel accumulate_and_run was not connected to ran, andthus could not disconnect from it.\n", - " warn(\n" - ] - } - ], + "outputs": [], "source": [ "replacee = wf.min_phase1.calc \n", "wf.min_phase1.calc = Workflow.create.pyiron_atomistics.CalcStatic" @@ -3488,14 +3428,6 @@ "id": "ed4a3a22-fc3a-44c9-9d4f-c65bc1288889", "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel ran was not connected to accumulate_and_run, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "name": "stdout", "output_type": "stream", @@ -3506,7 +3438,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "764286716ac94a30ba87c095e11d29a5", + "model_id": "3eda5eaed97d4ef5bd05db0a1aaa6808", "version_major": 2, "version_minor": 0 }, @@ -3527,7 +3459,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "c7c4f447758a4106b3ea7463796aed44", + "model_id": "e218374bcdcf4c818ce88b2410368050", "version_major": 2, "version_minor": 0 }, @@ -3558,14 +3490,6 @@ "id": "5a985cbf-c308-4369-9223-b8a37edb8ab1", "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel ran was not connected to accumulate_and_run, andthus could not disconnect from it.\n", - " warn(\n" - ] - }, { "name": "stdout", "output_type": "stream", @@ -3576,7 +3500,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "6d864a43e64f44aca3717b3911d4a2f9", + "model_id": "963e16ec783846d1885ed72dfe26c3c5", "version_major": 2, "version_minor": 0 }, @@ -3597,7 +3521,7 @@ { "data": { "application/vnd.jupyter.widget-view+json": { - "model_id": "0530268ddab7455584ba8f85f7da783c", + "model_id": "d13b542a098544c5a50e869948ccd5f5", "version_major": 2, "version_minor": 0 }, @@ -3663,7 +3587,7 @@ "output_type": "stream", "text": [ "None 1\n", - " 5\n" + " 5\n" ] } ], @@ -3749,7 +3673,7 @@ "output_type": "stream", "text": [ "None 1\n", - " 5\n", + " 5\n", "Finally 5\n", "b (Add):\n", "Inputs ['obj', 'other']\n", @@ -3810,7 +3734,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "6.009681912953965\n" + "6.0116940710067865\n" ] } ], @@ -3842,7 +3766,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "3.0886348159983754\n" + "2.92550689199561\n" ] } ], @@ -3889,6 +3813,501 @@ "Unfortunately, _nested_ executors are not yet working. So if you set a macro to use an executor, none of its (grand...)children may specify an executor." ] }, + { + "cell_type": "markdown", + "id": "4d3f2d37-9e35-425b-93a1-2c327685bbf4", + "metadata": {}, + "source": [ + "# For-loops\n", + "\n", + "Any node with an IO signature that is fixed at the class level (i.e. every `StaticNode`, which is all the standard ones except for a `Workflow` instance) can be transformed into a macro that loops over that node using the `Workflow.create.for_node` interface. Any input that is not explicity scattered using the `iter_on` or `zip_on` gets gets broadcast to _all_ copies of the body node. The result is a dataframe coupling looped input to body node output:" + ] + }, + { + "cell_type": "code", + "execution_count": 63, + "id": "e3538139-f814-43ba-aad2-f35be0dc2721", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Al: [0. 0. 0.]\n", + "tags: \n", + " indices: [0]\n", + "pbc: [ True True True]\n", + "cell: \n", + "Cell([[0.0, 2.05, 2.05], [2.05, 0.0, 2.05], [2.05, 2.05, 0.0]])" + ] + }, + "execution_count": 63, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "n = Workflow.create.pyiron_atomistics.Bulk(name=\"Al\", a=4.1)\n", + "n()" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "0b373764-b389-4c24-8086-f3d33a4f7fd7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
astructure
03.90[Atom('Al', [0.0, 0.0, 0.0], index=0)]
13.95[Atom('Al', [0.0, 0.0, 0.0], index=0)]
24.00[Atom('Al', [0.0, 0.0, 0.0], index=0)]
34.05[Atom('Al', [0.0, 0.0, 0.0], index=0)]
44.10[Atom('Al', [0.0, 0.0, 0.0], index=0)]
\n", + "
" + ], + "text/plain": [ + " a structure\n", + "0 3.90 [Atom('Al', [0.0, 0.0, 0.0], index=0)]\n", + "1 3.95 [Atom('Al', [0.0, 0.0, 0.0], index=0)]\n", + "2 4.00 [Atom('Al', [0.0, 0.0, 0.0], index=0)]\n", + "3 4.05 [Atom('Al', [0.0, 0.0, 0.0], index=0)]\n", + "4 4.10 [Atom('Al', [0.0, 0.0, 0.0], index=0)]" + ] + }, + "execution_count": 64, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "bulk_loop = Workflow.create.for_node(\n", + " Workflow.create.pyiron_atomistics.Bulk,\n", + " iter_on=(\"a\",),\n", + " name=\"Al\",\n", + " a=np.linspace(3.9, 4.1, 5).tolist()\n", + ")\n", + "\n", + "out = bulk_loop()\n", + "out.df" + ] + }, + { + "cell_type": "markdown", + "id": "c8481efb-d7a5-4395-9e46-aab6a0e004eb", + "metadata": {}, + "source": [ + "Any number of input channels can be specified to make a nested list over, and/or zipped over by passing the channel labels as tuples to the `iter_on` and `zip_on` arguments respectively. In case the body node uses the same labels for both (looped) input channels _and_ output channels, you will need to provide a map to the for-loop to prevent the resulting dataframe from having degenerate column names:" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "id": "6f486c87-f3d4-405f-a759-2ada12cb45e2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
abcdout_aout_bout_cout_de
013791379e
11381013810e
214791479e
31481014810e
415791579e
51581015810e
616791679e
71681016810e
823792379e
92381023810e
1024792479e
112481024810e
1225792579e
132581025810e
1426792679e
152681026810e
\n", + "
" + ], + "text/plain": [ + " a b c d out_a out_b out_c out_d e\n", + "0 1 3 7 9 1 3 7 9 e\n", + "1 1 3 8 10 1 3 8 10 e\n", + "2 1 4 7 9 1 4 7 9 e\n", + "3 1 4 8 10 1 4 8 10 e\n", + "4 1 5 7 9 1 5 7 9 e\n", + "5 1 5 8 10 1 5 8 10 e\n", + "6 1 6 7 9 1 6 7 9 e\n", + "7 1 6 8 10 1 6 8 10 e\n", + "8 2 3 7 9 2 3 7 9 e\n", + "9 2 3 8 10 2 3 8 10 e\n", + "10 2 4 7 9 2 4 7 9 e\n", + "11 2 4 8 10 2 4 8 10 e\n", + "12 2 5 7 9 2 5 7 9 e\n", + "13 2 5 8 10 2 5 8 10 e\n", + "14 2 6 7 9 2 6 7 9 e\n", + "15 2 6 8 10 2 6 8 10 e" + ] + }, + "execution_count": 65, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "@Workflow.wrap.as_function_node()\n", + "def FiveApart(a: int, b: int, c: int, d: int, e: str = \"foobar\"):\n", + " return a, b, c, d, e,\n", + "\n", + "for_instance = Workflow.create.for_node(\n", + " FiveApart,\n", + " iter_on=(\"a\", \"b\"),\n", + " zip_on=(\"c\", \"d\"),\n", + " a=[1, 2],\n", + " b=[3, 4, 5, 6],\n", + " c=[7, 8],\n", + " d=[9, 10, 11],\n", + " e=\"e\",\n", + " output_column_map={\n", + " \"a\": \"out_a\",\n", + " \"b\": \"out_b\",\n", + " \"c\": \"out_c\",\n", + " \"d\": \"out_d\"\n", + " }\n", + ")\n", + "\n", + "out = for_instance()\n", + "out.df" + ] + }, + { + "cell_type": "markdown", + "id": "22b688e2-d203-4795-a409-dfeaa978b595", + "metadata": {}, + "source": [ + "Once set, these inputs will _always_ be iterated on, and thus require list input, but the length of the input can be varied between runs of the node. Under the hood, the macro is destroying and recreating (many of) its subgraph nodes at each runtime -- so the interface is fixed, but the internal structure can vary. Note that we use the same standard as python, and zipped input is always truncated to the shortest zipping partner:" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "id": "10f284c8-9210-465f-b4d6-9aa4c0909b08", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
abcdout_aout_bout_cout_de
013791379e
\n", + "
" + ], + "text/plain": [ + " a b c d out_a out_b out_c out_d e\n", + "0 1 3 7 9 1 3 7 9 e" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "for_instance(a=[1], b=[3], c=[7]).df" + ] + }, { "cell_type": "markdown", "id": "f447531e-3e8c-4c7e-a579-5f9c56b75a5b", @@ -3933,7 +4352,7 @@ }, { "cell_type": "code", - "execution_count": 63, + "execution_count": 67, "id": "c8196054-aff3-4d39-a872-b428d329dac9", "metadata": {}, "outputs": [], @@ -3943,7 +4362,7 @@ }, { "cell_type": "code", - "execution_count": 64, + "execution_count": 68, "id": "ffd741a3-b086-4ed0-9a62-76143a3705b2", "metadata": {}, "outputs": [], @@ -3960,7 +4379,7 @@ }, { "cell_type": "code", - "execution_count": 65, + "execution_count": 69, "id": "3a22c622-f8c1-449b-a910-c52beb6a09c3", "metadata": {}, "outputs": [ @@ -3968,15 +4387,15 @@ "name": "stderr", "output_type": "stream", "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:373: UserWarning: A saved file was found for the node save_demo -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:376: UserWarning: A saved file was found for the node save_demo -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:373: UserWarning: A saved file was found for the node inp -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:376: UserWarning: A saved file was found for the node inp -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:373: UserWarning: A saved file was found for the node middle -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:376: UserWarning: A saved file was found for the node middle -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:373: UserWarning: A saved file was found for the node end -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:376: UserWarning: A saved file was found for the node end -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:373: UserWarning: A saved file was found for the node out -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/node.py:376: UserWarning: A saved file was found for the node out -- attempting to load it...(To delete the saved file instead, use `overwrite_save=True`)\n", " warnings.warn(\n" ] } @@ -3999,7 +4418,7 @@ }, { "cell_type": "code", - "execution_count": 66, + "execution_count": 70, "id": "0999d3e8-3a5a-451d-8667-a01dae7c1193", "metadata": {}, "outputs": [], @@ -4008,100 +4427,23 @@ " reloaded.storage.delete()" ] }, - { - "cell_type": "markdown", - "id": "1f012460-19af-45f7-98aa-a0ad5b8e6faa", - "metadata": {}, - "source": [ - "## Meta-nodes and flow control\n", - "\n", - "A meta-node is a function that produces a node _class_ instedad of a node _instance_.\n", - "Right now, these are used to produce parameterized flow-control nodes, which take an node class as input and return a new macro class that builds some graph using the passed node class, e.g. for- and while-loops.\n", - "\n", - "Note: The body (and condition) node classes passed to for- and while-loops must be importable, i.e. they can come from a node package, or be defined here in the notebook (importable from `__main__`), but you can't use, e.g., a node defined _inside_ the scope of some other function.\n", - "\n", - "### For-loops\n", - "\n", - "One meta node is a for-loop builder, which creates a macro with $n$ internal instances of the \"loop body\" node class, and a new IO interface.\n", - "The new input allows you to specify which input channels are being looped over -- such that the macro input for this channel is interpreted as list-like and distributed to all the copies of the nodes separately --, and which is _not_ being looped over -- and thus interpreted as the loop body node would normally interpret the input and passed to all copies equally.\n", - "All of the loop body outputs are then collected as a list of length $n$.\n", - "\n", - "We follow a convention that inputs and outputs being looped over are indicated by their channel labels being ALL CAPS.\n", - "\n", - "In the example below, we loop over the bulk structure node to create structures with different lattice constants:" - ] - }, - { - "cell_type": "code", - "execution_count": 67, - "id": "0b373764-b389-4c24-8086-f3d33a4f7fd7", - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/io_preview.py:253: OutputLabelsNotValidated: Could not find the source code to validate BulkForA5 output labels against the number of returned values -- proceeding without validation\n", - " warnings.warn(\n" - ] - }, - { - "data": { - "text/plain": [ - "[14.829749999999995,\n", - " 15.407468749999998,\n", - " 15.999999999999998,\n", - " 16.60753125,\n", - " 17.230249999999995]" - ] - }, - "execution_count": 67, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "n = 5\n", - "\n", - "bulk_loop = Workflow.create.meta.for_loop(\n", - " Workflow.create.pyiron_atomistics.Bulk,\n", - " n,\n", - " iterate_on=(\"a\",),\n", - ")()\n", - "\n", - "out = bulk_loop(\n", - " name=\"Al\", # Sent equally to each body node\n", - " A=np.linspace(3.9, 4.1, n).tolist(), # Scattered across body nodes\n", - " # The rest of the values need to be filled\n", - " # (We don't currently pass body defaults to the loop node)\n", - " # but just get broadcast to each body node\n", - " crystalstructure=None,\n", - " c=None,\n", - " covera=None,\n", - " u=None,\n", - " orthorhombic=False,\n", - " cubic=False\n", - ")\n", - "\n", - "[struct.cell.volume for struct in out.STRUCTURE] \n", - "# output is a list collected from copies of the body node, as indicated by CAPS label" - ] - }, { "cell_type": "markdown", "id": "4e7ed210-dbc2-4afa-825e-b91168baff25", "metadata": {}, "source": [ - "## While-loops\n", + "# While-loops\n", + "\n", + "Similar to for-loops, we can also create a while-loop, which takes both a body node and a condition node. The condition node must be a single-output `Function` node returning a `bool` type. Instead of creating copies of the body node, the body node gets re-run until the condition node returns `False`.\n", "\n", - "We can also create a while-loop, which takes both a body node and a condition node. The condition node must be a single-output `Function` node returning a `bool` type. Instead of creating copies of the body node, the body node gets re-run until the condition node returns `False`.\n", + "You _must_ specify the data connection so that the body node passes information to the condition node. You may optionally also loop output of the body node back to input of the body node to change the input at each iteration. Right now this is done with horribly ugly string tuples, but we're working on improving this interface and making it more like the for-loop.\n", "\n", - "You _must_ specify the data connection so that the body node passes information to the condition node. You may optionally also loop output of the body node back to input of the body node to change the input at each iteration. Right now this is done with horribly ugly string tuples, but we're still working on it." + "Note: The body (and condition) node classes passed to while-loops must be importable, i.e. they can come from a node package, or be defined here in the notebook (importable from `__main__`), but you can't use, e.g., a node defined _inside_ the scope of some other function." ] }, { "cell_type": "code", - "execution_count": 68, + "execution_count": 71, "id": "0dd04b4c-e3e7-4072-ad34-58f2c1e4f596", "metadata": {}, "outputs": [ @@ -4109,18 +4451,8 @@ "name": "stderr", "output_type": "stream", "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/io_preview.py:253: OutputLabelsNotValidated: Could not find the source code to validate AddWhileLessThan_m3900845772041641930 output labels against the number of returned values -- proceeding without validation\n", - " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to a, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to b, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to other, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel run was not connected to true, andthus could not disconnect from it.\n", - " warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel run was not connected to ran, andthus could not disconnect from it.\n", - " warn(\n" + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/io_preview.py:261: OutputLabelsNotValidated: Could not find the source code to validate AddWhileLessThan_6300416345671079692 output labels against the number of returned values -- proceeding without validation\n", + " warnings.warn(\n" ] } ], @@ -4164,7 +4496,7 @@ }, { "cell_type": "code", - "execution_count": 69, + "execution_count": 72, "id": "2dfb967b-41ac-4463-b606-3e315e617f2a", "metadata": {}, "outputs": [ @@ -4188,7 +4520,7 @@ }, { "cell_type": "code", - "execution_count": 70, + "execution_count": 73, "id": "2e87f858-b327-4f6b-9237-c8a557f29aeb", "metadata": {}, "outputs": [ @@ -4196,23 +4528,23 @@ "name": "stdout", "output_type": "stream", "text": [ - "0.540 > 0.2\n", - "0.285 > 0.2\n", - "0.946 > 0.2\n", - "0.437 > 0.2\n", - "0.558 > 0.2\n", - "0.010 <= 0.2\n", - "Finally 0.010\n" + "0.717 > 0.2\n", + "0.384 > 0.2\n", + "0.334 > 0.2\n", + "0.223 > 0.2\n", + "0.219 > 0.2\n", + "0.333 > 0.2\n", + "0.834 > 0.2\n", + "0.167 <= 0.2\n", + "Finally 0.167\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/io_preview.py:253: OutputLabelsNotValidated: Could not find the source code to validate RandomWhileGreaterThan_m6305331635963844247 output labels against the number of returned values -- proceeding without validation\n", - " warnings.warn(\n", - "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/channels.py:176: UserWarning: The channel user_input was not connected to threshold, andthus could not disconnect from it.\n", - " warn(\n" + "/Users/huber/work/pyiron/pyiron_workflow/pyiron_workflow/io_preview.py:261: OutputLabelsNotValidated: Could not find the source code to validate RandomWhileGreaterThan_715325418919625042 output labels against the number of returned values -- proceeding without validation\n", + " warnings.warn(\n" ] } ], diff --git a/pyiron_workflow/channels.py b/pyiron_workflow/channels.py index 81be6e02b..c7d975175 100644 --- a/pyiron_workflow/channels.py +++ b/pyiron_workflow/channels.py @@ -11,7 +11,6 @@ import typing from abc import ABC, abstractmethod import inspect -from warnings import warn from pyiron_workflow.has_interface_mixins import HasChannel, HasLabel, UsesState from pyiron_workflow.has_to_dict import HasToDict @@ -172,11 +171,6 @@ def disconnect(self, *others: Channel) -> list[tuple[Channel, Channel]]: self.connections.remove(other) other.disconnect(self) destroyed_connections.append((self, other)) - else: - warn( - f"The channel {self.label} was not connected to {other.label}, and" - f"thus could not disconnect from it." - ) return destroyed_connections def disconnect_all(self) -> list[tuple[Channel, Channel]]: diff --git a/pyiron_workflow/create.py b/pyiron_workflow/create.py index 183e062a3..75e077f3a 100644 --- a/pyiron_workflow/create.py +++ b/pyiron_workflow/create.py @@ -64,6 +64,13 @@ def __init__(self): # this if-clause and just letting users of python <3.10 hit an error. self.register("pyiron_workflow.node_library.standard", "standard") + @property + @lru_cache(maxsize=1) + def for_node(self): + from pyiron_workflow.for_loop import for_node + + return for_node + @property @lru_cache(maxsize=1) def macro_node(self): @@ -82,12 +89,11 @@ def Workflow(self): @lru_cache(maxsize=1) def meta(self): from pyiron_workflow.transform import inputs_to_list, list_to_outputs - from pyiron_workflow.loops import for_loop, while_loop + from pyiron_workflow.loops import while_loop from pyiron_workflow.snippets.dotdict import DotDict return DotDict( { - for_loop.__name__: for_loop, inputs_to_list.__name__: inputs_to_list, list_to_outputs.__name__: list_to_outputs, while_loop.__name__: while_loop, diff --git a/pyiron_workflow/for_loop.py b/pyiron_workflow/for_loop.py new file mode 100644 index 000000000..a9a672291 --- /dev/null +++ b/pyiron_workflow/for_loop.py @@ -0,0 +1,479 @@ +from __future__ import annotations + +from abc import ABC +from concurrent.futures import Executor +from functools import lru_cache +import itertools +import math +from typing import Any, ClassVar, Literal, Optional + +from pandas import DataFrame + +from pyiron_workflow.channels import NOT_DATA +from pyiron_workflow.composite import Composite +from pyiron_workflow.io_preview import StaticNode +from pyiron_workflow.snippets.factory import classfactory +from pyiron_workflow.transform import inputs_to_dict, inputs_to_dataframe, InputsToDict + + +def dictionary_to_index_maps( + data: dict, + nested_keys: Optional[list[str] | tuple[str, ...]] = None, + zipped_keys: Optional[list[str] | tuple[str, ...]] = None, +): + """ + Given a dictionary where some data is iterable, and list(s) of keys over + which to make a nested and/or zipped loop, return dictionaries mapping + these keys to all the indices of the data they hold. Zipped loops are + nested outside the nesting loops. + + Args: + data (dict): The dictionary of data, some of which must me iterable. + nested_keys (tuple[str, ...] | None): The keys whose data to make a + nested for-loop over. + zipped_keys (tuple[str, ...] | None): The keys whose data to make a + zipped for-loop over. + + Returns: + (tuple[dict[..., int], ...]): A tuple of dictionaries where each item + maps the dictionary key to an index for that key's value. + + Raises: + (KeyError): If any of the provided keys are not keys of the provided + dictionary. + (TypeError): If any of the data held in a provided key does cannot be + operated on with `len`. + (ValueError): If neither set of keys to iterate on is provided, or if + all values being iterated over have a length of zero. + """ + + try: + nested_data_lengths = ( + [] + if (nested_keys is None or len(nested_keys) == 0) + else list(len(data[key]) for key in nested_keys) + ) + except TypeError as e: + raise TypeError( + f"Could not parse nested lengths -- Does one of the keys {nested_keys} " + f"have non-iterable data?" + ) from e + n_nest = math.prod(nested_data_lengths) if len(nested_data_lengths) > 0 else 0 + + try: + n_zip = ( + 0 + if (zipped_keys is None or len(zipped_keys) == 0) + else min(len(data[key]) for key in zipped_keys) + ) + except TypeError as e: + raise TypeError( + f"Could not parse zipped lengths -- Does one of the keys {zipped_keys} " + f"have non-iterable data?" + ) from e + + def nested_generator(): + return itertools.product(*[range(n) for n in nested_data_lengths]) + + def nested_index_map(nested_indices): + return { + nested_keys[i_key]: nested_index + for i_key, nested_index in enumerate(nested_indices) + } + + def zipped_generator(): + return range(n_zip) + + def zipped_index_map(zipped_index): + return {key: zipped_index for key in zipped_keys} + + def merge(d1, d2): + d1.update(d2) + return d1 + + if n_nest > 0 and n_zip > 0: + key_index_maps = tuple( + merge(nested_index_map(nested_indices), zipped_index_map(zipped_index)) + for nested_indices, zipped_index in itertools.product( + nested_generator(), zipped_generator() + ) + ) + elif n_nest > 0: + key_index_maps = tuple( + nested_index_map(nested_indices) for nested_indices in nested_generator() + ) + elif n_zip > 0: + key_index_maps = tuple( + zipped_index_map(zipped_index) for zipped_index in zipped_generator() + ) + else: + if nested_keys is None and zipped_keys is None: + raise ValueError( + "At least one of `nested_keys` or `zipped_keys` must be specified." + ) + else: + raise ValueError( + "Received keys to iterate over, but all values had length 0." + ) + + return key_index_maps + + +class UnmappedConflictError(ValueError): + """ + When a for-node gets a body whose output label conflicts with looped a input + label and no map was provided to avoid this. + """ + + +class MapsToNonexistentOutputError(ValueError): + """ + When a for-node tries to map body node output channels that don't exist. + """ + + +class For(Composite, StaticNode, ABC): + """ + Specifies fixed fields of some other node class to iterate over, but allows the + length of looped input to vary by dynamically destroying and recreating (most of) + its subgraph at run-time. + + Collects looped output and collates them with looped input values in a dataframe. + + The :attr:`body_node_executor` gets applied to each body node instance on each + run. + """ + + _body_node_class: ClassVar[type[StaticNode]] + _iter_on: ClassVar[tuple[str, ...]] = () + _zip_on: ClassVar[tuple[str, ...]] = () + + def __init_subclass__(cls, output_column_map=None, **kwargs): + super().__init_subclass__(**kwargs) + + unmapped_conflicts = ( + set(cls._body_node_class.preview_inputs().keys()) + .intersection(cls._iter_on + cls._zip_on) + .intersection(cls._body_node_class.preview_outputs().keys()) + .difference(() if output_column_map is None else output_column_map.keys()) + ) + if len(unmapped_conflicts) > 0: + raise UnmappedConflictError( + f"The body node {cls._body_node_class.__name__} has channel labels " + f"{unmapped_conflicts} that appear as both (looped) input _and_ output " + f"for {cls.__name__}. All such channels require a map to produce new, " + f"unique column names for the output." + ) + + maps_to_nonexistent_output = set( + {} if output_column_map is None else output_column_map.keys() + ).difference(cls._body_node_class.preview_outputs().keys()) + if len(maps_to_nonexistent_output) > 0: + raise MapsToNonexistentOutputError( + f"{cls.__name__} tried to map body node output(s) " + f"{maps_to_nonexistent_output} to new column names, but " + f"{cls._body_node_class.__name__} has no such outputs." + ) + + cls._output_column_map = output_column_map + + @classmethod + @property + @lru_cache(maxsize=1) + def output_column_map(cls) -> dict[str, str]: + """ + How to transform body node output labels to dataframe column names. + """ + map_ = {k: k for k in cls._body_node_class.preview_outputs().keys()} + overrides = {} if cls._output_column_map is None else cls._output_column_map + for body_label, column_name in overrides.items(): + map_[body_label] = column_name + return map_ + + def __init__( + self, + *args, + label: Optional[str] = None, + parent: Optional[Composite] = None, + overwrite_save: bool = False, + run_after_init: bool = False, + storage_backend: Optional[Literal["h5io", "tinybase"]] = None, + save_after_run: bool = False, + strict_naming: bool = True, + body_node_executor: Optional[Executor] = None, + **kwargs, + ): + super().__init__( + *args, + label=label, + parent=parent, + overwrite_save=overwrite_save, + run_after_init=run_after_init, + storage_backend=storage_backend, + save_after_run=save_after_run, + strict_naming=strict_naming, + **kwargs, + ) + self.body_node_executor = None + + def _setup_node(self) -> None: + super()._setup_node() + input_nodes = [] + for channel in self.inputs: + n = self.create.standard.UserInput( + channel.default, label=channel.label, parent=self + ) + n.inputs.user_input.type_hint = channel.type_hint + channel.value_receiver = n.inputs.user_input + input_nodes.append(n) + self.starting_nodes = input_nodes + self._input_node_labels = tuple(n.label for n in input_nodes) + + def on_run(self): + self._build_body() + return super().on_run() + + def _build_body(self): + """ + Construct instances of the body node based on input length, and wire them to IO. + """ + iter_maps = dictionary_to_index_maps( + self.inputs.to_value_dict(), + nested_keys=self._iter_on, + zipped_keys=self._zip_on, + ) + + self._clean_existing_subgraph() + + self.dataframe = inputs_to_dataframe(len(iter_maps)) + self.dataframe.outputs.df.value_receiver = self.outputs.df + + for n, channel_map in enumerate(iter_maps): + body_node = self._body_node_class(label=f"body_{n}", parent=self) + body_node.executor = self.body_node_executor + row_collector = self._build_collector_node(n) + + self._connect_broadcast_input(body_node) + for label, i in channel_map.items(): + self._connect_looped_input(body_node, row_collector, label, i) + + self._collect_output_from_body(body_node, row_collector) + + self.dataframe.inputs[f"row_{n}"] = row_collector + + self.set_run_signals_to_dag_execution() + + def _clean_existing_subgraph(self): + for label in self.child_labels: + if label not in self._input_node_labels: + self.remove_child(label) + else: + # Re-run the user input node so it has up-to-date output, otherwise + # when we inject a getitem node -- which will try to run automatically + # -- it will see data it can work with, but if that data happens to + # have the wrong length it may successfully auto-run on the wrong thing + # and throw an error! + self.children[label]() + # TODO: Instead of deleting _everything_ each time, try and re-use stuff + + def _build_collector_node(self, row_number): + # Iterated inputs + row_specification = { + key: (self._body_node_class.preview_inputs()[key][0], NOT_DATA) + for key in self._iter_on + self._zip_on + } + # Outputs + row_specification.update( + { + self.output_column_map[key]: (hint, NOT_DATA) + for key, hint in self._body_node_class.preview_outputs().items() + } + ) + return inputs_to_dict( + row_specification, parent=self, label=f"row_collector_{row_number}" + ) + + def _connect_broadcast_input(self, body_node: StaticNode) -> None: + """Connect broadcast macro input to each body node.""" + for broadcast_label in set(self.preview_inputs().keys()).difference( + self._iter_on + self._zip_on + ): + self.inputs[broadcast_label].value_receiver = body_node.inputs[ + broadcast_label + ] + + def _connect_looped_input( + self, + body_node: StaticNode, + row_collector: InputsToDict, + looped_input_label: str, + i: int, + ) -> None: + """Get item from macro input and connect it to body and collector nodes.""" + index_node = self.children[looped_input_label][i] # Inject getitem node + body_node.inputs[looped_input_label] = index_node + row_collector.inputs[looped_input_label] = index_node + + def _collect_output_from_body( + self, body_node: StaticNode, row_collector: InputsToDict + ) -> None: + """Pass body node output to the collector node.""" + for label, body_out in body_node.outputs.items(): + row_collector.inputs[self.output_column_map[label]] = body_out + + @classmethod + @lru_cache(maxsize=1) + def _build_inputs_preview(cls) -> dict[str, tuple[Any, Any]]: + preview = {} + for label, (hint, default) in cls._body_node_class.preview_inputs().items(): + # TODO: Leverage hint and default, listing if it's looped on + if label in cls._zip_on + cls._iter_on: + hint = list if hint is None else list[hint] + default = NOT_DATA # TODO: Figure out a generator pattern to get lists + preview[label] = (hint, default) + return preview + + @classmethod + def _build_outputs_preview(cls) -> dict[str, Any]: + return {"df": DataFrame} + + +def _for_node_class_name( + body_node_class: type[StaticNode], iter_on: tuple[str, ...], zip_on: tuple[str, ...] +): + iter_fields = ( + "" if len(iter_on) == 0 else "Iter" + "".join(k.title() for k in iter_on) + ) + zip_fields = "" if len(zip_on) == 0 else "Zip" + "".join(k.title() for k in zip_on) + return f"{For.__name__}{body_node_class.__name__}{iter_fields}{zip_fields}" + + +@classfactory +def for_node_factory( + body_node_class: type[StaticNode], + iter_on: tuple[str, ...] = (), + zip_on: tuple[str, ...] = (), + output_column_map: dict | None = None, + /, +): + return ( + _for_node_class_name(body_node_class, iter_on, zip_on), + (For,), + { + "_body_node_class": body_node_class, + "_iter_on": iter_on, + "_zip_on": zip_on, + }, + {"output_column_map": output_column_map}, + ) + + +def for_node( + body_node_class, + *node_args, + iter_on=(), + zip_on=(), + output_column_map: Optional[dict[str, str]] = None, + **node_kwargs, +): + """ + Makes a new :class:`For` node which internally creates instances of the + :param:`body_node_class` and loops input onto them in nested and/or zipped loop(s). + + Output is a single channel, `"df"`, which holds a :class:`pandas.DataFrame` whose + rows couple (looped) input to their respective body node outputs. + + The internal node structure gets re-created each run, so the same inputs must + consistently be iterated over, but their lengths can change freely. + + An executor can be applied to all body node instances at run-time by assigning it + to the :attr:`body_node_executor` attribute of the for-node. + + Args: + body_node_class type[StaticNode]: The class of node to loop on. + *node_args: Regular positional node arguments. + iter_on (tuple[str, ...]): Input labels in the :param:`body_node_class` to + nested-loop on. + zip_on (tuple[str, ...]): Input labels in the :param:`body_node_class` to + zip-loop on. + output_column_map (dict[str, str] | None): A map for generating dataframe + column names (values) from body node output channel labels (keys). + Necessary iff the body node has the same label for an output channel and + an input channel being looped over. (Default is None, just use the output + channel labels as columb names.) + **node_kwargs: Regular keyword node arguments. + + Returns: + (For): An instance of a dynamically-subclassed :class:`For` node. + + Examples: + >>> from pyiron_workflow import Workflow + >>> + >>> @Workflow.wrap.as_function_node("together") + ... def FiveTogether(a: int, b: int, c: int, d: int, e: str = "foobar"): + ... return (a, b, c, d, e), + >>> + >>> for_instance = Workflow.create.for_node( + ... FiveTogether, + ... iter_on=("a", "b"), + ... zip_on=("c", "d"), + ... a=[1, 2], + ... b=[3, 4, 5, 6], + ... c=[7, 8], + ... d=[9, 10, 11], + ... e="e" + ... ) + >>> + >>> out = for_instance() + >>> type(out.df) + + + Internally, the loop node has made a bunch of body nodes, as well as nodes to + index and collect data + >>> len(for_instance) + 48 + + We get one dataframe row for each possible combination of looped input + >>> len(out.df) + 16 + + We are stuck iterating on the fields we defined, but we can change the length + of the input and the loop node's body will get reconstructed at run-time to + accommodate this + >>> out = for_instance(a=[1], b=[3], d=[7]) + >>> len(for_instance), len(out) + (12, 1) + + Note that if we had simply returned each input individually, without any output + labels on the node, we'd need to specify a map on the for-node so that the + (looped) input and output columns on the resulting dataframe are all unique: + >>> @Workflow.wrap.as_function_node() + ... def FiveApart(a: int, b: int, c: int, d: int, e: str = "foobar"): + ... return a, b, c, d, e, + >>> + >>> for_instance = Workflow.create.for_node( + ... FiveApart, + ... iter_on=("a", "b"), + ... zip_on=("c", "d"), + ... a=[1, 2], + ... b=[3, 4, 5, 6], + ... c=[7, 8], + ... d=[9, 10, 11], + ... e="e", + ... output_column_map={ + ... "a": "out_a", + ... "b": "out_b", + ... "c": "out_c", + ... "d": "out_d" + ... } + ... ) + >>> + >>> out = for_instance() + >>> out.df.columns + Index(['a', 'b', 'c', 'd', 'out_a', 'out_b', 'out_c', 'out_d', 'e'], dtype='object') + + """ + for_node_factory.clear(_for_node_class_name(body_node_class, iter_on, zip_on)) + cls = for_node_factory(body_node_class, iter_on, zip_on, output_column_map) + cls.preview_io() + return cls(*node_args, **node_kwargs) diff --git a/pyiron_workflow/loops.py b/pyiron_workflow/loops.py index e7f9028d1..ec303db54 100644 --- a/pyiron_workflow/loops.py +++ b/pyiron_workflow/loops.py @@ -12,119 +12,6 @@ from pyiron_workflow.node import Node -def for_loop( - loop_body_class: type[Node], - length: int, - iterate_on: str | tuple[str] | list[str], -) -> type[Macro]: - """ - An _extremely rough_ second draft of a for-loop meta-node. - - Takes a node class, how long the loop should be, and which input(s) of the provided - node class should be looped over (given as strings of the channel labels) and - builds a macro that scatters some input and broadcasts the rest, then operates on - a zip of all the scattered input (so it had better be the same length). - - - Makes copies of the provided node class, i.e. the "body node" - - Labels in :param:`iterate_on` must correspond to `loop_body_class` input channels, - and the for-loop node then expects list-like input for these with ALL CAPS - labeling, and this gets scattered to the children. - - All other input simply gets broadcast to each child. - - Output channels correspond to input channels, but are lists of the children and - labeled in ALL CAPS - - Warnings: - The loop body class must be importable. E.g. it can come from a node package or - be defined in `__main__`, but not defined inside the scope of some other - function. - - Examples: - - >>> from pyiron_workflow import Workflow - >>> - >>> denominators = list(range(1, 5)) - >>> bulk_loop = Workflow.create.meta.for_loop( - ... Workflow.create.standard.Divide, - ... len(denominators), - ... iterate_on = ("other",), - ... )() - >>> bulk_loop.inputs.obj = 1 - >>> bulk_loop.inputs.OTHER = denominators - >>> bulk_loop().TRUEDIV - [1.0, 0.5, 0.3333333333333333, 0.25] - - TODO: - - - Refactor like crazy, it's super hard to read and some stuff is too hard-coded - - How to handle passing executors to the children? Maybe this is more - generically a Macro question? - - Is it possible to somehow dynamically adapt the held graph depending on the - length of the input values being iterated over? E.g. rebuilding the graph - every run call. - - Allow a different mode, or make a different meta node, that makes all possible - pairs of body nodes given the input being looped over instead of just - :param:`length` - - Provide enter and exit magic methods so we can `for` or `with` this fancy-like - """ - input_preview = loop_body_class.preview_inputs() - output_preview = loop_body_class.preview_outputs() - - # Ensure `iterate_on` is in the input - iterate_on = [iterate_on] if isinstance(iterate_on, str) else iterate_on - incommensurate_input = set(iterate_on).difference(input_preview.keys()) - if len(incommensurate_input) > 0: - raise ValueError( - f"Cannot loop on {incommensurate_input}, as it is not an input channel " - f"of {loop_body_class.__name__}; please choose from among " - f"{list(input_preview)}" - ) - - # Build code components that need an f-string, slash, etc. - output_labels = ", ".join(f'"{l.upper()}"' for l in output_preview.keys()).rstrip( - " " - ) - macro_args = ", ".join( - l.upper() if l in iterate_on else l for l in input_preview.keys() - ).rstrip(" ") - body_label = 'f"body{n}"' - item_access = "[{n}]" - body_kwargs = ", ".join( - f"{l}={l.upper()}[n]" if l in iterate_on else f"{l}={l}" - for l in input_preview.keys() - ).rstrip(" ") - input_label = 'f"item_{n}"' - returns = ", ".join( - f'self.children["{label.upper()}"]' for label in output_preview.keys() - ) - node_name = f'{loop_body_class.__name__}For{"".join([l.title() for l in sorted(iterate_on)])}{length}' - - # Assemble components into a decorated for-loop macro - for_loop_code = dedent( - f""" - @Macro.wrap.as_macro_node({output_labels}) - def {node_name}(self, {macro_args}): - from {loop_body_class.__module__} import {loop_body_class.__name__} - - for label in [{output_labels}]: - inputs_to_list({length}, label=label, parent=self) - - for n in range({length}): - body_node = {loop_body_class.__name__}( - {body_kwargs}, - label={body_label}, - parent=self - ) - for label in {list(output_preview.keys())}: - self.children[label.upper()].inputs[{input_label}] = body_node.outputs[label] - - return {returns} - """ - ) - - exec(for_loop_code) - return locals()[node_name] - - def while_loop( loop_body_class: type[Node], condition_class: type[Function], diff --git a/tests/integration/test_workflow.py b/tests/integration/test_workflow.py index 9e44e1cd4..71e080deb 100644 --- a/tests/integration/test_workflow.py +++ b/tests/integration/test_workflow.py @@ -105,26 +105,23 @@ def sqrt(value=0): def test_for_loop(self): Workflow.register("static.demo_nodes", "demo") - n = 5 - - bulk_loop = pyiron_workflow.loops.for_loop( - Workflow.create.demo.OptionallyAdd, - n, - iterate_on=("y",), - )() - base = 42 - to_add = list(range(n)) - out = bulk_loop( - x=base, # Sent equally to each body node - Y=to_add, # Distributed across body nodes + to_add = list(range(5)) + bulk_loop = Workflow.create.for_node( + Workflow.create.demo.OptionallyAdd, + iter_on=("y",), + x=base, # Broadcast + y=to_add # Scattered ) + out = bulk_loop() - for output, expectation in zip(out.SUM, [base + v for v in to_add]): + for output, expectation in zip( + out.df["sum"].values.tolist(), + [base + v for v in to_add] + ): self.assertAlmostEqual( output, expectation, - msg="Output should be list result of each individiual result" ) def test_while_loop(self): diff --git a/tests/unit/test_for_loop.py b/tests/unit/test_for_loop.py new file mode 100644 index 000000000..61296e067 --- /dev/null +++ b/tests/unit/test_for_loop.py @@ -0,0 +1,320 @@ +from concurrent.futures import ThreadPoolExecutor +from itertools import product +from time import perf_counter +import unittest + +from pandas import DataFrame + +from pyiron_workflow.for_loop import ( + dictionary_to_index_maps, + for_node, + UnmappedConflictError, + MapsToNonexistentOutputError +) +from pyiron_workflow.function import as_function_node +from pyiron_workflow.node_library.standard import Sleep + + +class TestDictionaryToIndexMaps(unittest.TestCase): + + def test_no_keys(self): + data = {"key": 5} + with self.assertRaises(ValueError): + dictionary_to_index_maps(data) + + def test_empty_nested_keys(self): + data = {"key1": [1, 2, 3], "key2": [4, 5, 6]} + with self.assertRaises(ValueError): + dictionary_to_index_maps(data, nested_keys=()) + + def test_empty_zipped_keys(self): + data = {"key1": [1, 2, 3], "key2": [4, 5, 6]} + with self.assertRaises(ValueError): + dictionary_to_index_maps(data, zipped_keys=()) + + def test_nested_non_iterable_data(self): + data = {"key1": [1, 2, 3], "key2": 5} + with self.assertRaises(TypeError): + dictionary_to_index_maps(data, nested_keys=("key1", "key2")) + + def test_zipped_non_iterable_data(self): + data = {"key1": [1, 2, 3], "key2": 5} + with self.assertRaises(TypeError): + dictionary_to_index_maps(data, zipped_keys=("key1", "key2")) + + def test_valid_data_nested_only(self): + data = {"key1": [1, 2, 3], "key2": [4, 5]} + nested_keys = ("key1", "key2") + expected_maps = tuple( + {nested_keys[i]: idx for i, idx in enumerate(indices)} + for indices in product(range(len(data["key1"])), range(len(data["key2"]))) + ) + self.assertEqual( + expected_maps, + dictionary_to_index_maps(data, nested_keys=nested_keys), + ) + + def test_valid_data_zipped_only(self): + data = {"key1": [1, 2, 3], "key2": [4, 5]} + zipped_keys = ("key1", "key2") + expected_maps = tuple( + {key: idx for key in zipped_keys} + for idx in range(min(len(data["key1"]), len(data["key2"]))) + ) + self.assertEqual( + expected_maps, + dictionary_to_index_maps(data, zipped_keys=zipped_keys), + ) + + def test_valid_data_nested_and_zipped(self): + data = { + "nested1": [2, 3], + "nested2": [4, 5, 6], + "zipped1": [7, 8, 9, 10], + "zipped2": [11, 12, 13, 14, 15] + } + nested_keys = ("nested1", "nested2") + zipped_keys = ("zipped1", "zipped2") + expected_maps = tuple( + { + nested_keys[0]: n_idx, + nested_keys[1]: n_idx2, + zipped_keys[0]: z_idx, + zipped_keys[1]: z_idx2 + } + for n_idx, n_idx2 in product( + range(len(data["nested1"])), + range(len(data["nested2"])) + ) + for z_idx, z_idx2 in zip( + range(len(data["zipped1"])), + range(len(data["zipped2"])) + ) + ) + self.assertEqual( + expected_maps, + dictionary_to_index_maps(data, nested_keys=nested_keys, zipped_keys=zipped_keys), + ) + + +@as_function_node("together") +def FiveTogether( + a: int = 0, + b: int = 1, + c: int = 2, + d: int = 3, + e: str = "foobar", +): + return (a, b, c, d, e,), + + +class TestForNode(unittest.TestCase): + def test_iter_only(self): + for_instance = for_node( + FiveTogether, + iter_on=("a", "b",), + a=[42, 43, 44], + b=[13, 14], + ) + out = for_instance(e="iter") + self.assertIsInstance(out.df, DataFrame,) + self.assertEqual( + len(out.df), + 3 * 2, + msg="Expect nested loops" + ) + self.assertListEqual( + out.df.columns.tolist(), + ["a", "b", "together"], + msg="Dataframe should only hold output and _looped_ input" + ) + self.assertTupleEqual( + out.df["together"][1], + ((42, 14, 2, 3, "iter"),), + msg="Iter should get nested, broadcast broadcast, else take default" + ) + + def test_zip_only(self): + for_instance = for_node( + FiveTogether, + zip_on=("c", "d",), + e="zip" + ) + out = for_instance(c=[100, 101], d=[-1, -2, -3]) + self.assertEqual( + len(out.df), + 2, + msg="Expect zipping with the python convention of truncating to shortest" + ) + self.assertListEqual( + out.df.columns.tolist(), + ["c", "d", "together"], + msg="Dataframe should only hold output and _looped_ input" + ) + self.assertTupleEqual( + out.df["together"][1], + ((0, 1, 101, -2, "zip"),), + msg="Zipped should get zipped, broadcast broadcast, else take default" + ) + + def test_iter_and_zip(self): + for_instance = for_node( + FiveTogether, + iter_on=("a", "b",), + a=[42, 43, 44], + b=[13, 14], + zip_on=("c", "d",), + e="both" + ) + out = for_instance(c=[100, 101], d=[-1, -2, -3]) + self.assertEqual( + len(out.df), + 3 * 2 * 2, + msg="Zipped stuff is nested with the individually nested fields" + ) + self.assertListEqual( + out.df.columns.tolist(), + ["a", "b", "c", "d", "together"], + msg="Dataframe should only hold output and _looped_ input" + ) + # We don't actually care if the order of nesting changes, but make sure the + # iters are getting nested and zipped stay together + self.assertTupleEqual( + out.df["together"][0], + ((42, 13, 100, -1, "both"),), + msg="All start" + ) + self.assertTupleEqual( + out.df["together"][1], + ((42, 13, 101, -2, "both"),), + msg="Bump zipped together" + ) + self.assertTupleEqual( + out.df["together"][2], + ((42, 14, 100, -1, "both"),), + msg="Back to start of zipped, bump _one_ iter" + ) + + def test_dynamic_length(self): + for_instance = for_node( + FiveTogether, + iter_on=("a", "b",), + a=[42, 43, 44], + b=[13, 14], + zip_on=("c", "d",), + c=[100, 101], + d=[-1, -2, -3] + ) + self.assertEqual( + 3 * 2 * 2, + len(for_instance().df), + msg="Sanity check" + ) + self.assertEqual( + 1, + len(for_instance(a=[0], b=[1], c=[2]).df), + msg="Should be able to re-run with different input lengths" + ) + + def test_column_mapping(self): + @as_function_node() + def FiveApart( + a: int = 0, + b: int = 1, + c: int = 2, + d: int = 3, + e: str = "foobar", + ): + return a, b, c, d, e, + + with self.subTest("Successful map"): + for_instance = for_node( + FiveApart, + iter_on=("a", "b"), + zip_on=("c", "d"), + a=[1, 2], + b=[3, 4, 5], + c=[7, 8], + d=[9, 10, 11], + e="e", + output_column_map={ + "a": "out_a", + "b": "out_b", + "c": "out_c", + "d": "out_d" + } + ) + self.assertEqual( + 4 + 5, # loop inputs + outputs + len(for_instance().df.columns), + msg="When all conflicting names are remapped, we should have no trouble" + ) + + with self.subTest("Insufficient map"): + with self.assertRaises( + UnmappedConflictError, + msg="Leaving conflicting channels unmapped should raise an error" + ): + for_node( + FiveApart, + iter_on=("a", "b"), + zip_on=("c", "d"), + a=[1, 2], + b=[3, 4, 5], + c=[7, 8], + d=[9, 10, 11], + e="e", + output_column_map={ + # "a": "out_a", + "b": "out_b", + "c": "out_c", + "d": "out_d" + } + ) + + with self.subTest("Excessive map"): + with self.assertRaises( + MapsToNonexistentOutputError, + msg="Trying to map something that isn't there should raise an error" + ): + for_node( + FiveApart, + iter_on=("a", "b"), + zip_on=("c", "d"), + a=[1, 2], + b=[3, 4, 5], + c=[7, 8], + d=[9, 10, 11], + e="e", + output_column_map={ + "a": "out_a", + "b": "out_b", + "c": "out_c", + "d": "out_d", + "not_a_key_on_the_body_node_outputs": "anything" + } + ) + + def test_body_node_executor(self): + t_sleep = 2 + for_parallel = for_node( + Sleep, + iter_on=("t",) + ) + t_start = perf_counter() + n_procs = 4 + with ThreadPoolExecutor(max_workers=n_procs) as exe: + for_parallel.body_node_executor = exe + for_parallel(t=n_procs*[t_sleep]) + dt = perf_counter() - t_start + grace = 1.1 + self.assertLess( + dt, + grace * t_sleep, + msg=f"Parallelization over children should result in faster completion. " + f"Expected limit {grace} x {t_sleep} = {grace * t_sleep} -- got {dt}" + ) + + +if __name__ == "__main__": + unittest.main()