Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
eb372b0
docs: add collections usage
lmeyerov Dec 29, 2025
b9f00ff
docs: add collections notebook and gfql ast example
lmeyerov Dec 29, 2025
b30a176
docs: link collections in overview guides
lmeyerov Dec 29, 2025
7023f8c
docs: link color encodings from collections
lmeyerov Dec 30, 2025
86f3b98
docs: replace absolute collections links
lmeyerov Dec 30, 2025
59ba590
docs: expand collections guidance and validation
lmeyerov Dec 30, 2025
76d86cb
docs: add quick collections tips
lmeyerov Dec 30, 2025
6f125d5
docs: clarify when to use collections
lmeyerov Dec 30, 2025
21ca5d0
docs: add collections overlap example
lmeyerov Dec 30, 2025
29f3847
docs: mention collections helper constructors
lmeyerov Dec 30, 2025
7f5555c
docs: note helpers wrap ast to gfql chain
lmeyerov Dec 30, 2025
3d35f9f
docs: use collections helper constructors
lmeyerov Dec 30, 2025
60d63be
docs: use collections helpers in examples
lmeyerov Dec 30, 2025
fc05db7
docs: add collections wire protocol note
lmeyerov Dec 30, 2025
3cda05f
docs: note collections after gfql
lmeyerov Dec 30, 2025
ebf8f6e
docs: use helper imports in collections examples
lmeyerov Dec 30, 2025
264bad0
docs: show graph-level collection chain
lmeyerov Dec 31, 2025
2325c29
docs: add plot call in collections notebook
lmeyerov Dec 31, 2025
a1e0fa1
docs: use honeypot dataset in collections notebook
lmeyerov Dec 31, 2025
c6575ba
docs: clarify collections usage
lmeyerov Jan 2, 2026
25c7bc5
docs: trigger ci
lmeyerov Jan 13, 2026
874f367
docs: fix layout settings cross-reference for PDF build
lmeyerov Apr 23, 2026
6731d37
docs: retrigger readthedocs build
lmeyerov Apr 23, 2026
93f5ea2
docs: fix collections doc review follow-ups
lmeyerov Apr 23, 2026
a9980d6
docs: retrigger readthedocs status
lmeyerov Apr 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions demos/more_examples/graphistry_features/collections.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Collections in PyGraphistry\n",
"\n",
"Collections define labeled subsets of a graph (nodes, edges, or subgraphs) using full GFQL. They enable advanced, layered styling that overrides base encodings when you need precise highlights.\n",
"\n",
"Use collections when you want:\n",
"- baseline encodings (for example, by entity type) plus overlays for alerts or critical paths\n",
"- multiple overlapping highlights with a priority order\n",
"- a UI panel for toggling focused subsets on and off\n",
"\n",
"Collections are evaluated in priority order, with higher priority collections overriding lower ones for styling.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this notebook, we build sets using GFQL AST helpers, combine them with intersections, and apply node and edge colors. Collections can be based on nodes, edges, or multi-step graph traversals (Chain).\n"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from pathlib import Path\n",
"import pandas as pd\n",
"import graphistry\n",
"from graphistry import collection_set, collection_intersection, n, e_forward, Chain\n",
"\n",
"edges = pd.read_csv(Path('demos/data/honeypot.csv'))\n",
Comment thread
lmeyerov marked this conversation as resolved.
"g = graphistry.edges(edges, \"attackerIP\", \"victimIP\")\n"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"# Use Chain to select subgraphs (nodes + edges) by edge attributes\n",
"collections = [\n",
" collection_set(\n",
" expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n",
" id='netapi',\n",
" name='MS08067 (NetAPI)',\n",
" node_color='#00BFFF',\n",
" edge_color='#00BFFF',\n",
" ),\n",
" collection_set(\n",
" expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n",
" id='port445',\n",
" name='Port 445',\n",
" node_color='#32CD32',\n",
" edge_color='#32CD32',\n",
" ),\n",
" collection_intersection(\n",
" sets=['netapi', 'port445'],\n",
" name='NetAPI + 445',\n",
" node_color='#AABBCC',\n",
" edge_color='#AABBCC',\n",
" ),\n",
"]\n",
"\n",
"g2 = g.collections(\n",
" collections=collections,\n",
" show_collections=True,\n",
" collections_global_node_color='CCCCCC',\n",
" collections_global_edge_color='CCCCCC',\n",
")\n",
"\n",
"g2._url_params\n"
]
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"# Render (requires graphistry.register(...))\n",
"g2.plot()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notes and validation\n",
"\n",
"- Order matters: earlier collections override later ones.\n",
"- Use collections for priority-based subsets and overlaps; use `encode_*` for simple column-driven colors.\n",
"- Helper constructors: `graphistry.collection_set(...)` and `graphistry.collection_intersection(...)` return JSON-friendly dicts (AST inputs wrap to `gfql_chain`).\n",
"- Provide `id` for sets used by intersections.\n",
"- Global colors apply to nodes/edges not in any collection; `#` is optional.\n",
"- Use `validate='strict'` to raise, or `warn=False` to silence warnings.\n",
"\n",
"Wire protocol and pre-encoded strings:\n",
"\n",
"```python\n",
"collections_wire = [\n",
" {\n",
" \"type\": \"set\",\n",
" \"name\": \"Wire Protocol Example\",\n",
" \"node_color\": \"#AA00AA\",\n",
" \"expr\": {\n",
" \"type\": \"gfql_chain\",\n",
" \"gfql\": [\n",
" {\"type\": \"Node\", \"filter_dict\": {\"status\": \"purchased\"}}\n",
" ]\n",
" }\n",
" }\n",
"]\n",
"g.collections(collections=collections_wire)\n",
"\n",
"g.collections(collections=encoded_collections, encode=False)\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run `g2.plot()` in a notebook session with valid credentials to render inline.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overlap priority example\n",
"\n",
"Earlier collections override later ones when they overlap.\n",
"\n",
"```python\n",
"collections_priority = [\n",
" collection_set(\n",
" expr=Chain([n(), e_forward({\"vulnName\": \"MS08067 (NetAPI)\"}), n()]),\n",
" id=\"netapi\",\n",
" name=\"MS08067 (NetAPI)\",\n",
" node_color=\"#FFAA00\",\n",
" edge_color=\"#FFAA00\",\n",
" ),\n",
" collection_set(\n",
" expr=Chain([n(), e_forward({\"victimPort\": 445.0}), n()]),\n",
" id=\"port445\",\n",
" name=\"Port 445\",\n",
" node_color=\"#00BFFF\",\n",
" edge_color=\"#00BFFF\",\n",
" ),\n",
"]\n",
"g.collections(collections=collections_priority)\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For more on color encodings, see the [Color encodings notebook](encodings-colors.ipynb).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.x"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
22 changes: 22 additions & 0 deletions docs/source/10min.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,28 @@ Example visualization:

Now, edges are colored based on the type of vulnerability, helping you distinguish different attack types.

Advanced: Collections for layered highlights
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Use collections when you want GFQL-driven subsets (nodes, edges, or subgraphs) to override base encodings.
This is useful for overlays like alerts or critical paths that take precedence over your normal color rules.

.. code-block:: python

from graphistry import collection_set, n

collections = [
collection_set(
expr=n({"vip": True}),
name="VIP",
node_color="#FF8800",
)
]
g.collections(collections=collections, show_collections=True).plot()

See the :doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>` and
:doc:`GFQL docs </gfql/index>` for full details.

Adjusting Sizes, Labels, Icons, Badges, and More
------------------------------------------------

Expand Down
5 changes: 5 additions & 0 deletions docs/source/cheatsheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -560,6 +560,11 @@ g.encode_point_color('type', as_categorical=True,
categorical_mapping={"cat": "red", "sheep": "blue"}, default_mapping='#CCC')
```

For subset-based coloring and conditional styling across multiple encodings, use Collections
via `g.collections(...)` with GFQL AST helpers. See
{doc}`Layout settings </visualization/layout/settings>`
and the [Collections tutorial notebook](demos/more_examples/graphistry_features/collections.ipynb).

For more in-depth examples, check out the tutorials on [colors](https://github.com/graphistry/pygraphistry/tree/master/demos/more_examples/graphistry_features/encodings-colors.ipynb).

### Custom icons and badges
Expand Down
3 changes: 3 additions & 0 deletions docs/source/gfql/quick.rst
Original file line number Diff line number Diff line change
Expand Up @@ -637,6 +637,9 @@ Run graph algorithms like PageRank, community detection, and layouts directly wi
# Results have x, y coordinates for visualization
result.plot()

Tip: For subset-based coloring after GFQL, use ``result.collections(...)`` and see
:doc:`/visualization/layout/settings`.

Remote Graph References
-----------------------

Expand Down
7 changes: 7 additions & 0 deletions docs/source/gfql/remote.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@ Run chain remotely and fetch results

``gfql_remote()`` accepts the same input types as local ``gfql()``:

.. note::
Collections are visualization URL settings; apply them after GFQL results
(for example, ``g2.collections(...)``). The GFQL remote/upload APIs do not
accept collections payloads yet.
Comment thread
lmeyerov marked this conversation as resolved.

Method :meth:`chain_remote <graphistry.compute.ComputeMixin.ComputeMixin.chain_remote>` runs chain remotely and fetches the computed graph

- **Chain / List[ASTObject]**: Native GFQL chain syntax (as above).
- **Cypher string**: Compiled locally, sent as wire-protocol JSON.
- **ASTLet / Let dict**: DAG patterns with named bindings.
Expand Down
44 changes: 44 additions & 0 deletions docs/source/gfql/spec/wire_protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -595,6 +595,50 @@ Example:

**Note**: The `timezone` field is optional for DateTime values and defaults to "UTC" if omitted. This ensures consistent behavior across systems while allowing explicit timezone specification when needed.

## Collections Payloads

Collections are Graphistry visualization overlays that use GFQL wire protocol operations to define subsets
of nodes, edges, or subgraphs. They are applied in priority order, with earlier collections overriding later
ones for styling.

### Collection Set

Collection sets wrap GFQL operations in a `gfql_chain` object:

```json
{
"type": "set",
"id": "purchasers",
"name": "Purchasers",
"node_color": "#00BFFF",
"expr": {
"type": "gfql_chain",
"gfql": [
{"type": "Node", "filter_dict": {"status": "purchased"}}
]
}
}
```

### Collection Intersection

Intersections reference previously defined set IDs:

```json
{
"type": "intersection",
"name": "High Value Purchasers",
"node_color": "#AA00AA",
"expr": {
"type": "intersection",
"sets": ["purchasers", "vip"]
}
}
```

For Python examples and helper constructors, see the
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.

## Examples

### `MATCH ... RETURN` Row Pipeline
Expand Down
1 change: 1 addition & 0 deletions docs/source/notebooks/visualization.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Encodings
Sizes <../demos/more_examples/graphistry_features/encodings-sizes.ipynb>
Icons <../demos/more_examples/graphistry_features/encodings-icons.ipynb>
Badges <../demos/more_examples/graphistry_features/encodings-badges.ipynb>
Collections <../demos/more_examples/graphistry_features/collections.ipynb>

Geographic (Kepler.gl)
----------------------
Expand Down
19 changes: 19 additions & 0 deletions docs/source/visualization/10min.rst
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,25 @@ You can encode your graph attributes visually using colors, sizes, icons, and mo
- :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_color`
- :meth:`graphistry.PlotterBase.PlotterBase.encode_edge_icon`

* **Collections (advanced coloring)**: Define subsets using GFQL AST helpers and color them consistently:

.. code-block:: python

from graphistry import collection_set, n

collections = [
collection_set(
expr=n({"subscribed_to_newsletter": True}),
name="Subscribers",
node_color="#32CD32",
)
]
g.collections(collections=collections, show_collections=True).plot()

See :doc:`Layout settings <layout/settings>` and the
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
Tip: order matters (earlier collections override later ones) and intersections require set IDs.

* **Bind**: Simpler data-driven settings are done through :meth:`graphistry.PlotterBase.PlotterBase.bind`:

.. code-block:: python
Expand Down
2 changes: 2 additions & 0 deletions docs/source/visualization/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ Visualize
=============

We recommend getting started with :ref:`10 Minutes to PyGraphistry <10min-pygraphistry>`, :ref:`10 Minutes to Graphistry Visualization<10min-viz>`, and the :ref:`layout guide <layout-guide>`.
For advanced, subset-based coloring, see :ref:`Layout settings <layout-settings>` and the
:doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.

For static image export (documentation, reports), see the `static rendering tutorial <../demos/demos_databases_apis/graphviz/static_rendering.ipynb>`_.

Expand Down
38 changes: 38 additions & 0 deletions docs/source/visualization/layout/settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,44 @@ Use :meth:`graphistry.PlotterBase.PlotterBase.scene_settings` to modify the appe
- ``edge_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI)
- ``point_opacity``: Range 0.0 to 1.0 (0.0 fully transparent, 1.0 fully opaque, displayed as 0-100 in UI)

Encodings (Color, Size, Icons)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Use the ``encode_*`` methods to style nodes and edges based on columns (for example, color by entity type).
See the :doc:`Color encodings notebook </demos/more_examples/graphistry_features/encodings-colors>` for full examples.

Collections
~~~~~~~~~~~

Collections define labeled subsets (nodes, edges, or subgraphs) using full GFQL and apply layered styling
that overrides base encodings. Use them to call out alerts or critical paths on top of your standard color
encodings, with priority-based overrides when subsets overlap.

For a full walkthrough, see the :doc:`Collections tutorial notebook </demos/more_examples/graphistry_features/collections>`.
For GFQL syntax, see :doc:`GFQL documentation </gfql/index>`.
For schema details, see `Collections URL options <https://hub.graphistry.com/docs/api/1/rest/url/#url-collections>`_.

.. code-block:: python

from graphistry import collection_set, n

collections = [
collection_set(
expr=n({"subscribed_to_newsletter": True}),
id="newsletter_subscribers",
name="Newsletter Subscribers",
node_color="#32CD32",
)
]

g2 = g.collections(
collections=collections,
show_collections=True,
collections_global_node_color="CCCCCC",
collections_global_edge_color="CCCCCC",
)
g2.plot()


Styling the Background and Foreground
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
Loading