Plugins: Give more flexibility to the user how to init client by exrhizo · Pull Request #663 · graphistry/pygraphistry

exrhizo · 2025-06-10T16:53:56Z

XyzGraph as an XyzClientWrapper with convenience methods

Changes:

Give more flexibility to users how to init (so louie can evolve independently)
XyzGraph decouple from Plottable - This way we can be more confident about independence

Requested by Thomas

Client to only stay open during query and then close
- So I made a context manager & only keep the credentials on the Plottable

KustoGraph

kusto_query -> List[pd.DataFrame]
query_graph -> Plottable

It is very difficult to predict what kusto will output as far as nodes and edges, so we have an ergonomic hook to visualize graphs and the more flexible query, that just returns a list of dfs. query also has a hueristic for unpacking the nested tables from a graph project.

lmeyerov · 2025-06-10T17:37:12Z

        from .plugins.kustograph import KustoGraph
        res = copy.copy(self)
-        res._kustograph = KustoGraph(res, kusto_config)
+        res._kustograph = KustoGraph.from_config(kusto_config)


I'm more of a fan of the previous mode because it doesn't lose current self bindings from the Plottable::copy() , which is our typical pattern here. The important part for safety is it's a fresh base instance object for client-local state.

(and if the caller wants empty, they can start with an empty)

This is the crux of what I am trying to reverse

I don't think these XyzGraphs should know anything about what is bound to the Plottable, that is the whole issue we are having in louie with it being unclear if these are multi tenant / thread safe / global values involved

lmeyerov · 2025-06-10T17:41:56Z

+        nodes, edges = res._spannergraph.gql_to_graph(query)
+
+        # TODO(tcook): add more error handling here if nodes or edges are empty
+        return res.nodes(nodes, 'identifier').edges(edges, 'source', 'destination')


hardcoding bindings here is a little weird to me in that it moves control of the schema out from gql_to_graph, causing coordination confusion, so maybe it should return bindings too? at that point, i think it should just return a plottable as before...

if the goal is a fresh one, maybe:

g = res._spannergraph.gql_to_graph(query) return res.nodes(g._nodes, g._node).edges(g._edges, g._source, g._destination)

see the way I did it this time

lmeyerov · 2025-06-10T17:45:50Z

-            self.connection.close()
-            logger.info("Connection to Spanner database closed.")
+        self.connection.close()
+        logger.info("Closed Spanner connection.")


fwiw, does this mean the caller is supposed to do res_g = g.kusto_query_to_graph(query); res_g._kusto_client.close() ? Not a blocker for the this PR, but am curious

(In general, unsure what lifecycle pattern we want for reconnecting, closing connections, etc, getting back to the whole louie discussion)

In the past the clients are never closed -- here I am assuming that the SpannerGraph is basically a client wrapper with convenience functions

Usually a user is using the graphistry object that seems more optimized for notebooks then for memory management

If someone wants memory management, then I think they can instantiate the SpannerGraph directly, like I want to do in louie

lmeyerov · 2025-06-10T17:56:07Z



-    def gql_to_graph(self, res: Plottable, query: str) -> Plottable:
+    def gql_to_graph(self, res: Plottable, query: str) -> Tuple[pd.DataFrame, pd.DataFrame]:


I like having these explicit versions to help with explicit consumers, so we can do gql_df: str -> df and gql_graph: str -> plottable and consumer has known type

scenarios like louie, and maybe convenience ones for users, benefit from gql: str -> df | plottable, and up to the consumer to handle the union type

The weird thing to me is what if I don't want to chain in my Plottable

tanmoyio · 2025-06-10T17:09:14Z


-        # TODO(tcook): add more error handling here if nodes or edges are empty
-        return res.nodes(nodes_df, 'identifier').edges(edges_df, 'source', 'destination')
+        return nodes_df, edges_df


This will be problematic for louie. Louie expects plottable from gql_to_graph. Any specific reason for this?

Louie pins the graphistry version so we can change the interface, I don't feel as strongly about this - see comments to Leo

I'm a bit lost on internal methods vs public

wrt public:

see: Plugins: Give more flexibility to the user how to init client #663 (comment)

Louie afaict benefits from a generic str -> pd.DataFrame | Plottable, where louie can do its own isinstance(...) and handle appropriately

I think beginner/vibes-y python users would benefit from that more dynamic type signature too...

... and only when we do more 'serious' typed code would we want the explicit gql_to_g / gql_to_df variants (or maybe there is magic where we can gql<pd.DataFrame>(...)?). So I'd want "both" vs picking

I hit the same thing when we exposed the remote python+gfql endpoints and we wanted to support multiple return types

lmeyerov · 2025-06-10T18:22:04Z

 import time
 from typing import Any, List, Dict, Optional

+from google.cloud.spanner_dbapi.connection import connect, Connection


same imports issue: #663 (comment)

we can release in louie off of this PR sooner than later, but blocks landing as a public graphistry release

exrhizo · 2025-06-11T03:53:14Z

Merged into #659

Plugins: Give more flexibility to the user how to init client

94d84fd

exrhizo requested review from DataBoyTX and tanmoyio June 10, 2025 16:53

fix: errors in kustograph

155c87b

lmeyerov reviewed Jun 10, 2025

View reviewed changes

Comment thread graphistry/plugins/kustograph.py Outdated

lmeyerov reviewed Jun 10, 2025

View reviewed changes

tanmoyio reviewed Jun 10, 2025

View reviewed changes

lmeyerov reviewed Jun 10, 2025

View reviewed changes

exrhizo added 4 commits June 10, 2025 11:25

Plugin runtime fixes and respond to comments

92eb052

Plugins, open close session, kusto unwrap heuristic,

015e7a6

fix column types

b8a3e26

Kusto query and query_graph doc strings

0cb23a4

exrhizo merged commit 56a0f9d into feat/kusto Jun 11, 2025
17 of 27 checks passed

exrhizo deleted the feat/plugin-flexible-client branch June 11, 2025 03:53



		def gql_to_graph(self, res: Plottable, query: str) -> Plottable:
		def gql_to_graph(self, res: Plottable, query: str) -> Tuple[pd.DataFrame, pd.DataFrame]:

Uh oh!

Conversation

exrhizo commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lmeyerov Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

exrhizo Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lmeyerov Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

exrhizo commented Jun 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

exrhizo commented Jun 10, 2025 •

edited

Loading

lmeyerov Jun 10, 2025 •

edited

Loading

exrhizo Jun 10, 2025 •

edited

Loading

lmeyerov Jun 10, 2025 •

edited

Loading

lmeyerov Jun 10, 2025 •

edited

Loading

lmeyerov Jun 10, 2025 •

edited

Loading

lmeyerov Jun 10, 2025 •

edited

Loading