Proposal: Yaml-based abstraction of business logic, excluding node internals and formatting/layout

[aaronsteers]

I'd like to be able to commit the output of langflow to a repo in a human-readable fashion. And I'd like to then be able to modify the yaml

The export to json today essentially has a superset of what we'd want to manage in git.

Some cosmetic info, such as node x and y locations may be lost in the translation. I can think of three options for how to deal with cosmetic layout info:

• Option 1: layout info is not saved.
• Option 2: layout info is saved to a separate file.
• Option 3: layout info is optionally saved to yaml, or not, based on user preference.

In all of these options, if there is some missing node layout info, like missing x and y coords, then the library may put them in an awkward location, or else do its best to position them logically in a decent position relative to the nodes it connects to.

While it would be a significant amount of effort to decouple the business logic from the node/graph layout info, this extends the core value proposition of LangFlow being an accessible means of authoring workflows, and extends that value-prop to also have copy-pasteable and having the ability for users to yaml accessibility.

[ogabrielluiz]

I completely agree with you.
There are many attributes that are passed back to the backend that are used only in the frontend.

Your suggestions are valid and we have thought about some of them before.

To me, the approach we should take: don't save the layout info and have a simple logic to auto-layout (I think react-flow has that functionality) when loading them back.

Regarding YAML, It would be possible to translate the JSON into YAML but that would require a stable API for the templates which we are constantly adapting as we see more functions being implemented and we define better patterns.

The YAML could be something like:

Nodes:
	node_id1
		- LLMChain:
			- llm: OpenAI
			- prompt: PromptTemplate
	node_id2
		- OpenAI:
			- model_name: text-davinci-003
			etc.

But then we'd still have to save the graph info (nodes and edges) somewhere which would be kind of weird. We can brainstorm a bit more, I think.

Thanks for bringing this up.

[ogabrielluiz]

The Ids are the problem in this case. If I have two Tool nodes they need to be unique in some way.

I have this example flow that uses the PythonFunction node to generate an image.
I ran the following code:

import json
import yaml
with open('Imageflow.json', 'r') as f:
    data = json.load(f)

def template_to_dict(template):
    result = {}
    for key in template.keys():
       if key == '_type':
            result[key] = template[key]
            continue
       templ_dict = template[key]
       result[key] = templ_dict.get('value', None)
    return result

def nodes_to_list(nodes):
    result = []
    for node in nodes:
        result.append(node_to_dict(node))
    return result

def edges_to_list(edges):
    result = []
    for edge in edges:
        result.append({'source': edge['source'], 'target': edge['target']})
    return result

def node_to_dict(node):
    result = {}
    result['id'] = node['id']
    result['template'] = template_to_dict(node['data']['node']['template'])
    return result



# data json has a data key, which with edges and nodes keys
nodes = data['data']['nodes']
edges = data['data']['edges']

nodes_list = nodes_to_list(nodes)
edges_list = edges_to_list(edges)

workflow_data = {
    'nodes': nodes_list,
    'edges': edges_list
}

yaml_data = yaml.dump(workflow_data, sort_keys=False)
with open('Imageflow.yaml', 'w') as f:
    yaml.dump(workflow_data, f)

The result is this:

edges:
  - source: dndnode_7
    target: dndnode_1
  - source: dndnode_1
    target: dndnode_3
  - source: dndnode_4
    target: dndnode_3
  - source: dndnode_8
    target: dndnode_3
nodes:
  - id: dndnode_1
    template:
      _type: Tool
      description: Use this to pass a prompt and generate an image with DALL-E2
      func: null
      name: ImageGenerator
      return_direct: true
  - id: dndnode_3
    template:
      _type: initailize_agent
      agent: zero-shot-react-description
      llm: null
      memory: null
      tools: null
  - id: dndnode_4
    template:
      _type: OpenAI
      batch_size: 20
      best_of: 1
      cache: null
      client: null
      frequency_penalty: 0
      logit_bias: null
      max_retries: 6
      max_tokens: 256
      model_kwargs: null
      model_name: text-davinci-003
      n: 1
      openai_api_key: sk-
      presence_penalty: 0
      request_timeout: null
      streaming: false
      temperature: "0.0"
      top_p: 1
      verbose: null
  - id: dndnode_7
    template:
      _type: python_function
      code:
        "import openai\nfrom langflow import cache_manager\nimport io\nimport base64\n\
        import os\nfrom PIL import Image\n\ndef create_image(prompt: str) -> str:\n\
        \    openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n\n    response = openai.Image.create(\n\
        \        prompt=prompt,\n        n=1,\n        size=\"256x256\",\n        response_format=\"\
        b64_json\",\n    )\n    # load the image\n    for image_dict in response[\"\
        data\"]:\n        image_data = base64.b64decode(image_dict[\"b64_json\"])\n\
        \        # Cache image in LangFlow\n        cache_manager.add_image(\"webinar\"\
        , Image.open(io.BytesIO(image_data)))\n        # save the image to disk\n  \
        \      with open(\"image.png\", \"wb\") as f:\n            f.write(image_data)\n\
        \        \n\n    return \"Image generated!\""
  - id: dndnode_8
    template:
      _type: ConversationBufferMemory
      ai_prefix: AI
      chat_memory: null
      human_prefix: Human
      input_key: null
      memory_key: history
      output_key: null
      return_messages: null

I think it is a pretty good result. The frontend would have to process the imported file differently probably here:
https://github.com/logspace-ai/langflow/blob/69c7bbb0a9cf8764e404899b16ad37e22de606a6/src/frontend/src/contexts/tabsContext.tsx#L139

It is a lot cleaner with a lot less data being passed around.

What do you think of this approach?

We got to keep in mind that we have to rebuild the graph.

[aaronsteers]

It is a lot cleaner with a lot less data being passed around.

What do you think of this approach?

Love it! 👍

We got to keep in mind that we have to rebuild the graph.

Yeah - makes sense!

@ogabrielluiz - The example yaml you have above looks great. It's a nice balance between generic and graph-like declarations, while also being meaningfully editable by a human. It passes my general tests for something being git-friendly or devops-friendly:

1. A reviewer on a PR can see what is changed and approve the change without needing to spin up the GUI client.
2. A creator can pretty easily make small changes (tweaking parameters, fixing typos, etc.) directly in code (yaml, in this case).
3. A creator can copy-paste sections of the code to duplicate a component and tweak manually the second version. (They'd presumably also need to accurately update the edges definitions, but it is definitely feasible.)

All of these (for me at least) are signs that creators and reviewers can iterate quickly and efficiently together - which I think is important when the tool starts being used in more production use cases.

Wdyt?

[ogabrielluiz]

I completely agree with your points. We should iterate over the solution with them in mind for sure.

One issue (I'm dealing with it right now) is that changes to LangChain API or ours break older exports. The Yaml would help this a lot because we can simply edit it to remove or add certain parameters plus, the templates wouldn't be tied to the imported file.

[ogabrielluiz]

I think the most work will come from implementing auto-layout in ReactFlow.

[aaronsteers]

One issue (I'm dealing with it right now) is that changes to LangChain API or ours break older exports. The Yaml would help this a lot because we can simply edit it to remove or add certain parameters plus, the templates wouldn't be tied to the imported file.

Makes sense! 👍

I think the most work will come from implementing auto-layout in ReactFlow.

If layout would otherwise be a blocker, what do you think about temporarily adding to each node item something like position: [x, y] or canvas_position: [x, y] in the yaml format?

Presumably that could continue to exist for a while, preserving general layout and flow during serialization/deserialization - and in a backwards-compatible way. If a user were to manually omit or delete the [x, y] positions with the yaml file, that would not break any behaviors, but would just lead to a very messy/chaotic layout - until/unless such time as an auto-layout feature is added in the future. Conversely, when auto-layout feature exists, the load operation could either ignore or apply use the [x, y] as overrides, without a huge impact to users either way as long as behaviors are consistent and documented.

Maybe not necessary if auto-layout turns out to be a smaller scope... but wdyt in general?

[aaronsteers]

Just as a bookmark for later discussion, but not necessary in a first iteration:

1. Deterministic ordering of outputs is one thing we should watch out for in the long run. Meaning, if no changes are made between exports, the file should not reorder declarations: node or edge. (Tools that don't do this often create large diffs in git even when nothing has changed.)
2. Might be worth using a library like Ruamel.yaml, which is slower than PyYaml, but has the advantage of preserving cosmetic changes between import and export. Can be tackled later, but might be an easy early win to start with this library and get certain behaviors out of box.