Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add new node Input type for data in table format #2635

Merged
merged 55 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
d0b74dc
feat: add Table component and related functionality
anovazzi1 Jul 11, 2024
a904ae3
feat: add Edit Data trigger to TableNodeComponent
anovazzi1 Jul 11, 2024
e4bb1d7
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 11, 2024
2a28a32
feat: add TableSchema class for defining table structure
ogabrielluiz Jul 11, 2024
94432a1
feat: add TableMixin class for table-related functionality
ogabrielluiz Jul 11, 2024
5e993a8
feat: add TableInput class for table-related functionality
ogabrielluiz Jul 11, 2024
6482d8b
feat: add TableInput to io module
ogabrielluiz Jul 11, 2024
a9625b2
feat: update Column model in table schema
ogabrielluiz Jul 11, 2024
b3be858
feat: add displayEmptyAlert prop to TableComponent
anovazzi1 Jul 11, 2024
27d33fb
This commit improves the TableAutoCellRender component by adding supp…
anovazzi1 Jul 11, 2024
ffe83aa
feat: add FormatColumns function to utils.ts
anovazzi1 Jul 11, 2024
2116ed4
feat: enhance TableNodeComponent with FormatColumns function
anovazzi1 Jul 11, 2024
2ff19a9
chore: Update TableNodeComponent and TableComponent
anovazzi1 Jul 11, 2024
58e6c5b
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 11, 2024
86b1dfa
feat: Update TableNodeComponent and TableComponent
anovazzi1 Jul 12, 2024
155ee70
feat: initialize table field values as DataFrame
ogabrielluiz Jul 11, 2024
ffddc39
feat: Enhance TableNodeComponent with duplicateRow function
anovazzi1 Jul 12, 2024
2a5a80b
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 12, 2024
caafa75
feat: Remove "text" from basic_types in FormatColumns function
anovazzi1 Jul 12, 2024
8ea5ecb
fix: alingment bug on AgGrid cell
anovazzi1 Jul 12, 2024
498bbb0
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 12, 2024
e24a8ee
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 12, 2024
c69509f
Styled the Open Table button on TableNodeComponent
lucaseduoli Jul 16, 2024
5f5b234
Fixed type of ref on tableComponent
lucaseduoli Jul 16, 2024
f7d5b68
Creaed a TableModal component, that receives the props that are passe…
lucaseduoli Jul 16, 2024
0dbf012
Used the TableModal on the TableNodeComponent
lucaseduoli Jul 16, 2024
56690e7
Fixed looks of TableModal
lucaseduoli Jul 16, 2024
4df6b6d
Added description set on tableModal
lucaseduoli Jul 16, 2024
1cfed20
Add description field to TableNodeComponent
lucaseduoli Jul 16, 2024
924ec7f
Fixed text of description if info is not provided
lucaseduoli Jul 16, 2024
5d1b6ec
Added TableComponent in tableNodeCellRenderer
lucaseduoli Jul 16, 2024
e784a7b
Added styling based on editNode
lucaseduoli Jul 16, 2024
953b426
Added Auto Size to table modal
lucaseduoli Jul 16, 2024
415c626
refactor: update TableOptions component styling and behavior
ogabrielluiz Jul 16, 2024
130cb0a
chore: Remove unnecessary imports and initialize empty columns array …
ogabrielluiz Jul 16, 2024
b959053
feat: Add default values for sortable and filterable in Column model
ogabrielluiz Jul 16, 2024
5daab25
feat(utils.ts): add check for empty columns array in FormatColumns fu…
ogabrielluiz Jul 16, 2024
3fc5286
feat: Add validation for TableInput value in inputs.py
ogabrielluiz Jul 16, 2024
b5f6e85
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 16, 2024
13c7c8a
feat: extend editable field to json field
anovazzi1 Jul 17, 2024
1430e7d
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 17, 2024
c9b398b
feat: Add validation for TableInput value in inputs.py
ogabrielluiz Jul 18, 2024
ec98ec8
feat(validate.py): add exception handling to catch and re-raise Valid…
ogabrielluiz Jul 18, 2024
f39e3eb
chore: Refactor error message in build_custom_component_template func…
ogabrielluiz Jul 18, 2024
8cf1311
fix(validate.py): improve error message formatting in create_class fu…
ogabrielluiz Jul 18, 2024
88d0d94
feat: Update TableMixin to support TableSchema or list of Columns
ogabrielluiz Jul 18, 2024
df98ae9
feat: Update TableNodeComponent to generate backend columns from value
anovazzi1 Jul 18, 2024
264bd9f
Refactor extractColumnsFromRows function to return only ColDef objects
anovazzi1 Jul 18, 2024
1b61987
Merge branch 'main' into TableInput
ogabrielluiz Jul 18, 2024
46e61b5
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 18, 2024
cdf240e
refactor: Generate backend columns from value in TableNodeComponent
anovazzi1 Jul 19, 2024
bb04ddc
feat: Update TableNodeComponent to handle number and date properly
anovazzi1 Jul 19, 2024
00575c3
fix bug that delete all rows on modal close
anovazzi1 Jul 19, 2024
f9ad887
Merge branch 'main' into TableInput
ogabrielluiz Jul 22, 2024
94a0970
[autofix.ci] apply automated fixes
autofix-ci[bot] Jul 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/backend/base/langflow/custom/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,7 @@ def build_custom_component_template(
raise HTTPException(
status_code=400,
detail={
"error": (f"Something went wrong while building the custom component. Hints: {str(exc)}"),
"error": (f"Error building Component: {str(exc)}"),
"traceback": traceback.format_exc(),
},
) from exc
Expand Down
8 changes: 8 additions & 0 deletions src/backend/base/langflow/graph/vertex/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from enum import Enum
from typing import TYPE_CHECKING, Any, AsyncIterator, Callable, Dict, Iterator, List, Mapping, Optional, Set

import pandas as pd
from loguru import logger

from langflow.exceptions.component import ComponentBuildException
Expand Down Expand Up @@ -373,6 +374,13 @@ def _build_params(self):
params[field_name] = val
elif isinstance(val, str):
params[field_name] = val != ""
elif field.get("type") == "table" and val is not None:
# check if the value is a list of dicts
# if it is, create a pandas dataframe from it
if isinstance(val, list) and all(isinstance(item, dict) for item in val):
params[field_name] = pd.DataFrame(val)
else:
raise ValueError(f"Invalid value type {type(val)} for field {field_name}")
elif val is not None and val != "":
params[field_name] = val

Expand Down
2 changes: 2 additions & 0 deletions src/backend/base/langflow/inputs/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
PromptInput,
SecretStrInput,
StrInput,
TableInput,
)

__all__ = [
Expand All @@ -34,4 +35,5 @@
"SecretStrInput",
"StrInput",
"MessageTextInput",
"TableInput",
]
15 changes: 15 additions & 0 deletions src/backend/base/langflow/inputs/input_mixin.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from langflow.field_typing.range_spec import RangeSpec
from langflow.inputs.validators import CoalesceBool
from langflow.schema.table import Column, TableSchema


class FieldTypes(str, Enum):
Expand All @@ -18,6 +19,7 @@ class FieldTypes(str, Enum):
FILE = "file"
PROMPT = "prompt"
OTHER = "other"
TABLE = "table"


SerializableFieldTypes = Annotated[FieldTypes, PlainSerializer(lambda v: v.value, return_type=str)]
Expand Down Expand Up @@ -136,3 +138,16 @@ class DropDownMixin(BaseModel):

class MultilineMixin(BaseModel):
multiline: CoalesceBool = True


class TableMixin(BaseModel):
table_schema: Optional[TableSchema | list[Column]] = None

@field_validator("table_schema")
@classmethod
def validate_table_schema(cls, v):
if isinstance(v, list) and all(isinstance(column, Column) for column in v):
return TableSchema(columns=v)
if isinstance(v, TableSchema):
return v
raise ValueError("table_schema must be a TableSchema or a list of Columns")
21 changes: 21 additions & 0 deletions src/backend/base/langflow/inputs/inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,29 @@
MultilineMixin,
RangeMixin,
SerializableFieldTypes,
TableMixin,
)


class TableInput(BaseInputMixin, MetadataTraceMixin, TableMixin, ListableInputMixin):
field_type: Optional[SerializableFieldTypes] = FieldTypes.TABLE
is_list: bool = True

@field_validator("value")
@classmethod
def validate_value(cls, v: Any, _info):
# Check if value is a list of dicts
if not isinstance(v, list):
raise ValueError(f"TableInput value must be a list of dictionaries or Data. Value '{v}' is not a list.")

for item in v:
if not isinstance(item, (dict, Data)):
raise ValueError(
f"TableInput value must be a list of dictionaries or Data. Item '{item}' is not a dictionary or Data."
)
return v


class HandleInput(BaseInputMixin, ListableInputMixin, MetadataTraceMixin):
"""
Represents an Input that has a Handle to a specific type (e.g. BaseLanguageModel, BaseRetriever, etc.)
Expand Down Expand Up @@ -338,4 +358,5 @@ class FileInput(BaseInputMixin, ListableInputMixin, FileMixin, MetadataTraceMixi
StrInput,
MessageTextInput,
MessageInput,
TableInput,
]
2 changes: 2 additions & 0 deletions src/backend/base/langflow/io/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
PromptInput,
SecretStrInput,
StrInput,
TableInput,
)
from langflow.template import Output

Expand All @@ -36,4 +37,5 @@
"StrInput",
"MessageTextInput",
"Output",
"TableInput",
]
31 changes: 31 additions & 0 deletions src/backend/base/langflow/schema/table.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from enum import Enum
from typing import List, Optional

from pydantic import BaseModel, Field, field_validator


class FormatterType(str, Enum):
date = "date"
text = "text"
number = "number"
json = "json"


class Column(BaseModel):
display_name: str
name: str
sortable: bool = Field(default=True)
filterable: bool = Field(default=True)
formatter: Optional[FormatterType | str] = None

@field_validator("formatter")
def validate_formatter(cls, value):
if isinstance(value, str):
return FormatterType(value)
if isinstance(value, FormatterType):
return value
raise ValueError("Invalid formatter type")


class TableSchema(BaseModel):
columns: List[Column]
10 changes: 8 additions & 2 deletions src/backend/base/langflow/utils/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
from types import FunctionType
from typing import Dict, List, Optional, Union

from pydantic import ValidationError

from langflow.field_typing.constants import CUSTOM_COMPONENT_SUPPORTED_TYPES


Expand Down Expand Up @@ -168,8 +170,12 @@ def create_class(code, class_name):

class_code = extract_class_code(module, class_name)
compiled_class = compile_class_code(class_code)

return build_class_constructor(compiled_class, exec_globals, class_name)
try:
return build_class_constructor(compiled_class, exec_globals, class_name)
except ValidationError as e:
messages = [error["msg"].split(",", 1) for error in e.errors()]
error_message = "\n".join([message[1] if len(message) > 1 else message[0] for message in messages])
raise ValueError(error_message) from e


def create_type_ignore_class():
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import TableNodeComponent from "@/components/TableNodeComponent";
import { cloneDeep } from "lodash";
import { ReactNode, useEffect, useRef, useState } from "react";
import { useHotkeys } from "react-hotkeys-hook";
Expand Down Expand Up @@ -524,6 +525,17 @@ export default function ParameterComponent({
/>
</div>
</Case>
<Case condition={left === true && type === "table"}>
<div className="mt-2 w-full">
<TableNodeComponent
description={info || "Add or edit data"}
columns={data.node?.template[name]?.table_schema?.columns}
onChange={handleOnNewValue}
tableTitle={data.node?.template[name]?.display_name ?? "Table"}
value={data.node?.template[name]?.value}
/>
</div>
</Case>

<Case
condition={
Expand Down
149 changes: 149 additions & 0 deletions src/frontend/src/components/TableNodeComponent/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
import TableModal from "@/modals/tableModal";
import { FormatColumns, generateBackendColumnsFromValue } from "@/utils/utils";
import { DataTypeDefinition, SelectionChangedEvent } from "ag-grid-community";
import { AgGridReact } from "ag-grid-react";
import { cloneDeep } from "lodash";
import { useMemo, useRef, useState } from "react";
import { ForwardedIconComponent } from "../../components/genericIconComponent";
import { TableComponentType } from "../../types/components";
import { Button } from "../ui/button";

export default function TableNodeComponent({
tableTitle,
description,
value,
onChange,
editNode = false,
id = "",
columns,
}: TableComponentType): JSX.Element {
const dataTypeDefinitions: {
[cellDataType: string]: DataTypeDefinition<any>;
} = useMemo(() => {
return {
// override `date` to handle custom date format `dd/mm/yyyy`
date: {
baseDataType: "date",
extendsDataType: "date",
valueParser: (params) => {
if (params.newValue == null) {
return null;
}
// convert from `dd/mm/yyyy`
const dateParts = params.newValue.split("/");
return dateParts.length === 3
? new Date(
parseInt(dateParts[2]),
parseInt(dateParts[1]) - 1,
parseInt(dateParts[0]),
)
: null;
},
valueFormatter: (params) => {
let date = params.value;
if (typeof params.value === "string") {
date = new Date(params.value);
}
// convert to `dd/mm/yyyy`
return date == null
? "‎"
: `${date.getDate()}/${date.getMonth() + 1}/${date.getFullYear()}`;
},
},
number: {
baseDataType: "number",
extendsDataType: "number",
valueFormatter: (params) =>
params.value == null ? "‎" : `${params.value}`,
},
};
}, []);
const [selectedNodes, setSelectedNodes] = useState<Array<any>>([]);
const agGrid = useRef<AgGridReact>(null);
const componentColumns = columns
? columns
: generateBackendColumnsFromValue(value ?? []);
const AgColumns = FormatColumns(componentColumns);
function setAllRows() {
if (agGrid.current && !agGrid.current.api.isDestroyed()) {
const rows: any = [];
agGrid.current.api.forEachNode((node) => rows.push(node.data));
onChange(rows);
}
}
function deleteRow() {
if (agGrid.current && selectedNodes.length > 0) {
agGrid.current.api.applyTransaction({
remove: selectedNodes.map((node) => node.data),
});
setSelectedNodes([]);
setAllRows();
}
}
function duplicateRow() {
if (agGrid.current && selectedNodes.length > 0) {
const toDuplicate = selectedNodes.map((node) => cloneDeep(node.data));
setSelectedNodes([]);
const rows: any = [];
onChange([...value, ...toDuplicate]);
}
}
function addRow() {
const newRow = {};
componentColumns.forEach((column) => {
newRow[column.name] = null;
});
onChange([...value, newRow]);
}

function updateComponent() {
setAllRows();
}
const editable = componentColumns.map((column) => {
const isCustomEdit =
column.formatter &&
(column.formatter === "text" || column.formatter === "json");
return {
field: column.name,
onUpdate: updateComponent,
editableCell: isCustomEdit ? false : true,
};
});

return (
<div className={"flex w-full items-center"}>
<div className="flex w-full items-center gap-3" data-testid={"div-" + id}>
<TableModal
dataTypeDefinitions={dataTypeDefinitions}
autoSizeStrategy={{ type: "fitGridWidth", defaultMinWidth: 100 }}
tableTitle={tableTitle}
description={description}
ref={agGrid}
onSelectionChanged={(event: SelectionChangedEvent) => {
setSelectedNodes(event.api.getSelectedNodes());
}}
rowSelection="multiple"
suppressRowClickSelection={true}
editable={editable}
pagination={true}
addRow={addRow}
onDelete={deleteRow}
onDuplicate={duplicateRow}
displayEmptyAlert={false}
className="h-full w-full"
columnDefs={AgColumns}
rowData={value}
>
<Button
variant="primary"
size={editNode ? "xs" : "default"}
className="w-full"
>
<ForwardedIconComponent name="Table" className="mt-px h-4 w-4" />
<span className="font-normal">Open Table</span>
</Button>
</TableModal>
</div>
</div>
);
}
18 changes: 13 additions & 5 deletions src/frontend/src/components/objectRender/index.tsx
Original file line number Diff line number Diff line change
@@ -1,12 +1,20 @@
import DictAreaModal from "../../modals/dictAreaModal";

export default function ObjectRender({ object }: { object: any }): JSX.Element {
//TODO check object type

export default function ObjectRender({
object,
setValue,
}: {
object: any;
setValue?: (value: any) => void;
}): JSX.Element {
let preview =
object === null || object === undefined ? "‎" : JSON.stringify(object);
if (object === null || object === undefined) {
}
return (
<DictAreaModal value={object}>
<DictAreaModal onChange={setValue} value={object ?? {}}>
<div className="flex h-full w-full items-center align-middle transition-all">
<div className="truncate">{JSON.stringify(object)}</div>
<div className="truncate">{preview}</div>
</div>
</DictAreaModal>
);
Expand Down
Loading