diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..5869a0eec --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,107 @@ +# Querybook + +Querybook is Pinterest's open-source Big Data IDE for discovering, creating, and sharing data analyses. It combines a rich-text editor, SQL query engine, charting, scheduling, and table documentation in a single web app. + +## Tech Stack + +- **Backend:** Python 3.10, Flask, SQLAlchemy (MySQL), Celery (Redis broker), Elasticsearch/OpenSearch, gevent + Flask-SocketIO (WebSockets), uWSGI (production) +- **Frontend:** React 17, TypeScript, Redux, Webpack 5, CodeMirror (SQL editor), Draft.js (rich text), Chart.js/D3/ReactFlow + +## Directory Layout + +- `querybook/server/` — Flask backend + - `app/` — app setup + - `datasources/` — REST API endpoints + - `logic/` — business logic + - `models/` — SQLAlchemy models + - `tasks/` — Celery tasks + - `lib/` — utilities, executors, metastores + - `env.py` — `QuerybookSettings` configuration +- `querybook/webapp/` — React/TypeScript frontend + - `components/` — React components + - `hooks/` — custom React hooks + - `redux/` — Redux store, actions, reducers + - `lib/` — frontend utilities + - `ui/` — reusable UI primitives + - `resource/` — API client layer +- `querybook/config/` — YAML config files +- `plugins/` — plugin stubs (extension point for custom behavior) +- `requirements/` — pip requirements (`base.txt`, `prod.txt`, `engine/*.txt`, `auth/*.txt`) +- `containers/` — Docker Compose files (dev, prod, test) +- `docs_website/` — Docusaurus documentation site +- `helm/` / `k8s/` — Kubernetes deployment manifests + +## Plugin System + +Querybook is extended via plugins without forking. The env var `QUERYBOOK_PLUGIN` (default `./plugins`) points to a directory where plugin modules are discovered by `lib.utils.import_helper.import_module_with_default()`. + +Each plugin module exports a well-known variable (e.g. `ALL_PLUGIN_EXECUTORS`) that the server merges with built-in defaults. + +Key plugin types: `executor_plugin`, `metastore_plugin`, `auth_plugin`, `api_plugin`, `exporter_plugin`, `result_store_plugin`, `notifier_plugin`, `event_logger_plugin`, `stats_logger_plugin`, `job_plugin`, `tasks_plugin`, `dag_exporter_plugin`, `ai_assistant_plugin`, `vector_store_plugin`, `webpage_plugin`, `monkey_patch_plugin`, `query_validation_plugin`, `query_transpilation_plugin`, `engine_status_checker_plugin`, `table_uploader_plugin`. + +## Configuration + +Priority: **env vars > `querybook_config.yaml` > `querybook_default_config.yaml`**. + +Key settings live in `querybook/server/env.py` (`QuerybookSettings`). + +## Running Locally + +Start the full stack (web server, worker, scheduler, and all dependencies) with Docker Compose: + +```bash +make +``` + +This brings up everything and serves the app at http://localhost:10001. This is the primary command for local development. + +To restart individual services without bouncing the full stack: + +```bash +make web # web server only +make worker # celery worker +make scheduler # celery beat +``` + +## Making Commits + +When preparing a PR, run the relevant checks. CI runs all of the following via GitHub Actions (`.github/workflows/`), but must be manually triggered by a maintainer. + +Always run tests via `make test`, which builds a `querybook-test` Docker image and runs checks inside it. This ensures an isolated, reproducible environment. Do not run test commands (pytest, yarn, webpack) directly on the host. + +`make test` runs both backend and frontend checks: +- **Backend** (anything under `querybook/server/`): pytest +- **Frontend** (anything under `querybook/webapp/`): TypeScript type checking, Jest unit tests, ESLint, and production build verification + +**Formatting (all changes) — common CI failure:** + +`make test` does **not** run Prettier. CI runs Prettier separately via `pre-commit`, so formatting issues are a frequent cause of CI failures. After running `make test`, also run Prettier on changed files before pushing: + +```bash +npx prettier --write +``` + +For a full formatting pass (Black for Python, Prettier for JS/TS, flake8): + +```bash +pre-commit run --all-files +``` + +## Maintaining This File + +**Include:** +- Repo purpose, tech stack, and high-level architecture +- Directory layout (key paths only) +- How to run, test, and lint locally +- Commit and PR workflow expectations +- Plugin system overview and extension points + +**Do not include:** +- Detailed API docs or function-level documentation +- Inline code examples longer than 5 lines +- Deployment runbooks or operational procedures (keep in README or docs/) +- Credentials, secrets, or internal URLs +- Information that changes frequently (version numbers, dependency lists) +- Content already covered in README.md +- Content that can be easily derived by AI agents (e.g. reading file trees, package.json) +- References to internal/proprietary repos — this is an open-source project diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 000000000..43c994c2d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/README.md b/README.md index 55e613010..3f6373464 100644 --- a/README.md +++ b/README.md @@ -106,4 +106,4 @@ Lineage & Analytics # Contributing Back -See [CONTRIBUTING](CONTRIBUTING.md). +See [CONTRIBUTING](CONTRIBUTING.md) for the full guide, including [how to run tests locally](docs_website/docs/developer_guide/contributing.mdx#testing). diff --git a/docs_website/docs/developer_guide/contributing.mdx b/docs_website/docs/developer_guide/contributing.mdx index cd8db994e..69882a6e1 100644 --- a/docs_website/docs/developer_guide/contributing.mdx +++ b/docs_website/docs/developer_guide/contributing.mdx @@ -65,6 +65,37 @@ To increase the chances that your pull request will be accepted: - Write tests for your changes - Write a good commit message +### Testing + +CI runs these checks on every pull request via GitHub Actions (a maintainer may need to approve the workflow run for external contributors). You should reproduce them locally before pushing. + +The simplest way to run the full test suite locally is: + +```sh +make test +``` + +This builds a Docker test image and runs both Python and Node tests in parallel, mirroring CI. + +If you only changed one side and want a faster feedback loop: + +```sh +# Backend only (changes under querybook/server/) +PYTHONPATH=querybook/server:plugins ./querybook/scripts/run_test --python + +# Frontend only (changes under querybook/webapp/) +./querybook/scripts/run_test --node +``` + +**Linting (all changes):** + +Pre-commit hooks (Black, Prettier, flake8) run automatically on `git commit` if installed, and in CI on every PR. To set them up locally: + +```sh +pip install pre-commit +pre-commit install +``` + ### Pull Request Format When you create a new pull request, please make sure it's title follows the specifications in https://www.conventionalcommits.org/en/v1.0.0/. This is to ensure automatic versioning. diff --git a/querybook/webapp/__tests__/hooks/queryEditor/useExecutionSnapshots.test.ts b/querybook/webapp/__tests__/hooks/queryEditor/useExecutionSnapshots.test.ts new file mode 100644 index 000000000..e07ab7f3e --- /dev/null +++ b/querybook/webapp/__tests__/hooks/queryEditor/useExecutionSnapshots.test.ts @@ -0,0 +1,37 @@ +import { addRunInputSnapshot } from 'hooks/queryEditor/useExecutionSnapshots'; + +describe('addRunInputSnapshot', () => { + test('adds a snapshot to an empty record', () => { + const result = addRunInputSnapshot({}, 1, 'SELECT 1'); + expect(result).toEqual({ 1: 'SELECT 1' }); + }); + + test('adds a new execution to existing snapshots', () => { + const prev = { 1: 'SELECT 1' }; + const result = addRunInputSnapshot(prev, 2, 'SELECT 2'); + expect(result).toEqual({ 1: 'SELECT 1', 2: 'SELECT 2' }); + }); + + test('overwrites an existing execution snapshot', () => { + const prev = { 1: 'SELECT old' }; + const result = addRunInputSnapshot(prev, 1, 'SELECT new'); + expect(result).toEqual({ 1: 'SELECT new' }); + }); + + test('does not mutate the original record', () => { + const prev = { 1: 'SELECT 1' }; + const result = addRunInputSnapshot(prev, 2, 'SELECT 2'); + expect(prev).toEqual({ 1: 'SELECT 1' }); + expect(result).not.toBe(prev); + }); + + test('handles many snapshots without pruning', () => { + let record: Record = {}; + for (let i = 0; i < 50; i++) { + record = addRunInputSnapshot(record, i, `SELECT ${i}`); + } + expect(Object.keys(record)).toHaveLength(50); + expect(record[0]).toBe('SELECT 0'); + expect(record[49]).toBe('SELECT 49'); + }); +}); diff --git a/querybook/webapp/__tests__/hooks/queryEditor/useStaleQueryWarning.test.ts b/querybook/webapp/__tests__/hooks/queryEditor/useStaleQueryWarning.test.ts new file mode 100644 index 000000000..2740a72f9 --- /dev/null +++ b/querybook/webapp/__tests__/hooks/queryEditor/useStaleQueryWarning.test.ts @@ -0,0 +1,210 @@ +import { + shouldComputeStaleWarning, + computeStaleWarningState, +} from 'hooks/queryEditor/useStaleQueryWarning'; + +describe('shouldComputeStaleWarning', () => { + const snapshots: Record = { + 1: 'SELECT * FROM t1', + 2: 'SELECT * FROM t2', + }; + + test('returns false when selectedExecutionId is null', () => { + expect( + shouldComputeStaleWarning(null, snapshots, 'SELECT * FROM t1') + ).toBe(false); + }); + + test('returns false when selectedExecutionId is undefined', () => { + expect( + shouldComputeStaleWarning(undefined, snapshots, 'SELECT * FROM t1') + ).toBe(false); + }); + + test('returns false when snapshot matches current input', () => { + expect( + shouldComputeStaleWarning(1, snapshots, 'SELECT * FROM t1') + ).toBe(false); + }); + + test('returns true when snapshot differs from current input', () => { + expect( + shouldComputeStaleWarning(1, snapshots, 'SELECT * FROM changed') + ).toBe(true); + }); + + test('returns false when execution has no snapshot and no initialQuery', () => { + expect( + shouldComputeStaleWarning(99, snapshots, 'SELECT * FROM t1') + ).toBe(false); + }); + + describe('initialQuery fallback', () => { + const initialQuery = 'SELECT 1'; + + test('uses initialQuery when no snapshot exists for the execution', () => { + expect( + shouldComputeStaleWarning( + 99, + snapshots, + 'SELECT 2', + initialQuery + ) + ).toBe(true); + }); + + test('returns false when current input matches initialQuery (no snapshot)', () => { + expect( + shouldComputeStaleWarning( + 99, + snapshots, + 'SELECT 1', + initialQuery + ) + ).toBe(false); + }); + + test('prefers in-memory snapshot over initialQuery', () => { + expect( + shouldComputeStaleWarning( + 1, + snapshots, + 'SELECT * FROM t1', + 'something else' + ) + ).toBe(false); + }); + + test('warns when input differs from snapshot even if initialQuery matches', () => { + expect( + shouldComputeStaleWarning( + 1, + snapshots, + 'SELECT * FROM changed', + 'SELECT * FROM changed' + ) + ).toBe(true); + }); + }); + + test('works with empty snapshots and no initialQuery', () => { + expect(shouldComputeStaleWarning(1, {}, 'any query')).toBe(false); + }); + + test('works with empty snapshots and initialQuery provided', () => { + expect(shouldComputeStaleWarning(1, {}, 'changed', 'original')).toBe( + true + ); + }); +}); + +describe('computeStaleWarningState', () => { + const snapshots: Record = { + 1: 'SELECT * FROM t1', + 2: 'SELECT * FROM t2', + }; + + const base = { + selectedExecutionId: 1 as number | null, + snapshots, + initialQuery: undefined as string | undefined, + }; + + test('returns null when live input matches snapshot', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM t1', + liveInput: 'SELECT * FROM t1', + }) + ).toBeNull(); + }); + + test('returns null when no execution is selected', () => { + expect( + computeStaleWarningState({ + ...base, + selectedExecutionId: null, + savedInput: 'SELECT * FROM t1', + liveInput: 'SELECT * FROM edited', + }) + ).toBeNull(); + }); + + test('returns "edited" when both live and saved differ from snapshot and not saving', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM edited', + liveInput: 'SELECT * FROM edited', + }) + ).toBe('edited'); + }); + + test('returns "unsaved" when live differs but saved still matches snapshot', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM t1', + liveInput: 'SELECT * FROM edited', + }) + ).toBe('unsaved'); + }); + + test('returns "unsaved" when both differ from snapshot but save is in-flight', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM edited', + liveInput: 'SELECT * FROM edited', + isSaving: true, + }) + ).toBe('unsaved'); + }); + + test('returns "unsaved" when live differs and saved matches snapshot while saving', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM t1', + liveInput: 'SELECT * FROM edited', + isSaving: true, + }) + ).toBe('unsaved'); + }); + + test('returns null when live matches snapshot even if saving', () => { + expect( + computeStaleWarningState({ + ...base, + savedInput: 'SELECT * FROM t1', + liveInput: 'SELECT * FROM t1', + isSaving: true, + }) + ).toBeNull(); + }); + + test('uses initialQuery fallback when no snapshot exists', () => { + expect( + computeStaleWarningState({ + ...base, + selectedExecutionId: 99, + savedInput: 'SELECT 1', + liveInput: 'SELECT 2', + initialQuery: 'SELECT 1', + }) + ).toBe('unsaved'); + }); + + test('returns "edited" with initialQuery when both differ', () => { + expect( + computeStaleWarningState({ + ...base, + selectedExecutionId: 99, + savedInput: 'SELECT 2', + liveInput: 'SELECT 2', + initialQuery: 'SELECT 1', + }) + ).toBe('edited'); + }); +}); diff --git a/querybook/webapp/components/DataDocQueryCell/DataDocQueryCell.tsx b/querybook/webapp/components/DataDocQueryCell/DataDocQueryCell.tsx index 81193ae60..cb5606bb0 100644 --- a/querybook/webapp/components/DataDocQueryCell/DataDocQueryCell.tsx +++ b/querybook/webapp/components/DataDocQueryCell/DataDocQueryCell.tsx @@ -13,6 +13,7 @@ import { DataDocQueryExecutions } from 'components/DataDocQueryExecutions/DataDo import { DataDocTableSamplingInfo } from 'components/DataDocTableSamplingInfo/DataDocTableSamplingInfo'; import { QueryCellTitle } from 'components/QueryCellTitle/QueryCellTitle'; import { runQuery, transformQuery } from 'components/QueryComposer/RunQuery'; +import { addRunInputSnapshot } from 'hooks/queryEditor/useExecutionSnapshots'; import { BoundQueryEditor } from 'components/QueryEditor/BoundQueryEditor'; import { IQueryEditorHandles } from 'components/QueryEditor/QueryEditor'; import { QueryPeerReviewModal } from 'components/QueryPeerReviewModal/QueryPeerReviewModal'; @@ -120,6 +121,8 @@ interface IState { showTableSamplingInfoModal: boolean; showPeerReviewModal: boolean; + executionRunInputById: Record; + transpilerConfig?: { toEngine: IQueryEngine; transpilerName: string; @@ -130,9 +133,11 @@ class DataDocQueryCellComponent extends React.PureComponent { private queryEditorRef = React.createRef(); private runButtonRef = React.createRef(); private commandInputRef = React.createRef(); + private readonly initialQuery: string; public constructor(props) { super(props); + this.initialQuery = props.query; this.state = { query: props.query, @@ -149,6 +154,7 @@ class DataDocQueryCellComponent extends React.PureComponent { samplingTables: {}, showTableSamplingInfoModal: false, showPeerReviewModal: false, + executionRunInputById: {}, }; } @@ -443,11 +449,17 @@ class DataDocQueryCellComponent extends React.PureComponent { return this.handleMetaChange('sample_rate', sampleRate); } + @bind + public getRunInputString(): string { + return ( + this.queryEditorRef.current?.getSelection?.() ?? this.state.query + ); + } + @bind public async getTransformedQuery() { const { templatedVariables = [] } = this.props; - const { query } = this.state; - const rawQuery = this.queryEditorRef.current?.getSelection?.() ?? query; + const rawQuery = this.getRunInputString(); return transformQuery( rawQuery, @@ -469,6 +481,17 @@ class DataDocQueryCellComponent extends React.PureComponent { return Object.keys(metadata).length === 0 ? null : metadata; } + @bind + public recordRunInputSnapshot(executionId: number, runInput: string) { + this.setState((prev) => ({ + executionRunInputById: addRunInputSnapshot( + prev.executionRunInputById, + executionId, + runInput + ), + })); + } + @bind public async executeQuery(options: { element: ElementType; @@ -502,6 +525,10 @@ class DataDocQueryCellComponent extends React.PureComponent { executionMetadata, peerReviewParams ); + this.recordRunInputSnapshot( + queryExecution.id, + this.state.query + ); return queryExecution.id; } ); @@ -596,12 +623,14 @@ class DataDocQueryCellComponent extends React.PureComponent { const executionMetadata = this.sampleRate > 0 ? { sample_rate: this.sampleRate } : null; - return this.props.createQueryExecution( + const queryExecution = await this.props.createQueryExecution( renderedQuery, this.engineId, this.props.cellId, executionMetadata ); + this.recordRunInputSnapshot(queryExecution.id, this.state.query); + return queryExecution; } } @@ -1077,6 +1106,10 @@ class DataDocQueryCellComponent extends React.PureComponent { onSamplingInfoClick={this.toggleShowTableSamplingInfoModal} hasSamplingTables={this.hasSamplingTables} sampleRate={this.sampleRate} + savedRunInput={this.props.query} + liveRunInput={this.state.query} + executionRunInputSnapshots={this.state.executionRunInputById} + initialQuery={this.initialQuery} /> ); } diff --git a/querybook/webapp/components/DataDocQueryExecutions/DataDocQueryExecutions.tsx b/querybook/webapp/components/DataDocQueryExecutions/DataDocQueryExecutions.tsx index 8badabfbe..d1c4b02b4 100644 --- a/querybook/webapp/components/DataDocQueryExecutions/DataDocQueryExecutions.tsx +++ b/querybook/webapp/components/DataDocQueryExecutions/DataDocQueryExecutions.tsx @@ -5,7 +5,7 @@ import React, { useMemo, useState, } from 'react'; -import { useDispatch } from 'react-redux'; +import { useDispatch, useSelector } from 'react-redux'; import { QueryExecutionPicker } from 'components/ExecutionPicker/QueryExecutionPicker'; import { QueryExecution } from 'components/QueryExecution/QueryExecution'; @@ -13,10 +13,13 @@ import { QueryExecutionDuration } from 'components/QueryExecution/QueryExecution import { QueryExecutionBar } from 'components/QueryExecutionBar/QueryExecutionBar'; import { DataDocContext } from 'context/DataDoc'; import { useMakeSelector } from 'hooks/redux/useMakeSelector'; +import { useStaleQueryWarning } from 'hooks/queryEditor/useStaleQueryWarning'; import { getQueryString } from 'lib/utils/query-string'; import * as queryExecutionsActions from 'redux/queryExecutions/action'; import * as queryExecutionsSelectors from 'redux/queryExecutions/selector'; +import { IStoreState } from 'redux/store/types'; import { StyledText } from 'ui/StyledText/StyledText'; +import { StaleQueryWarning } from './StaleQueryWarning'; interface IProps { docId: number; @@ -28,6 +31,11 @@ interface IProps { onSamplingInfoClick?: () => void; hasSamplingTables?: boolean; sampleRate: number; + + savedRunInput?: string; + liveRunInput?: string; + executionRunInputSnapshots?: Readonly>; + initialQuery?: string; } export const DataDocQueryExecutions: React.FunctionComponent = @@ -40,6 +48,10 @@ export const DataDocQueryExecutions: React.FunctionComponent = onSamplingInfoClick, hasSamplingTables, sampleRate, + savedRunInput, + liveRunInput, + executionRunInputSnapshots, + initialQuery, }) => { const { cellIdToExecutionId, onQueryCellSelectExecution } = useContext(DataDocContext); @@ -118,6 +130,25 @@ export const DataDocQueryExecutions: React.FunctionComponent = [queryExecutions] ); + const selectedExecution = queryExecutions[selectedExecutionIndex]; + + const isCellSaving = useSelector((state: IStoreState) => { + const savePromise = state.dataDoc.dataDocSavePromiseById[docId]; + if (!savePromise) { + return false; + } + return `cell-${cellId}` in savePromise.itemToSave; + }); + + const { warningState } = useStaleQueryWarning({ + selectedExecutionId: selectedExecution?.id ?? null, + snapshots: executionRunInputSnapshots ?? {}, + savedRunInput: savedRunInput ?? '', + liveRunInput: liveRunInput ?? '', + initialQuery, + isSaving: isCellSaving, + }); + const generateExecutionsPickerDOM = () => { const currentExecution = queryExecutions?.[selectedExecutionIndex]; @@ -128,6 +159,9 @@ export const DataDocQueryExecutions: React.FunctionComponent = return (
+ {warningState && ( + + )} = ); }; - const selectedExecution = queryExecutions[selectedExecutionIndex]; const queryExecutionDOM = selectedExecution && ( ; +} + +const TOOLTIP: Record, string> = { + edited: 'Changes are saved but not executed', + unsaved: 'Changes are unsaved and not executed', +}; + +export const StaleQueryWarning: React.FC = ({ variant }) => ( + + + Edited + +); diff --git a/querybook/webapp/components/QueryComposer/QueryComposer.tsx b/querybook/webapp/components/QueryComposer/QueryComposer.tsx index 2ba92bdc7..fca27e9b4 100644 --- a/querybook/webapp/components/QueryComposer/QueryComposer.tsx +++ b/querybook/webapp/components/QueryComposer/QueryComposer.tsx @@ -38,6 +38,7 @@ import { IQueryEngine } from 'const/queryEngine'; import { ISearchOptions, ISearchResult } from 'const/searchAndReplace'; import { SurveySurfaceType } from 'const/survey'; import { useDebounceState } from 'hooks/redux/useDebounceState'; +import { useExecutionSnapshots } from 'hooks/queryEditor/useExecutionSnapshots'; import { useSurveyTrigger } from 'hooks/ui/useSurveyTrigger'; import { useBrowserTitle } from 'hooks/useBrowserTitle'; import { useTrackView } from 'hooks/useTrackView'; @@ -424,6 +425,9 @@ const QueryComposer: React.FC = () => { ); const dispatch: Dispatch = useDispatch(); const { query, setQuery } = useQuery(dispatch, environmentId); + const reduxQuery = useSelector( + (state: IStoreState) => state.adhocQuery[environmentId]?.query ?? '' + ); const { engine, setEngineId, queryEngines, queryEngineById } = useEngine( dispatch, environmentId @@ -466,6 +470,16 @@ const QueryComposer: React.FC = () => { const [showPeerReviewModal, setShowPeerReviewModal] = useState(false); const hasPeerReviewFeature = engine?.feature_params?.peer_review; + const { + snapshots: executionRunInputById, + recordSnapshot: recordRunInputSnapshot, + } = useExecutionSnapshots(); + + const initialQueryRef = useRef(reduxQuery); + if (!initialQueryRef.current && reduxQuery) { + initialQueryRef.current = reduxQuery; + } + const runButtonRef = useRef(null); const clickOnRunButton = useCallback(() => { if (runButtonRef.current) { @@ -558,8 +572,9 @@ const QueryComposer: React.FC = () => { // Throttle to prevent double run await sleep(250); + const selectedQuery = getCurrentSelectedQuery(); const transformedQuery = await transformQuery( - getCurrentSelectedQuery(), + selectedQuery, engine.language, templatedVariables, engine, @@ -571,16 +586,17 @@ const QueryComposer: React.FC = () => { const queryId = await runQuery( transformedQuery, engine.id, - async (query, engineId) => { + async (q, engineId) => { const data = await dispatch( queryExecutionsAction.createQueryExecution( - query, + q, engineId, null, queryExecutionMetadata, peerReviewParams ) ); + recordRunInputSnapshot(data.id, query); return data.id; } ); @@ -602,12 +618,14 @@ const QueryComposer: React.FC = () => { getQueryExecutionMetadata, hasLintErrors, getCurrentSelectedQuery, + query, engine, templatedVariables, rowLimit, triggerSurvey, dispatch, setExecutionId, + recordRunInputSnapshot, ] ); @@ -750,6 +768,9 @@ const QueryComposer: React.FC = () => { } sampleRate={getSampleRate()} onUpdateQuery={updateAndRunQuery} + currentRunInput={query} + executionRunInputSnapshots={executionRunInputById} + initialQuery={initialQueryRef.current} />
diff --git a/querybook/webapp/components/QueryComposer/QueryComposerExecution.tsx b/querybook/webapp/components/QueryComposer/QueryComposerExecution.tsx index 241be0e39..b9349d798 100644 --- a/querybook/webapp/components/QueryComposer/QueryComposerExecution.tsx +++ b/querybook/webapp/components/QueryComposer/QueryComposerExecution.tsx @@ -8,10 +8,12 @@ import { queryStatusToStatusIcon, STATUS_TO_TEXT_MAPPING, } from 'const/queryStatus'; +import { useStaleQueryWarning } from 'hooks/queryEditor/useStaleQueryWarning'; import { IStoreState } from 'redux/store/types'; import { Level } from 'ui/Level/Level'; import { StatusIcon } from 'ui/StatusIcon/StatusIcon'; import { AccentText } from 'ui/StyledText/StyledText'; +import { StaleQueryWarning } from 'components/DataDocQueryExecutions/StaleQueryWarning'; interface IProps { id: number; @@ -19,6 +21,9 @@ interface IProps { hasSamplingTables?: boolean; sampleRate: number; onUpdateQuery?: (query: string, run?: boolean) => any; + currentRunInput?: string; + executionRunInputSnapshots?: Readonly>; + initialQuery?: string; } export const QueryComposerExecution: React.FunctionComponent = ({ @@ -27,11 +32,22 @@ export const QueryComposerExecution: React.FunctionComponent = ({ hasSamplingTables, sampleRate, onUpdateQuery, + currentRunInput, + executionRunInputSnapshots, + initialQuery, }) => { const execution = useSelector( (state: IStoreState) => state.queryExecutions.queryExecutionById[id] ); + const { warningState } = useStaleQueryWarning({ + selectedExecutionId: id, + snapshots: executionRunInputSnapshots ?? {}, + savedRunInput: currentRunInput ?? '', + liveRunInput: currentRunInput ?? '', + initialQuery, + }); + if (!execution) { return null; } @@ -49,6 +65,9 @@ export const QueryComposerExecution: React.FunctionComponent = ({
+ {warningState && ( + + )} {statusIcon} diff --git a/querybook/webapp/hooks/queryEditor/useExecutionSnapshots.ts b/querybook/webapp/hooks/queryEditor/useExecutionSnapshots.ts new file mode 100644 index 000000000..9bcc9e7bd --- /dev/null +++ b/querybook/webapp/hooks/queryEditor/useExecutionSnapshots.ts @@ -0,0 +1,24 @@ +import { useCallback, useState } from 'react'; + +export function addRunInputSnapshot( + prev: Record, + executionId: number, + runInput: string +): Record { + return { ...prev, [executionId]: runInput }; +} + +export function useExecutionSnapshots() { + const [snapshots, setSnapshots] = useState>({}); + + const recordSnapshot = useCallback( + (executionId: number, runInput: string) => { + setSnapshots((prev) => + addRunInputSnapshot(prev, executionId, runInput) + ); + }, + [] + ); + + return { snapshots, recordSnapshot }; +} diff --git a/querybook/webapp/hooks/queryEditor/useStaleQueryWarning.ts b/querybook/webapp/hooks/queryEditor/useStaleQueryWarning.ts new file mode 100644 index 000000000..4e17c31d7 --- /dev/null +++ b/querybook/webapp/hooks/queryEditor/useStaleQueryWarning.ts @@ -0,0 +1,113 @@ +import { useDebounce } from 'hooks/useDebounce'; + +const DEFAULT_DEBOUNCE_MS = 300; + +export type StaleWarningState = 'edited' | 'unsaved' | null; + +export function shouldComputeStaleWarning( + selectedExecutionId: number | null | undefined, + snapshots: Readonly>, + currentInput: string, + initialQuery?: string +): boolean { + if (selectedExecutionId == null) { + return false; + } + const snapshot = snapshots[selectedExecutionId] ?? initialQuery; + if (snapshot === undefined) { + return false; + } + return currentInput !== snapshot; +} + +export function computeStaleWarningState(options: { + selectedExecutionId: number | null | undefined; + snapshots: Readonly>; + savedInput: string; + liveInput: string; + initialQuery?: string; + isSaving?: boolean; +}): StaleWarningState { + const { + selectedExecutionId, + snapshots, + savedInput, + liveInput, + initialQuery, + isSaving = false, + } = options; + + const isLiveStale = shouldComputeStaleWarning( + selectedExecutionId, + snapshots, + liveInput, + initialQuery + ); + + if (!isLiveStale) { + return null; + } + + const isSavedStale = shouldComputeStaleWarning( + selectedExecutionId, + snapshots, + savedInput, + initialQuery + ); + + if (isSavedStale && !isSaving) { + return 'edited'; + } + + return 'unsaved'; +} + +export function useStaleQueryWarning(options: { + selectedExecutionId: number | null | undefined; + snapshots: Readonly>; + savedRunInput: string; + liveRunInput: string; + initialQuery?: string; + debounceMs?: number; + isSaving?: boolean; +}): { warningState: StaleWarningState } { + const { + selectedExecutionId, + snapshots, + savedRunInput, + liveRunInput, + initialQuery, + debounceMs = DEFAULT_DEBOUNCE_MS, + isSaving = false, + } = options; + + const debouncedLiveInput = useDebounce(liveRunInput, debounceMs); + + // Debounced check stays stable during typing to prevent flickering. + // Real-time check instantly suppresses false positives on initial page load + // (when liveRunInput briefly differs from the snapshot before the editor populates). + const debouncedState = computeStaleWarningState({ + selectedExecutionId, + snapshots, + savedInput: savedRunInput, + liveInput: debouncedLiveInput, + initialQuery, + isSaving, + }); + const realtimeState = computeStaleWarningState({ + selectedExecutionId, + snapshots, + savedInput: savedRunInput, + liveInput: liveRunInput, + initialQuery, + isSaving, + }); + + // Both must agree on a non-null state; if either is null, suppress. + const warningState = + debouncedState !== null && realtimeState !== null + ? debouncedState + : null; + + return { warningState }; +}