Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 19 additions & 8 deletions pkgs/website/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -152,15 +152,22 @@ export default defineConfig({
starlightLlmsTxt({
exclude: [
'index',
'**/index',
// Navigation-only index files (pure CardGrid hubs with no unique content)
'build/index',
'concepts/index',
'deploy/index',
'reference/index',
'tutorials/index',
'comparisons/index',
// Tutorials (lengthy, patterns covered elsewhere)
'tutorials/ai-web-scraper/*',
'concepts/naming-steps',
'deploy/tune-flow-config',
'get-started/faq',
// News/blog and non-technical content
'news/**',
'hire/**',
'build/configuring-retries',
'build/delaying-steps',
'author/**',
// Note: The following index files ARE included because they have valuable content:
// - build/starting-flows/index (comparison guide for starting flows)
// - reference/configuration/index (config philosophy and structure)
],
promote: [
'get-started/installation',
Expand Down Expand Up @@ -225,8 +232,12 @@ export default defineConfig({
link: '/build/organize-flow-code/',
},
{
label: 'Configuring retries',
link: '/build/configuring-retries/',
label: 'Retrying steps',
link: '/build/retrying-steps/',
},
{
label: 'Validation steps',
link: '/build/validation-steps/',
Comment on lines 234 to +240

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add redirect from old retries path

The page rename to /build/retrying-steps/ updates the navigation, but there is no corresponding redirect entry for the previous /build/configuring-retries/ URL. Any external links, bookmarks, or cached search results for the old path will now return 404s. The documentation guidelines in CLAUDE.md require adding redirects when renaming pages. Please add a mapping in redirects.config.mjs to forward /build/configuring-retries/ to /build/retrying-steps/ so historical links continue to work.

Useful? React with 👍 / 👎.

},
{
label: 'Delaying steps',
Expand Down
6 changes: 3 additions & 3 deletions pkgs/website/src/content/docs/build/delaying-steps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { Aside, CardGrid, LinkCard } from "@astrojs/starlight/components";
Use [`startDelay`](/reference/configuration/step-execution/#startdelay) to schedule steps for execution after a specified time period. Delays are relative to when the step's dependencies complete.

<Aside type="tip">
For retry configuration patterns, see [Configuring Retries](/build/configuring-retries/).
For retry configuration patterns, see [Retrying Steps](/build/retrying-steps/).
</Aside>

For detailed information about each configuration option, see the [Step Execution Options](/reference/configuration/step-execution/) reference.
Expand Down Expand Up @@ -82,8 +82,8 @@ new Flow({
description="Complete reference for all configuration options and their defaults"
/>
<LinkCard
title="Configuring Retries"
href="/build/configuring-retries/"
title="Retrying Steps"
href="/build/retrying-steps/"
description="Configure retry policies for different reliability requirements"
/>
<LinkCard
Expand Down
4 changes: 2 additions & 2 deletions pkgs/website/src/content/docs/build/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Now that you've created your first flow, learn how to structure your code, integ
description="Structure your flows and tasks for maintainability and reusability"
/>
<LinkCard
title="Configuring retries"
href="/build/configuring-retries/"
title="Retrying steps"
href="/build/retrying-steps/"
description="Configure retry policies for different step reliability requirements"
/>
<LinkCard
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Configuring Retries
title: Retrying Steps
description: Configure retry policies for different step reliability requirements.
sidebar:
order: 25
Expand All @@ -15,6 +15,56 @@ For scheduling delays between steps, see [Delaying Steps](/build/delaying-steps/

For detailed information about each configuration option, see the [Step Execution Options](/reference/configuration/step-execution/) reference.

## Understanding Failure Types

Not all failures should retry. Understanding the difference helps you configure appropriate retry policies.

### Transient Failures

Temporary problems that might succeed on retry:
- Network timeouts
- Rate limiting (429 responses)
- Temporary service unavailability (503 responses)
- Database connection issues

Configure with retries:
```typescript
.step({
slug: 'fetchExternalData',
maxAttempts: 5, // Retry transient failures
baseDelay: 2,
}, async (input) => await fetchFromAPI(input.run.url))
```

### Permanent Failures

Problems that will never succeed on retry:
- Invalid input format (malformed email, negative numbers)
- Missing required fields
- Business rule violations
- Schema validation errors

Configure without retries:
```typescript
.step({
slug: 'validInput',
maxAttempts: 1, // No retries for validation
}, (input) => {
if (!input.run.email) throw new Error('Email required');
return input.run;
})
```

<Aside type="note">
`maxAttempts: 1` means "run once, do not retry". If the step fails, it fails immediately without retry attempts.
</Aside>

<Aside type="caution" title="Current Limitation">
pgflow does not distinguish between transient and permanent failures automatically. All exceptions trigger retry logic based on `maxAttempts`. Use `maxAttempts: 1` for steps that perform validation or other operations that should fail fast.
</Aside>

For detailed guidance on validation patterns, see [Validation Steps](/build/validation-steps/).

## Guiding Principle

**Set conservative flow-level defaults, override per-step as needed.**
Expand Down Expand Up @@ -56,6 +106,11 @@ new Flow({
## Learn More

<CardGrid>
<LinkCard
title="Validation Steps"
href="/build/validation-steps/"
description="Use explicit validation steps to fail fast on invalid input"
/>
<LinkCard
title="Step Execution Options"
href="/reference/configuration/step-execution/"
Expand Down
87 changes: 87 additions & 0 deletions pkgs/website/src/content/docs/build/validation-steps.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
title: Validation Steps
description: Use explicit validation steps to fail fast on invalid input
sidebar:
order: 30
---

import { Aside } from '@astrojs/starlight/components';

Validation steps catch invalid input early, before expensive operations run. Structure them to fail fast with no retries.

## Why Explicit Validation Steps?

pgflow retries all exceptions based on `maxAttempts`. Without explicit validation, input errors waste retry attempts:

```typescript
// Without validation - wastes retry attempts on invalid input
new Flow<{ email: string }>({ slug: 'sendEmail', maxAttempts: 5 })
.step(
{ slug: 'send' },
async (input) => {
if (!input.run.email.includes('@')) {
throw new Error('Invalid email'); // Retries 5 times!
}
return await sendEmail(input.run.email);
}
)
```

With explicit validation, failures stop immediately:

```typescript
// With validation - fails immediately on invalid input
new Flow<{ email: string }>({ slug: 'sendEmail' })
.step(
{ slug: 'validInput', maxAttempts: 1 },
(input) => {
if (!input.run.email.includes('@')) {
throw new Error('Invalid email');
}
return input.run;
}
)
.step(
{ slug: 'send', dependsOn: ['validInput'], maxAttempts: 5 },
async (input) => await sendEmail(input.validInput.email)
)
```

## Keep Validation Synchronous

Validation steps should be fast, synchronous functions that check input format and structure. Avoid async operations like database queries or API calls - those belong in separate steps with appropriate retry configuration.

```typescript
// Good: Fast, synchronous validation
.step(
{ slug: 'validOrder', maxAttempts: 1 },
(input) => {
const { amount, items } = input.run;

if (amount <= 0) throw new Error('amount must be positive');
if (!items?.length) throw new Error('items cannot be empty');

return input.run;
}
)

// Bad: Async checks in validation
.step(
{ slug: 'validCustomer', maxAttempts: 1 },
async (input) => {
// Database lookups belong in separate steps with retries
const exists = await checkCustomerExists(input.run.customerId);
if (!exists) throw new Error('Customer not found');
return input.run;
}
)
```

<Aside type="tip">
Validation steps check **format and structure** (email format, positive numbers, array length). Existence checks and external validations belong in regular steps with retry configuration.
</Aside>

## See Also

- [Retrying Steps](/build/retrying-steps/) - Understanding failure types and retry policies
- [Context Object](/concepts/context-object/) - Accessing validated input in dependent steps
2 changes: 1 addition & 1 deletion pkgs/website/src/content/docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ npx pgflow@latest install
</Card>

<Card title="🔄 Automatic retries">
Built-in retry logic with exponential backoff for flaky AI APIs. When OpenAI times out or rate-limits, only that step retries - your workflow continues. Configure max attempts and delays per step, no retry code needed. [Learn more →](/build/configuring-retries/)
Built-in retry logic with exponential backoff for flaky AI APIs. When OpenAI times out or rate-limits, only that step retries - your workflow continues. Configure max attempts and delays per step, no retry code needed. [Learn more →](/build/retrying-steps/)
</Card>

<Card title="⚡ Parallel array processing">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,9 @@ new Flow({ slug: 'my_flow' })

<CardGrid>
<LinkCard
title="Configuring Retries"
title="Retrying Steps"
description="Configure retry policies for different step reliability requirements."
href="/build/configuring-retries/"
href="/build/retrying-steps/"
/>
<LinkCard
title="Delaying Steps"
Expand Down