Adding examples for date filters #197
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -59,3 +59,206 @@ To learn more about using filters, check out our docs on limiting data using fil | |
| | is after | Only pulls in rows where the dates for the field selected are strictly after the date you entered. | | ||
| | is on or after | Only pulls in rows where the dates for the field selected are on or after the date you entered. | | ||
| | is between | Only pulls in rows where the dates for the field selected are on or between the dates you entered: "between 2001-12-23 and 2003-01-02". | | ||
|
|
||
| <Accordion title="Examples of date or timestamp filters"> | ||
|
|
||
| ## Date/Timestamp Filter Reference Guide | ||
|
|
||
| The below examples show possible date/timestamp filter combinations and their corresponding SQL outputs. All examples use BigQuery syntax and assume: | ||
| - Current timestamp: `2025-10-24 15:30:00` | ||
| - Example field: `orders.created_at` | ||
| - Timezone: UTC | ||
| - Week starts on Monday | ||
|
|
||
| ## Timestamp Data Type Filters | ||
|
|
||
| ### Current Period Filters | ||
|
|
||
| #### Current (In The Current) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Current minute | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at < TIMESTAMP('2025-10-24 15:31:00')` | | ||
| | Current hour | `orders.created_at >= TIMESTAMP('2025-10-24 15:00:00') AND orders.created_at < TIMESTAMP('2025-10-24 16:00:00')` | | ||
| | Current day | `orders.created_at >= TIMESTAMP('2025-10-24 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-25 00:00:00')` | | ||
| | Current week | `orders.created_at >= TIMESTAMP('2025-10-21 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-28 00:00:00')` | | ||
| | Current month | `orders.created_at >= TIMESTAMP('2025-10-01 00:00:00') AND orders.created_at < TIMESTAMP('2025-11-01 00:00:00')` | | ||
| | Current quarter | `orders.created_at >= TIMESTAMP('2025-10-01 00:00:00') AND orders.created_at < TIMESTAMP('2026-01-01 00:00:00')` | | ||
| | Current year | `orders.created_at >= TIMESTAMP('2025-01-01 00:00:00') AND orders.created_at < TIMESTAMP('2026-01-01 00:00:00')` | | ||
|
|
||
| ### Past Period Filters | ||
|
|
||
| #### Last N Periods (in the last) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Last 1 minute | `orders.created_at >= TIMESTAMP('2025-10-24 15:29:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 hour | `orders.created_at >= TIMESTAMP('2025-10-24 14:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 day | `orders.created_at >= TIMESTAMP('2025-10-23 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 week | `orders.created_at >= TIMESTAMP('2025-10-17 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 month | `orders.created_at >= TIMESTAMP('2025-09-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 quarter | `orders.created_at >= TIMESTAMP('2025-07-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 year | `orders.created_at >= TIMESTAMP('2024-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:30:00')` | | ||
|
|
||
| #### Last N Completed Periods (in the last, Completed) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Last 1 completed minute | `orders.created_at >= TIMESTAMP('2025-10-24 15:29:00') AND orders.created_at < TIMESTAMP('2025-10-24 15:30:00')` | | ||
| | Last 1 completed hour | `orders.created_at >= TIMESTAMP('2025-10-24 14:00:00') AND orders.created_at < TIMESTAMP('2025-10-24 15:00:00')` | | ||
| | Last 1 completed day | `orders.created_at >= TIMESTAMP('2025-10-23 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-24 00:00:00')` | | ||
| | Last 1 completed week | `orders.created_at >= TIMESTAMP('2025-10-13 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-20 00:00:00')` | | ||
| | Last 1 completed month | `orders.created_at >= TIMESTAMP('2025-09-01 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-01 00:00:00')` | | ||
| | Last 1 completed quarter | `orders.created_at >= TIMESTAMP('2025-07-01 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-01 00:00:00')` | | ||
| | Last 1 completed year | `orders.created_at >= TIMESTAMP('2024-01-01 00:00:00') AND orders.created_at < TIMESTAMP('2025-01-01 00:00:00')` | | ||
|
|
||
| ### Future Period Filters | ||
|
|
||
| #### Next N Periods (In The Next) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Next 1 minute | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 15:31:00')` | | ||
| | Next 1 hour | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-24 16:30:00')` | | ||
| | Next 1 day | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-25 15:30:00')` | | ||
| | Next 1 week | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-10-31 15:30:00')` | | ||
| | Next 1 month | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2025-11-24 15:30:00')` | | ||
| | Next 1 quarter | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2026-01-24 15:30:00')` | | ||
| | Next 1 year | `orders.created_at >= TIMESTAMP('2025-10-24 15:30:00') AND orders.created_at <= TIMESTAMP('2026-10-24 15:30:00')` | | ||
|
|
||
| #### Next N Completed Periods (In The Next, Completed) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Next 1 completed minute | `orders.created_at >= TIMESTAMP('2025-10-24 15:31:00') AND orders.created_at < TIMESTAMP('2025-10-24 15:32:00')` | | ||
| | Next 1 completed hour | `orders.created_at >= TIMESTAMP('2025-10-24 16:00:00') AND orders.created_at < TIMESTAMP('2025-10-24 17:00:00')` | | ||
| | Next 1 completed day | `orders.created_at >= TIMESTAMP('2025-10-25 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-26 00:00:00')` | | ||
| | Next 1 completed week | `orders.created_at >= TIMESTAMP('2025-10-27 00:00:00') AND orders.created_at < TIMESTAMP('2025-11-03 00:00:00')` | | ||
| | Next 1 completed month | `orders.created_at >= TIMESTAMP('2025-11-01 00:00:00') AND orders.created_at < TIMESTAMP('2025-12-01 00:00:00')` | | ||
| | Next 1 completed quarter | `orders.created_at >= TIMESTAMP('2026-01-01 00:00:00') AND orders.created_at < TIMESTAMP('2026-04-01 00:00:00')` | | ||
| | Next 1 completed year | `orders.created_at >= TIMESTAMP('2026-01-01 00:00:00') AND orders.created_at < TIMESTAMP('2027-01-01 00:00:00')` | | ||
|
|
||
| #### Within Custom Range | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Between 2 dates | `orders.created_at >= TIMESTAMP('2025-10-01 00:00:00') AND orders.created_at <= TIMESTAMP('2025-10-31 23:59:59')` | | ||
| | On exact date | `orders.created_at >= TIMESTAMP('2025-10-24 00:00:00') AND orders.created_at < TIMESTAMP('2025-10-25 00:00:00')` | | ||
|
|
||
| ### Notes | ||
|
There was a problem hiding this comment. might be a good opportunity to mention that timestamps are always set to UTC? we have this issue that could be closed by adding a few notes here lightdash/lightdash#7926 |
||
|
|
||
| 1. **Completed periods** always: | ||
| - Start at the beginning of a period (00:00:00) | ||
| - End at the beginning of the next period | ||
| - Don't include partial periods | ||
|
|
||
| 2. **Rolling periods** (non-completed): | ||
| - Use the current time as the reference point | ||
| - Look backward/forward the specified amount | ||
| - Include partial periods | ||
|
|
||
| 3. **Current periods**: | ||
| - Always start at the beginning of the current period | ||
| - End at the beginning of the next period | ||
| - Example: Current month starts at 1st of the month | ||
|
|
||
| 4. **Week handling**: | ||
| - Weeks start on Monday by default | ||
| - Can be configured to start on Sunday | ||
|
There was a problem hiding this comment. technically possible to start on any day, but maybe not worth saying since who would do that...? |
||
| - Week boundaries are always at midnight (00:00:00) | ||
|
|
||
|
|
||
| ## Date Data Type Filters | ||
|
|
||
| ### Current Period Filters | ||
|
|
||
| #### Current (In The Current) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Current day | `orders.created_date = DATE('2025-10-24')` | | ||
| | Current week | `orders.created_date >= DATE('2025-10-21') AND orders.created_date < DATE('2025-10-28')` | | ||
| | Current month | `orders.created_date >= DATE('2025-10-01') AND orders.created_date < DATE('2025-11-01')` | | ||
| | Current quarter | `orders.created_date >= DATE('2025-10-01') AND orders.created_date < DATE('2026-01-01')` | | ||
| | Current year | `orders.created_date >= DATE('2025-01-01') AND orders.created_date < DATE('2026-01-01')` | | ||
|
|
||
| ### Past Period Filters | ||
|
|
||
| #### Last N Periods (in the last) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Last 1 day | `orders.created_date >= DATE('2025-10-23') AND orders.created_date <= DATE('2025-10-24')` | | ||
| | Last 7 days | `orders.created_date >= DATE('2025-10-17') AND orders.created_date <= DATE('2025-10-24')` | | ||
| | Last 30 days | `orders.created_date >= DATE('2025-09-24') AND orders.created_date <= DATE('2025-10-24')` | | ||
| | Last 90 days | `orders.created_date >= DATE('2025-07-26') AND orders.created_date <= DATE('2025-10-24')` | | ||
| | Last 365 days | `orders.created_date >= DATE('2024-10-24') AND orders.created_date <= DATE('2025-10-24')` | | ||
|
|
||
| #### Last N Completed Periods (in the last, Completed) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Last 1 completed day | `orders.created_date = DATE('2025-10-23')` | | ||
| | Last 1 completed week | `orders.created_date >= DATE('2025-10-13') AND orders.created_date < DATE('2025-10-20')` | | ||
| | Last 1 completed month | `orders.created_date >= DATE('2025-09-01') AND orders.created_date < DATE('2025-10-01')` | | ||
| | Last 1 completed quarter | `orders.created_date >= DATE('2025-07-01') AND orders.created_date < DATE('2025-10-01')` | | ||
| | Last 1 completed year | `orders.created_date >= DATE('2024-01-01') AND orders.created_date < DATE('2025-01-01')` | | ||
|
|
||
| ### Future Period Filters | ||
|
|
||
| #### Next N Periods (In The Next) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Next 1 day | `orders.created_date >= DATE('2025-10-24') AND orders.created_date <= DATE('2025-10-25')` | | ||
| | Next 7 days | `orders.created_date >= DATE('2025-10-24') AND orders.created_date <= DATE('2025-10-31')` | | ||
| | Next 30 days | `orders.created_date >= DATE('2025-10-24') AND orders.created_date <= DATE('2025-11-23')` | | ||
| | Next 90 days | `orders.created_date >= DATE('2025-10-24') AND orders.created_date <= DATE('2026-01-22')` | | ||
| | Next 365 days | `orders.created_date >= DATE('2025-10-24') AND orders.created_date <= DATE('2026-10-24')` | | ||
|
|
||
| #### Next N Completed Periods (In The Next, Completed) | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Next 1 completed day | `orders.created_date = DATE('2025-10-25')` | | ||
| | Next 1 completed week | `orders.created_date >= DATE('2025-10-27') AND orders.created_date < DATE('2025-11-03')` | | ||
| | Next 1 completed month | `orders.created_date >= DATE('2025-11-01') AND orders.created_date < DATE('2025-12-01')` | | ||
| | Next 1 completed quarter | `orders.created_date >= DATE('2026-01-01') AND orders.created_date < DATE('2026-04-01')` | | ||
| | Next 1 completed year | `orders.created_date >= DATE('2026-01-01') AND orders.created_date < DATE('2027-01-01')` | | ||
|
|
||
| #### Within Custom Range | ||
|
|
||
| | Filter | SQL Output | | ||
| |--------|------------| | ||
| | Between 2 dates | `orders.created_date >= DATE('2025-10-01') AND orders.created_date <= DATE('2025-10-31')` | | ||
| | On exact date | `orders.created_date = DATE('2025-10-24')` | | ||
|
|
||
| ### Notes | ||
|
|
||
| 1. **Key differences from timestamp filters**: | ||
|
There was a problem hiding this comment. maybe a bullet to mention that no timezone conversion is done on dates, so it's a good solution if people are having trouble with timezones (assuming they only need reporting at date granularity) |
||
| - No time components in any filters | ||
| - Single day comparisons use equality (`=`) instead of ranges | ||
| - `DATE()` function used instead of `TIMESTAMP()` | ||
|
|
||
| 2. **Completed periods** always: | ||
| - Start at the beginning of a period | ||
| - End at the beginning of the next period | ||
| - Don't include partial periods | ||
| - For single days, use equality instead of ranges | ||
|
|
||
| 3. **Rolling periods**: | ||
| - Use the current date as the reference point | ||
| - Count in full days (N days forward/backward) | ||
| - Include the current date in the range | ||
|
|
||
| 4. **Current periods**: | ||
| - For single day: use equality | ||
| - For longer periods: use standard ranges | ||
| - Example: Current month is all days from 1st to last day | ||
|
|
||
| 5. **Week handling**: | ||
| - Weeks start on Monday by default | ||
| - Can be configured to start on Sunday | ||
| - Full days only, no time components | ||
|
|
||
| </Accordion> | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think remove the accordion and make it a new section with H2 header