Merge branch 'master' into feature/add-cubejs-tasks

changes/pr5016.yml

@@ -0,0 +1,5 @@
+task:
+  - "Allow all Postgres task parameters to be configured at runtime as well as init - [#4377](https://github.com/PrefectHQ/prefect/pull/5016)"
+
+contributor:
+  - "[Adam Brusselback](https://github.com/Tostino)"

changes/pr5173.yaml

@@ -0,0 +1,5 @@
+enhancement:
+  - "Add authentication as parameter for SnowflakeQuery and SnowflakeQueriesFromFile tasks - [#5173](https://github.com/PrefectHQ/prefect/pull/5173)"
+
+contributor:
+  - "[Gaylord Cherencey](https://github.com/gcherencey)"

changes/pr5265.yaml

@@ -0,0 +1,5 @@
+task:
+  - "Add 'Firebolt' to Prefect Task Library  - [#5265](https://github.com/PrefectHQ/prefect/pull/5265)"
+
+contributor:
+  - "[Raghav Sharma](https://github.com/raghavSharmaSigmoid)"

changes/pr5273.yaml

@@ -0,0 +1,5 @@
+enhancement:
+  - "Allow dbt Cloud task to use custom domain for Enterprise customers- [#5273](https://github.com/PrefectHQ/prefect/pull/5273)"
+
+contributor:
+  - "[Henning Holgersen](https://github.com/radbrt)"

changes/pr5276.yaml

@@ -0,0 +1,5 @@
+feature:
+  - "Add Mixpanel Tasks - [#5276](https://github.com/PrefectHQ/prefect/pull/5276)"
+
+contributor:
+  - "[Alessandro Lollo](https://github.com/AlessandroLollo)"

changes/pr5278.yaml

@@ -0,0 +1,5 @@
+feature:
+  - "Add Zendesk Tickets Incremental Export Task - [#5278](https://github.com/PrefectHQ/prefect/pull/5278)"
+
+contributor:
+  - "[Alessandro Lollo](https://github.com/AlessandroLollo)"

changes/pr5292.yaml

@@ -0,0 +1,5 @@
+task:
+  - "Robust communication with server instances - [#5292](https://github.com/PrefectHQ/prefect/pull/5292)"
+
+contributor:
+  - "[Ahmed Ezzat](https://github.com/bitthebyte)"

changes/pr_5260.yaml

@@ -0,0 +1,6 @@
+
+fix:
+  - "AirbyteConnectionTask.run() requires parameters that should be optional - [#5260](https://github.com/PrefectHQ/prefect/pull/5260)"
+
+contributor:
+  - "[Michał Zawadzki](https://github.com/Trymzet)"

docs/core/concepts/schedules.md

@@ -37,11 +37,13 @@ Prefect `Schedules` have three components:
 
 These three components allow users to combine simple functions into complex behavior.
 
+Note that many examples here use the [Pendulum](https://pendulum.eustace.io/) Python package for easy datetime manipulation. Pendulum isn’t required, but it’s employed in many schedule use cases, such as start and end dates.
+
 ### Clocks
 
 #### Interval clocks
 
-The most basic Prefect clock is the `IntervalClock`. It takes an `interval` argument and emits events on a regular basis. An optional `start_date` can be provided, in which case the intervals will be relative to that date; an `end_date` can be provided as well.
+The most basic Prefect clock is the [`IntervalClock`](/api/latest/schedules/clocks.html#intervalclock). It takes an `interval` argument and emits events on a regular basis. An optional `start_date` can be provided, in which case the intervals will be relative to that date; an `end_date` can be provided as well.
 
 Prefect does not support sub-minute schedules.
 
@@ -56,9 +58,11 @@ schedule.next(5)
 ```
 
 ::: tip Time Zones
-Want to pin your schedule to a time zone? Specify a `start_date` corresponding to that time zone for your clock e.g.:
+Want to pin your schedule to a time zone? Specify a `start_date` corresponding to that time zone for your clock:
 
 ```python
+import pendulum
+
 schedules.clocks.IntervalClock(
     start_date=pendulum.datetime(2019, 1, 1, tz="America/New_York"),
     interval=timedelta(days=1)
@@ -75,7 +79,7 @@ Note that this behavior is different from the `CronClock`.
 
 #### Cron clocks
 
-Clocks can also be generated from cron strings with Prefect's `CronClock`.
+Clocks can also be generated from cron strings with the Prefect [`CronClock`](/api/latest/schedules/clocks.html#cronclock).
 
 ```python
 from datetime import timedelta
@@ -88,14 +92,14 @@ schedule.next(5)
 ```
 
 ::: warning Daylight Saving Time
-If the `CronClock's` start time is provided with a DST-observing timezone, then the schedule will adjust itself. Cron's rules for DST are based on clock times, not intervals. This means that an hourly cron schedule will fire on every new clock hour, not every elapsed hour; for example, when clocks are set back this will result in a two-hour pause as the schedule will fire _the first time_ 1am is reached and _the first time_ 2am is reached, 120 minutes later. Longer schedules, such as one that fires at 9am every morning, will automatically adjust for DST.
+If the `CronClock` start time is provided with a DST-observing timezone, then the schedule will adjust itself. Cron's rules for DST are based on clock times, not intervals. This means that an hourly cron schedule will fire on every new clock hour, not every elapsed hour; for example, when clocks are set back this will result in a two-hour pause as the schedule will fire _the first time_ 1am is reached and _the first time_ 2am is reached, 120 minutes later. Longer schedules, such as one that fires at 9am every morning, will automatically adjust for DST.
 
 Note that this behavior is different from the `IntervalClock`.
 :::
 
 #### Date clocks
 
-For more ad-hoc schedules, Prefect provides a `DatesClock` that only fires on specific, user-provided dates.
+For more ad-hoc schedules, Prefect provides a [`DatesClock`](/api/latest/schedules/clocks.html#datesclock) that only fires on specific, user-provided dates.
 
 ```python
 from datetime import timedelta
@@ -109,7 +113,27 @@ schedule = Schedule(
 schedule.next(2)
 ```
 
-#### Varying Parameter Values <Badge text="0.9.2+"/>
+#### Recurrence Rule Clocks
+
+The Prefect [`RRuleClock`](/api/latest/schedules/clocks.html#rruleclock) supports [iCal recurrence rules](https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html) (RRules), which provide convenient syntax for creating repetitive schedules. Schedules can repeat on a frequency from yearly down to every minute. 
+
+`RRuleClock` uses the [dateutil rrule module](https://dateutil.readthedocs.io/en/stable/rrule.html) to specify iCal recurrence rules.
+
+For example, `RRuleClock` can specify a schedule recurring every day for a week:
+
+```python
+from dateutil.rrule import rrule, DAILY
+import pendulum
+from prefect.schedules import Schedule
+from prefect.schedules.clocks import RRuleClock
+
+start_date = pendulum.now().add(days=1)
+r_rule = rrule(freq=DAILY, count=7)
+
+schedule = Schedule(clocks=[RRuleClock(r_rule, start_date=start_date)])
+```
+
+#### Varying Parameter Values
 
 All clocks support an optional `parameter_defaults` argument that allows users to specify varying `Parameter` values for each flow run generated from this clock.  For example, suppose we have the following flow that logs the value of the `Parameter` that is passed to it:
 

docs/core/getting_started/install.md

@@ -33,10 +33,6 @@ pipenv install --pre prefect
 
 ::::
 
-:::warning Python 3.9
-Prefect support for Python 3.9 is experimental and extras are not expected to work yet as we wait for required packages to be updated.
-:::
-
 ## Installing optional dependencies
 
 Prefect ships with a number of optional dependencies, which can be installed using "extras" syntax:

docs/outline.toml

@@ -453,6 +453,11 @@ classes = [
         "Zip",
         ]
 
+[pages.tasks.firebolt]
+title = "Firebolt Tasks"
+module = "prefect.tasks.firebolt"
+classes = ["FireboltQuery"]
+
 [pages.tasks.gcp]
 title = "Google Cloud Tasks"
 module = "prefect.tasks.gcp"
@@ -566,6 +571,11 @@ title = "SodaSpark Tasks"
 module = "prefect.tasks.sodaspark"
 classes = ["SodaSparkScan"]
 
+[pages.tasks.mixpanel]
+title = "Mixpanel Tasks"
+module = "prefect.tasks.mixpanel"
+classes = ["MixpanelExportTask"]
+
 [pages.tasks.databricks]
 title = "Databricks Tasks"
 module = "prefect.tasks.databricks"
@@ -769,4 +779,9 @@ verified_task = true
 [pages.tasks.cubejs]
 title = "Cube.js Tasks"
 module = "prefect.tasks.cubejs"
-classes = ["CubeJSQueryTask"]
+classes = ["CubeJSQueryTask"]
+
+[pages.tasks.zendesk]
+title = "Zendesk Tasks"
+module = "prefect.tasks.zendesk"
+classes = ["ZendeskTicketsIncrementalExportTask"]

requirements.txt

@@ -19,7 +19,7 @@ pyyaml >= 3.13
 python-box >= 5.1.0
 python-slugify >= 1.2.6
 pytz >= 2018.7
-requests >= 2.25, < 2.27
+requests >= 2.25, < 2.28
 tabulate >= 0.8.0
 toml >= 0.9.4
 urllib3 >= 1.26.0, < 1.27

setup.py

@@ -56,6 +56,7 @@ def run(self):
     "dask_cloudprovider": ["dask_cloudprovider[aws] >= 0.2.0"],
     "dev": dev_requires + test_requires,
     "dropbox": ["dropbox ~= 9.0"],
+    "firebolt": ["firebolt-sdk >= 0.2.1"],
     "ge": ["great_expectations >= 0.13.8, < 0.14", "mistune < 2"],
     "gcp": [
         "google-cloud-bigquery >= 1.6.0, < 3.0",

src/prefect/client/client.py

@@ -727,7 +727,7 @@ def _request(
             headers.update(self._attached_headers)
 
         session = requests.Session()
-        retry_total = 6 if prefect.config.backend == "cloud" else 1
+        retry_total = 6
         retries = requests.packages.urllib3.util.retry.Retry(
             total=retry_total,
             backoff_factor=1,

src/prefect/schedules/clocks.py

@@ -390,8 +390,12 @@ class RRuleClock(Clock):
     Clock that supports iCal style RRule (recurrence rules).
 
     See below links for helpful info:
-        - https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html
-        - https://dateutil.readthedocs.io/en/stable/rrule.html
+
+    IETF iCalendar specification RFC5545:
+    https://icalendar.org/iCalendar-RFC-5545/3-8-5-3-recurrence-rule.html
+
+    dateutil rrule module, an implementation of the iCal specification recurrence rules:
+    https://dateutil.readthedocs.io/en/stable/rrule.html
 
     Args:
         - rrule_obj (rrulebase): an rrule or rruleset object

src/prefect/tasks/airbyte/airbyte.py

@@ -95,6 +95,8 @@ def _get_connection_status(self, session, airbyte_base_url, connection_id):
             )
             self.logger.info(response.json())
 
+            response.raise_for_status()
+
             # check whether a schedule exists ...
             schedule = response.json()["schedule"]
             if schedule:
@@ -189,10 +191,10 @@ def _get_job_status(self, session, airbyte_base_url, job_id):
     )
     def run(
         self,
-        airbyte_server_host: str,
-        airbyte_server_port: int,
-        airbyte_api_version: str,
-        connection_id: str,
+        airbyte_server_host: str = None,
+        airbyte_server_port: int = None,
+        airbyte_api_version: str = None,
+        connection_id: str = None,
         poll_interval_s: int = 15,
     ) -> dict:
         """