Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/styles/config/vocabularies/Aurora/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ autocommit
hostname
[Kk]eepalive
[Bb]ackoff
[Xx]ray
[Cc]loudwatch
5 changes: 2 additions & 3 deletions .github/workflows/integration_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ jobs:
matrix:
python-version: [ "3.8", "3.11" ]
engine_version: ["lts", "latest"]
environment: ["mysql", "pg"]

steps:
- name: 'Clone repository'
Expand Down Expand Up @@ -59,15 +60,13 @@ jobs:

- name: 'Run Integration Tests'
run: |
./gradlew --no-parallel --no-daemon test-python-${{ matrix.python-version }} --info
./gradlew --no-parallel --no-daemon test-python-${{ matrix.python-version }}-${{ matrix.environment }} --info
env:
RDS_CLUSTER_DOMAIN: ${{ secrets.DB_CONN_SUFFIX }}
RDS_DB_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
AWS_ACCESS_KEY_ID: ${{ env.TEMP_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ env.TEMP_AWS_SECRET_ACCESS_KEY }}
AWS_SESSION_TOKEN: ${{ env.TEMP_AWS_SESSION_TOKEN }}
AURORA_MYSQL_DB_ENGINE_VERSION: ${{ secrets.engine_version }}
AURORA_PG_ENGINE_VERSION: ${{ matrix.engine_version }}

- name: 'Archive results'
if: always()
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/integration_tests_codebuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ jobs:
matrix:
python-version: [ "3.8", "3.11" ]
environment: [ "mysql", "pg" ]
engine_version: ["latest"]

runs-on: ubuntu-latest
environment: ${{ matrix.environment }}_integ
Expand Down Expand Up @@ -67,13 +66,13 @@ jobs:
AWS_SECRET_ACCESS_KEY: ${{ env.TEMP_AWS_SECRET_ACCESS_KEY }}
AWS_SESSION_TOKEN: ${{ env.TEMP_AWS_SESSION_TOKEN }}
RDS_ENDPOINT: ${{ secrets.RDS_ENDPOINT }}
AURORA_MYSQL_DB_ENGINE_VERSION: ${{ secrets.engine_version }}
AURORA_PG_ENGINE_VERSION: ${{ matrix.engine_version }}
AURORA_MYSQL_DB_ENGINE_VERSION: "latest"
AURORA_PG_ENGINE_VERSION: "latest"

- name: 'Archive results'
if: always()
uses: actions/upload-artifact@v3
with:
name: pytest-integration-report-${{ matrix.environment }}
name: pytest-integration-report
path: ./tests/integration/container/reports
retention-days: 5
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ The following table lists the connection properties used with the AWS Advanced P
### Using the AWS Advanced Python Driver

Technical documentation regarding the functionality of the AWS Advanced Python Driver will be maintained in this GitHub repository. Since the AWS Advanced Python Driver requires an underlying Python driver, please refer to the individual driver's documentation for driver-specific information.
To find all the documentation and concrete examples on how to use the AWS Advanced Python Driver, please refer to the [AWS Advanced Python Driver Documentation](./docs/Documentation.md) page.
To find all the documentation and concrete examples on how to use the AWS Advanced Python Driver, please refer to the [AWS Advanced Python Driver Documentation](./docs/README) page.

### Known Limitations

Expand Down Expand Up @@ -196,6 +196,7 @@ For all other questions, please use [GitHub discussions](https://github.com/awsl
3. Make any changes on your fork. Make sure you are aware of the requirements for the project (e.g. do not require Python 3.7 if we are supporting Python 3.8 and higher).
4. Create a pull request from your fork.
5. Pull requests need to be approved and merged by maintainers into the main branch. <br />

> [!NOTE]\
> Before making a pull request, [run all tests](./docs/development-guide/DevelopmentGuide.md#running-the-tests) and verify everything is passing.

Expand All @@ -213,7 +214,7 @@ This `aws-advanced-python-wrapper` is being tested against the following Communi

| Database | Versions |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| MySQL | 8.0.32 |
| MySQL | 8.3.0 |
| PostgreSQL | 15.2 |
| Aurora MySQL | MySQL 8.0.mysql_aurora.3.02.2 (Wire-compatible with MySQL 8.0.23 onward. For more details see [here](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/AuroraMySQL.Updates.3022.html)) |
| Aurora PostgreSQL | 14.7 and 15.2 (Compatible with PostgreSQL 14.7 and 15.2, see release notes [here](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraPostgreSQLReleaseNotes/AuroraPostgreSQL.Updates.html)) |
Expand Down
18 changes: 17 additions & 1 deletion docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,13 @@ Before using the AWS Advanced Python Driver, you must install:
- Your choice of underlying Python driver.
- To use the wrapper with Aurora with PostgreSQL compatibility, install [Psycopg](https://github.com/psycopg/psycopg).
- To use the wrapper with Aurora with MySQL compatibility, install [MySQL Connector/Python](https://github.com/mysql/mysql-connector-python).
> [!NOTE]\
> The driver has been verified on Psycopg 3.1.12+ and MySQL Connector/Python 8.1.0+. Compatibility with prior versions have not been tested.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
> The driver has been verified on Psycopg 3.1.12+ and MySQL Connector/Python 8.1.0+. Compatibility with prior versions have not been tested.
> The driver has been verified on Psycopg 3.1.12+ and MySQL Connector/Python 8.1.0+. Compatibility with prior versions has not been tested.


## Obtaining the AWS Advanced Python Driver

You can install the AWS Advanced Python Driver and the underlying Python drivers via [pip](https://pip.pypa.io/en/stable/).
The order of installation does not matter.

To use the AWS Advanced Python Driver with Psycopg for Aurora PostgreSQL, run:

Expand All @@ -39,10 +42,23 @@ awsconn = AwsWrapperConnection.connect(
Connection.connect,
"host=database.cluster-xyz.us-east-1.rds.amazonaws.com dbname=db user=john password=pwd",
plugins="failover",
wrapper_dialect="aurora-pg",
autocommit=True
)
```

Similarly, to start using the driver with MySQL Connector/Python, you need to pass the connect function to the `AwsWrapperConnection#connect` method as shown in the following example:
```python
awsconn = AwsWrapperConnection.connect(
mysql.connector.Connect,
host="database.cluster-xyz.us-east-1.rds.amazonaws.com",
database="mysql",
user="admin",
password="pwd",
plugins="failover",
autocommit=True
)
```

The `AwsWrapperConnection#connect` method accepts the connection configuration through both the connection string and the keyword arguments.
You can either pass the connection configuration entirely through the connection string, entirely though the keyword arguments, or through both the connection string and the keywords arguments as shown below.

Expand Down
File renamed without changes.
2 changes: 1 addition & 1 deletion docs/development-guide/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ During execution, plugins may utilize the plugin service to help its execution b
> - plugin service
> - loaded plugin classes
> - Multiple Python Connection objects opened to the same database server will have separate sets of instances mentioned above.
> - All plugins share the same instance of plugin service and the same instance of host list provider.
> - All plugins share the same instance of plugin service and the same instance of host list provider.
20 changes: 10 additions & 10 deletions docs/development-guide/DevelopmentGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,21 +15,21 @@ You can now make changes in the repository.

The AWS Advanced Python Driver uses the following tests to verify its correctness:

| Tests | Description |
|-----------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| Unit tests | Tests for AWS Advanced Python Driver correctness. |
| Failover integration tests | Driver-specific tests for different reader and writer failover workflows using the Failover Connection Plugin. |
| Enhanced failure monitoring integration tests | Driver-specific tests for the enhanced failure monitoring functionality using the Host Monitoring Connection Plugin. |
| AWS authentication integration tests | Driver-specific tests for AWS authentication methods with the AWS Secrets Manager Plugin or the AWS IAM Authentication Plugin. |
| Tests | Description |
|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
| Unit tests | Tests for AWS Advanced Python Driver correctness. |
| Failover integration tests | Driver-specific tests for different reader and writer failover workflows using the Failover Connection Plugin. |
| Enhanced failure monitoring integration tests | Driver-specific tests for the enhanced failure monitoring functionality using the Host Monitoring Connection Plugin. |
| AWS authentication integration tests | Driver-specific tests for AWS authentication methods with the AWS Secrets Manager Plugin or the AWS IAM Authentication Plugin. |

### Performance Tests

The AWS Advanced Python Driver has the following tests to verify its performance:

| Tests | Description |
|-----------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| Connection plugin manager benchmarks | The [benchmarks](../../benchmarks/README.md) subproject measures the overhead from executing Python method calls with multiple connection plugins enabled. |
| Manually-triggered performance tests | The [failover plugin performance tests](../../tests/integration/container/test_failover_performance.py) and [enhanced failure monitoring performance tests](../..tests/integration/container/test_read_write_splitting_performance.py) measure the plugins' performance under different configurations |
| Tests | Description |
|--------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Connection plugin manager benchmarks | The [benchmarks](../../benchmarks/README.md) subproject measures the overhead from executing Python method calls with multiple connection plugins enabled. |
| Manually-triggered performance tests | The [failover plugin performance tests](../../tests/integration/container/test_failover_performance.py) and [enhanced failure monitoring performance tests](../..tests/integration/container/test_read_write_splitting_performance.py) measure the plugins' performance under different configurations |

### Running the Tests

Expand Down
8 changes: 4 additions & 4 deletions docs/development-guide/IntegrationTests.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ This documentation walks through the requirements and steps required to run and
### Aurora Test Requirements

- An AWS account with:
- RDS permissions
- RDS permissions.
- EC2 permissions so integration tests can add the current IP address in the Aurora cluster's EC2 security group.
- For more information,
see: [Setting Up for Amazon RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html).
- XRay and Cloudwatch permissions for the integration tests to report telemetries.
- For more information, see: [Setting Up for Amazon RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html).

## Recommendations for ease of use

Expand Down Expand Up @@ -85,7 +85,7 @@ unset FILTER # Done testing the IAM tests, unset FILTER

Remote path: `/app`

6. Click OK, Apply, OK.
4. Click OK, Apply, OK.
3. At the top right of the Pycharm window, click the dropdown arrow and select your newly created debug configuration
4. Click the bug symbol to start the debug configuration. Pycharm will begin listening for a debug connection
5. Navigate to your Pycharm git bash terminal. In the terminal, navigate to the project root directory. Execute a gradle
Expand Down
2 changes: 1 addition & 1 deletion docs/development-guide/PluginService.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Plugin Service
# Plugin Service

<div style="center"><img src="../images/plugin_service.png" alt="diagram for the plugin service design"/></div>

Expand Down
2 changes: 1 addition & 1 deletion docs/using-the-python-driver/DatabaseDialects.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,4 @@ Once the custom dialect class has been created, tell the AWS Advanced Python Dri
```python
custom_dialect: Dialect = CustomDialect()
DialectManager.custom_dialect = custom_dialect
```
```
1 change: 0 additions & 1 deletion docs/using-the-python-driver/DriverDialects.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ By default, the driver dialect is determined based on the Connection function pa
>
> The `wrapper_driver_dialect` parameter is not required. When it is not provided by the user, the AWS Advanced Python Driver will determine which of the existing target driver dialects to use based on the Connect function passed to the driver. If target driver specific implementation is not found, the AWS Advanced Python Driver will use a generic target driver dialect.


### List of Available Driver Codes

Driver Dialect codes specify which driver dialect class to use.
Expand Down
Loading