fix: DATASET_READ_TABLE read permissions #1237
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
May 1, 2024 13:47
… the table is deleted
SofiaSazonova
changed the title
|
Checks
Give permissions
Remove permissions
Database chages
|
|
Also solves #1171 steps 2 and 3 Step1. Grant permissions to env.admin --> |
added 4 commits
May 1, 2024 16:39
|
After conversation with @noah-paige I understood that for step 1 I have done the opposite thing. So, reverting it |
noah-paige
reviewed
May 1, 2024
backend/dataall/core/permissions/db/resource_policy/resource_policy_repositories.py
Outdated
Show resolved
Hide resolved
noah-paige
reviewed
May 1, 2024
noah-paige
reviewed
May 1, 2024
noah-paige
reviewed
May 1, 2024
...nd/migrations/versions/458572580709_remove_dataset_table_read_permissions_from_env_admins.py
Outdated
Show resolved
Hide resolved
noah-paige
reviewed
May 1, 2024
backend/migrations/versions/59ab197fe648_add_backfill_read_table_permissions.py
Outdated
Show resolved
Hide resolved
|
Just 1 pending comment on migration script Otherwise I think this PR is good and it solves issue #1171 but I am not sure it solves #1173 completely. For that one I think it is more related to the following scenario:
Similarly for revokes we remove the permissions then process the revoke - so a requestor loses permissions and then the revoke may fail Maybe it is best that we add permissions post-approve processing of share and we remove permissions post-revoke processing of share cc: @SofiaSazonova @dlpzx |
|
@noah-paige good catch! |
…is processed successfully
|
@dlpzx @noah-paige I placed attachement/deletion of permissions inside share processor workers. So, the permissions are only granted|revoked when the share item is processed successfully. |
added 2 commits
May 2, 2024 17:00
noah-paige
reviewed
May 3, 2024
backend/dataall/modules/dataset_sharing/services/data_sharing_service.py
Outdated
Show resolved
Hide resolved
noah-paige
reviewed
May 3, 2024
...d/dataall/modules/dataset_sharing/services/share_processors/s3_access_point_process_share.py
Outdated
Show resolved
Hide resolved
noah-paige
reviewed
May 3, 2024
| @@ -1,4 +1,5 @@ | |||
| from unittest.mock import MagicMock | |||
| from dataall.base.context import set_context, RequestContext | |||
There was a problem hiding this comment.
do we need these imports?
There was a problem hiding this comment.
did you push commit to remove this? Still see in after pulling latest changes
noah-paige
reviewed
May 3, 2024
...nd/migrations/versions/458572580709_remove_dataset_table_read_permissions_from_env_admins.py
Outdated
Show resolved
Hide resolved
noah-paige
approved these changes
May 3, 2024
noah-paige
added a commit
that referenced
this pull request
Jun 25, 2024
commit 6968e67c
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:12:45 GMT-0400 (Eastern Daylight Time)
Get to v2.5.0
commit 93ff7725
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 08:00:38 GMT-0400 (Eastern Daylight Time)
Update version.json (#1264)
Release info update
commit e718d861
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 07:29:27 GMT-0400 (Eastern Daylight Time)
fix permission query (#1263)
### Feature or Bugfix
- Bugfix
### Detail
- The filter -- array of permissions' NAMES, so in order to query
policies correctly we need to add join
- The filter 'share_type' and 'share_item_status' must be string
- IMPORTANT: in block "finally" the param session was used, but session
was defined only in "try" block. So, the lock failed to be released.
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 479b8f3f
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Wed May 08 2024 10:29:36 GMT-0400 (Eastern Daylight Time)
Add encryption and tag immutability to ECR repository (#1224)
### Feature or Bugfix
- Bugfix
### Detail
- Currently the ecr repository created do not have encryption and tag
immutability enabled which is identified by checkov scans. This fix is
to enable both.
### Relates
[- <URL or Ticket>](https://github.com/data-dot-all/dataall/issues/1200)
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
N/A
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? N/A
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? N/A
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
No. This is with default encryption
- Are you introducing any new policies/roles/users? N/A
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 2f885773
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Wed May 08 2024 09:22:40 GMT-0400 (Eastern Daylight Time)
Multiple permission roots (#1259)
### Feature or Bugfix
- Bugfix
### Detail
- GET_DATASET_TABLE (FOLDER) permissions are granted to the group only
if they are not granted already
- these permissions are removed if group is not admin|steward and there
are no other shares of this item.
### Relates
- #1174
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit c4cc07ee
Author: Petros Kalos <petrkalos@gmail.com>
Date: Wed May 08 2024 08:54:02 GMT-0400 (Eastern Daylight Time)
explicitly specify dataset_client s3 endpoint_url (#1260)
* AWS requires that the endpoint_url should be explicitly specified for
some regions
* Remove misleading CORS error message, the upload step can fail for
many reason
### Feature or Bugfix
- Bugfix
### Detail
Resolves #778
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 40defe8e
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue May 07 2024 11:52:17 GMT-0400 (Eastern Daylight Time)
Generic dataset module and specific s3_datasets module - part 1 (Rename datasets as s3_datasets) (#1250)
### Feature or Bugfix
- Refactoring
### Detail
- Rename `datasets` module to `s3_datasets` module
This PR is the first step to extract a generic datasets_base module that
implements the undifferentiated concepts of Dataset in data.all.
s3_datasets will use this base module to implement the specific
implementation for S3 datatasets.
### Relates
- #1123
- #955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 74a303cb
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:26:09 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests (#1253)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 2f33320c
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:25:03 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /backend/dataall/base/cdkproxy (#1252)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 0b49633f
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:24:34 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests_new/integration_tests (#1254)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 08862420
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Tue May 07 2024 02:15:15 GMT-0400 (Eastern Daylight Time)
Updated lambda_api.py to add encryption for lambda env vars for custo… (#1255)
Feature or Bugfix
Bugfix
Detail
The environment variables for the lambda functions are not encrypted in
cdk which are identified by checkov scans. This fix is to enable kms
encryption for the lambda environment variables.
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based on
[OWASP 10](https://owasp.org/Top10/en/).
Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? N/A
Is the input sanitized? N/A
What precautions are you taking before deserializing the data you
consume? N/A
Is injection prevented by parametrizing queries? N/A
Have you ensured no eval or similar functions are used? N/A
Does this PR introduce any functionality or component that requires
authorization? N/A
How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
Are you logging failed auth attempts? N/A
Are you using or adding any cryptographic features? N/A
Do you use a standard proven implementations? N/A
Are the used keys controlled by the customer? Where are they stored? the
KMS keys are generated by cdk and are used to encrypt the environment
variables for all lambda functions in the lambda-api stack
Are you introducing any new policies/roles/users? - N/A
Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit ed7cc3eb
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon May 06 2024 09:32:30 GMT-0400 (Eastern Daylight Time)
Add order_by for paginated queries (#1249)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- This PR aims to solve the following
- (1) for particular queries (identified as ones that perform
`.outerjoin()` operations and have results paginated with `paginate()`
function - sometimes the returned query results is *less than* the limit
set by the pageSize of the paginate function even when the total count
is greater than the pageSize
- Ex 1: 11 envs total, `query_user_environments()` returning 9 envs on
1st page + 2 on 2nd page
- Ex 2: 10 envs total, `query_user_environments()` returning 9 envs on
1st page + no 2nd page
- Believe this is to be happening due to the way SQLAlchemy is
"uniquing" the records resulted from an outerjoin and then returning
that result back to the frontend
- Adding a `.distinct()` check on the query ensures each distinct record
is returned (tested successfully)
- (2) Currently we often times do not implement an `.order_by()`
condition for the query used in `paginate()` and do not have a stable
way of preserving order of the items returned from a query (i.e. when
navigating through pages of response)
- A generally good practice seems to include an `order_by()` on a column
or set of columns
- For each query used in `paginate()` this PR adds an `order_by()`
condition (full list in comments below)
Can read a bit more context from related issue linked below
### Relates
- https://github.com/data-dot-all/dataall/issues/1241
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 98e67fa8
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 12:21:57 GMT-0400 (Eastern Daylight Time)
fix: DATASET_READ_TABLE read permissions (#1237)
### Feature or Bugfix
- Bugfix
### Detail
- backfill DATASET_READ_TABLE permissions
- delete this permissions, when dataset tables are revoked or deteled
-
### Relates
- #1173
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 18e2f509
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 10:14:52 GMT-0400 (Eastern Daylight Time)
Fix local test groups listing for listGroups query (#1239)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Locally when trying to invite a team to Env or Org we call listGroups
and the returned `LOCAL_TEST_GROUPS` is not returning the proper data
type expected
### Relates
N/A
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit a0be03c4
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 10:12:34 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-5 FINAL DELETE DATASETS_BASE (#1242)
### Feature or Bugfix
- Refactoring
### Detail
After all the previous PRs are merged, there should be no circular
dependencies between `datasets` and `datasets_sharing`. We can now
proceed to:
- move `datasets_base` models, repositories, permissions and enums to
`datasets`
- adjust the `__init__` files to establish the `datasets_sharing`
depends on `datasets`
- adjust the Module interfaces to ensure that all necessary dataset
models... are imported in the interface for sharing
Next steps:
- share_notifications paramter to dataset_sharing in config.json
### Relates
#955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b68b40c1
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 10:12:11 GMT-0400 (Eastern Daylight Time)
bugfix: EnvironmentGroup can remove other groups (#1234)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Now, if the group can't update other group, it also can not remove
them.
-
### Relates
- #1212
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 264539b5
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 05:23:11 GMT-0400 (Eastern Daylight Time)
Fix Alembic Migration: has table checks (#1240)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Fix `has_table()` check to ensure dropping the tables if the exists as
part of alembic migration upgrade
- Fix `DatasetLock nullable=True`
### Relates
- https://github.com/data-dot-all/dataall/issues/1165
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? No
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? No
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? No
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
N/A
- Are you introducing any new policies/roles/users? No
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 42a5f6bd
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 02:24:09 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-4 (#1214)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1213
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Use interface to resolve dataset roles related to datasets shared
and implement logic in the dataset_sharing module
- [X] Extend and clean-up stewards share permissions through interface
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 6d3f2d45
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Thu May 02 2024 10:55:00 GMT-0400 (Eastern Daylight Time)
[After 2.4]Core Refactoring part5 (#1194)
### Feature or Bugfix
- Refactoring
### Detail
- focus on core/environments
- move logic from resolvers to services
- create s3_client in base/aws --> TO BE REFACTORED. Needs to be merged
with dataset_sharind/aws/s3_client
### Relates
- #741
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 2ea24cbb
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Thu May 02 2024 08:22:12 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-3 (#1213)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1187
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Creates an interface to execute checks and clean-ups of data
sharing objects when dataset objects are deleted (initially it was going
to be an db interface, but I think it is better in the service)
- [X] Move listDatasetShares query to dataset_sharing module in
https://github.com/data-dot-all/dataall/pull/1185
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 750a5ec8
Author: Anushka Singh <anushka.singh.2511@gmail.com>
Date: Wed May 01 2024 12:28:18 GMT-0400 (Eastern Daylight Time)
Feature:1221 - Make visibility of auto-approval toggle configurable based on confidentiality (#1223)
### Feature or Bugfix
- Feature
### Detail
- Users should be able to disable visibility of auto-approval toggle
with code. For example, at our company, we require that shares always go
through approval process if their confidentiality classification is
Secret. We dont even want to give the option to users to be able to set
autoApproval enabled to ensure they dont do so by mistake and end up
over sharing.
Video demo:
https://github.com/data-dot-all/dataall/issues/1221#issuecomment-2077412044
### Relates
- https://github.com/data-dot-all/dataall/issues/1221
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 82044689
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Wed May 01 2024 12:26:42 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-2 (#1187)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1185
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- Split the getDatasetAssumeRole API into 2 APIs, one for dataset owners
role (in datasets module) and another one for share requester roles (in
datasets_sharing module)
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 5173419f
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed May 01 2024 12:24:42 GMT-0400 (Eastern Daylight Time)
Fix so listValidEnvironments called only once (#1238)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When request access to a share on data.all the query to
`listValidEnvironments` used to be called twice which (depending on how
long for query results to return) could cause the environment initially
selected to disappear
### Relates
- Continuation of https://github.com/data-dot-all/dataall/issues/916
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 7656ea86
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Apr 30 2024 07:13:01 GMT-0400 (Eastern Daylight Time)
Add integration tests on a real API client and integrate the tests in CICD (#1219)
### Feature or Bugfix
- Feature
### Detail
Add integration tests that use a real Client to execute different
validation actions.
- Define the Client and the way API calls are posted to API Gateway in
the conftest
- Define the Cognito users and the different fixtures needed for all
tests
- Write tests for the Organization core module as example
- Add feature flag in `cdk.json` called `with_approval_tests` that can
be defined at the deployment environment level. If set to True, a
CodeBuild stage running the tests is created.
### Relates
- https://github.com/data-dot-all/dataall/issues/1220
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b963fe81
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon Apr 29 2024 09:26:36 GMT-0400 (Eastern Daylight Time)
Notification link routes to a share request page (#1227)
### Feature or Bugfix
<!-- please choose -->
- Feature
### Detail
- in notification object field `target_uri = 'shareUri|DataSetUri'`
- this value is parsed and used to redirect user to a relevant Share
Request page
### Relates
- #1115
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 6386fe14
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Mon Apr 29 2024 07:32:00 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2 (#1185)
### Feature or Bugfix
- Refactoring
### Detail
Remove and move logic from dataset to datasets_sharing module. This is
needed as explained in full PR [AFTER 2.4] Refactor: uncouple datasets
and dataset_sharing modules #1179
- [X] Moves the verify dataset shares mutation to the datasets_sharing
module
- [X] Move dataset_subscription task to dataset_sharing
- [X] Move listDatasetShares query to dataset_sharing module
- [X] Remove unused `shares` field from the Dataset graphql type as it
was not used in the frontend: listDatasets, listOwnedDatasets,
listDatasetsOwnedByEnvGroup, listDatasetsCreatedInEnvironment and
getDataset
- [x] Move getSharedDatasetTables to data_sharing module and fix
reference to DatasetService
I am aware that some of the queries and mutations that this PR moves
look a bit odd in the dataset_sharing module, but this will be solved
once data sharing is divided into dataset_sharing_base and
s3_dataset_sharing.
### Relates
#1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
aut…
noah-paige
added a commit
that referenced
this pull request
Jun 25, 2024
commit a06c8cba
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:37:05 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit aee98cf7
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:34:24 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit 5ca55303
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:57:41 GMT-0400 (Eastern Daylight Time)
remove unused imports
commit 8f8bf3dd
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:56:39 GMT-0400 (Eastern Daylight Time)
restrict access to the share logs
commit 9137da9b
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:28:32 GMT-0400 (Eastern Daylight Time)
share Logs button is available only for dataset Admins and stewards
commit fcb16bd9
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:46:25 GMT-0400 (Eastern Daylight Time)
getShareLogs query
commit 0503a3bb
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:21:25 GMT-0400 (Eastern Daylight Time)
Logs modal in Share View
commit bab2f3e6
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 09:09:18 GMT-0400 (Eastern Daylight Time)
Add confirmation pop-ups for deletion of team roles and groups (#1231)
### Feature or Bugfix
- Feature
### Detail
Pop ups added for:
- deletion team from environment
- deletion of the consumption role
- deletion of group from Organization
### Relates
- #942
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 93ff7725
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 08:00:38 GMT-0400 (Eastern Daylight Time)
Update version.json (#1264)
Release info update
commit e718d861
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 07:29:27 GMT-0400 (Eastern Daylight Time)
fix permission query (#1263)
### Feature or Bugfix
- Bugfix
### Detail
- The filter -- array of permissions' NAMES, so in order to query
policies correctly we need to add join
- The filter 'share_type' and 'share_item_status' must be string
- IMPORTANT: in block "finally" the param session was used, but session
was defined only in "try" block. So, the lock failed to be released.
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 479b8f3f
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Wed May 08 2024 10:29:36 GMT-0400 (Eastern Daylight Time)
Add encryption and tag immutability to ECR repository (#1224)
### Feature or Bugfix
- Bugfix
### Detail
- Currently the ecr repository created do not have encryption and tag
immutability enabled which is identified by checkov scans. This fix is
to enable both.
### Relates
[- <URL or Ticket>](https://github.com/data-dot-all/dataall/issues/1200)
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
N/A
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? N/A
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? N/A
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
No. This is with default encryption
- Are you introducing any new policies/roles/users? N/A
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 2f885773
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Wed May 08 2024 09:22:40 GMT-0400 (Eastern Daylight Time)
Multiple permission roots (#1259)
### Feature or Bugfix
- Bugfix
### Detail
- GET_DATASET_TABLE (FOLDER) permissions are granted to the group only
if they are not granted already
- these permissions are removed if group is not admin|steward and there
are no other shares of this item.
### Relates
- #1174
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit c4cc07ee
Author: Petros Kalos <petrkalos@gmail.com>
Date: Wed May 08 2024 08:54:02 GMT-0400 (Eastern Daylight Time)
explicitly specify dataset_client s3 endpoint_url (#1260)
* AWS requires that the endpoint_url should be explicitly specified for
some regions
* Remove misleading CORS error message, the upload step can fail for
many reason
### Feature or Bugfix
- Bugfix
### Detail
Resolves #778
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 40defe8e
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue May 07 2024 11:52:17 GMT-0400 (Eastern Daylight Time)
Generic dataset module and specific s3_datasets module - part 1 (Rename datasets as s3_datasets) (#1250)
### Feature or Bugfix
- Refactoring
### Detail
- Rename `datasets` module to `s3_datasets` module
This PR is the first step to extract a generic datasets_base module that
implements the undifferentiated concepts of Dataset in data.all.
s3_datasets will use this base module to implement the specific
implementation for S3 datatasets.
### Relates
- #1123
- #955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 74a303cb
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:26:09 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests (#1253)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 2f33320c
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:25:03 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /backend/dataall/base/cdkproxy (#1252)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 0b49633f
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:24:34 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests_new/integration_tests (#1254)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 08862420
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Tue May 07 2024 02:15:15 GMT-0400 (Eastern Daylight Time)
Updated lambda_api.py to add encryption for lambda env vars for custo… (#1255)
Feature or Bugfix
Bugfix
Detail
The environment variables for the lambda functions are not encrypted in
cdk which are identified by checkov scans. This fix is to enable kms
encryption for the lambda environment variables.
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based on
[OWASP 10](https://owasp.org/Top10/en/).
Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? N/A
Is the input sanitized? N/A
What precautions are you taking before deserializing the data you
consume? N/A
Is injection prevented by parametrizing queries? N/A
Have you ensured no eval or similar functions are used? N/A
Does this PR introduce any functionality or component that requires
authorization? N/A
How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
Are you logging failed auth attempts? N/A
Are you using or adding any cryptographic features? N/A
Do you use a standard proven implementations? N/A
Are the used keys controlled by the customer? Where are they stored? the
KMS keys are generated by cdk and are used to encrypt the environment
variables for all lambda functions in the lambda-api stack
Are you introducing any new policies/roles/users? - N/A
Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit ed7cc3eb
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon May 06 2024 09:32:30 GMT-0400 (Eastern Daylight Time)
Add order_by for paginated queries (#1249)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- This PR aims to solve the following
- (1) for particular queries (identified as ones that perform
`.outerjoin()` operations and have results paginated with `paginate()`
function - sometimes the returned query results is *less than* the limit
set by the pageSize of the paginate function even when the total count
is greater than the pageSize
- Ex 1: 11 envs total, `query_user_environments()` returning 9 envs on
1st page + 2 on 2nd page
- Ex 2: 10 envs total, `query_user_environments()` returning 9 envs on
1st page + no 2nd page
- Believe this is to be happening due to the way SQLAlchemy is
"uniquing" the records resulted from an outerjoin and then returning
that result back to the frontend
- Adding a `.distinct()` check on the query ensures each distinct record
is returned (tested successfully)
- (2) Currently we often times do not implement an `.order_by()`
condition for the query used in `paginate()` and do not have a stable
way of preserving order of the items returned from a query (i.e. when
navigating through pages of response)
- A generally good practice seems to include an `order_by()` on a column
or set of columns
- For each query used in `paginate()` this PR adds an `order_by()`
condition (full list in comments below)
Can read a bit more context from related issue linked below
### Relates
- https://github.com/data-dot-all/dataall/issues/1241
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 98e67fa8
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 12:21:57 GMT-0400 (Eastern Daylight Time)
fix: DATASET_READ_TABLE read permissions (#1237)
### Feature or Bugfix
- Bugfix
### Detail
- backfill DATASET_READ_TABLE permissions
- delete this permissions, when dataset tables are revoked or deteled
-
### Relates
- #1173
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 18e2f509
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 10:14:52 GMT-0400 (Eastern Daylight Time)
Fix local test groups listing for listGroups query (#1239)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Locally when trying to invite a team to Env or Org we call listGroups
and the returned `LOCAL_TEST_GROUPS` is not returning the proper data
type expected
### Relates
N/A
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit a0be03c4
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 10:12:34 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-5 FINAL DELETE DATASETS_BASE (#1242)
### Feature or Bugfix
- Refactoring
### Detail
After all the previous PRs are merged, there should be no circular
dependencies between `datasets` and `datasets_sharing`. We can now
proceed to:
- move `datasets_base` models, repositories, permissions and enums to
`datasets`
- adjust the `__init__` files to establish the `datasets_sharing`
depends on `datasets`
- adjust the Module interfaces to ensure that all necessary dataset
models... are imported in the interface for sharing
Next steps:
- share_notifications paramter to dataset_sharing in config.json
### Relates
#955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b68b40c1
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 10:12:11 GMT-0400 (Eastern Daylight Time)
bugfix: EnvironmentGroup can remove other groups (#1234)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Now, if the group can't update other group, it also can not remove
them.
-
### Relates
- #1212
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 264539b5
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 05:23:11 GMT-0400 (Eastern Daylight Time)
Fix Alembic Migration: has table checks (#1240)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Fix `has_table()` check to ensure dropping the tables if the exists as
part of alembic migration upgrade
- Fix `DatasetLock nullable=True`
### Relates
- https://github.com/data-dot-all/dataall/issues/1165
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? No
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? No
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? No
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
N/A
- Are you introducing any new policies/roles/users? No
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 42a5f6bd
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 02:24:09 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-4 (#1214)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1213
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Use interface to resolve dataset roles related to datasets shared
and implement logic in the dataset_sharing module
- [X] Extend and clean-up stewards share permissions through interface
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 6d3f2d45
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Thu May 02 2024 10:55:00 GMT-0400 (Eastern Daylight Time)
[After 2.4]Core Refactoring part5 (#1194)
### Feature or Bugfix
- Refactoring
### Detail
- focus on core/environments
- move logic from resolvers to services
- create s3_client in base/aws --> TO BE REFACTORED. Needs to be merged
with dataset_sharind/aws/s3_client
### Relates
- #741
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 2ea24cbb
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Thu May 02 2024 08:22:12 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-3 (#1213)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1187
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Creates an interface to execute checks and clean-ups of data
sharing objects when dataset objects are deleted (initially it was going
to be an db interface, but I think it is better in the service)
- [X] Move listDatasetShares query to dataset_sharing module in
https://github.com/data-dot-all/dataall/pull/1185
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 750a5ec8
Author: Anushka Singh <anushka.singh.2511@gmail.com>
Date: Wed May 01 2024 12:28:18 GMT-0400 (Eastern Daylight Time)
Feature:1221 - Make visibility of auto-approval toggle configurable based on confidentiality (#1223)
### Feature or Bugfix
- Feature
### Detail
- Users should be able to disable visibility of auto-approval toggle
with code. For example, at our company, we require that shares always go
through approval process if their confidentiality classification is
Secret. We dont even want to give the option to users to be able to set
autoApproval enabled to ensure they dont do so by mistake and end up
over sharing.
Video demo:
https://github.com/data-dot-all/dataall/issues/1221#issuecomment-2077412044
### Relates
- https://github.com/data-dot-all/dataall/issues/1221
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 82044689
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Wed May 01 2024 12:26:42 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-2 (#1187)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1185
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- Split the getDatasetAssumeRole API into 2 APIs, one for dataset owners
role (in datasets module) and another one for share requester roles (in
datasets_sharing module)
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 5173419f
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed May 01 2024 12:24:42 GMT-0400 (Eastern Daylight Time)
Fix so listValidEnvironments called only once (#1238)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When request access to a share on data.all the query to
`listValidEnvironments` used to be called twice which (depending on how
long for query results to return) could cause the environment initially
selected to disappear
### Relates
- Continuation of https://github.com/data-dot-all/dataall/issues/916
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 7656ea86
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Apr 30 2024 07:13:01 GMT-0400 (Eastern Daylight Time)
Add integration tests on a real API client and integrate the tests in CICD (#1219)
### Feature or Bugfix
- Feature
### Detail
Add integration tests that use a real Client to execute different
validation actions.
- Define the Client and the way API calls are posted to API Gateway in
the conftest
- Define the Cognito users and the different fixtures needed for all
tests
- Write tests for the Organization core module as example
- Add feature flag in `cdk.json` called `with_approval_tests` that can
be defined at the deployment environment level. If set to True, a
CodeBuild stage running the tests is created.
### Relates
- https://github.com/data-dot-all/dataall/issues/1220
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b963fe81
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon Apr 29 2024 09:26:36 GMT-0400 (Eastern Daylight Time)
Notification link routes to a share request page (#1227)
### Feature or Bugfix
<!-- please choose -->
- Feature
### Detail
- in notification object field `target_uri = 'shareUri|DataSetUri'`
- this value is parsed and used to redirect user to a relevant Share
Request page
### Relates
- #1115
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the da…
noah-paige
added a commit
that referenced
this pull request
Jun 25, 2024
commit d8497c55
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:19:35 GMT-0400 (Eastern Daylight Time)
fix
commit 199ab505
Author: Admin/noahpaig-Isengard <Admin/noahpaig-Isengard>
Date: Mon May 20 2024 15:16:14 GMT-0400 (Eastern Daylight Time)
Conflicts resolved in the console.
commit ad415575
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:13:06 GMT-0400 (Eastern Daylight Time)
Merge branch 'chatbot-test' into noah-main-2
commit 2893efc7
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:10:20 GMT-0400 (Eastern Daylight Time)
Merge branch 'chatbot-test' into noah-main-2
commit caad12e1
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:12:47 GMT-0400 (Eastern Daylight Time)
ruff
commit a73b7110
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:09:19 GMT-0400 (Eastern Daylight Time)
Remove hardcoding
commit 7b32a8f7
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:07:29 GMT-0400 (Eastern Daylight Time)
Chatbot POC
commit e25e5815
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:08:32 GMT-0400 (Eastern Daylight Time)
Merge branch '1215-share-logs' into noah-main-2
commit a06c8cba
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:37:05 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit a5626670
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Fri May 17 2024 17:19:25 GMT-0400 (Eastern Daylight Time)
make ruff happy
commit aee98cf7
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:34:24 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit c3ee2f7c
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Fri May 17 2024 17:11:00 GMT-0400 (Eastern Daylight Time)
PR comments
commit 5ca55303
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:57:41 GMT-0400 (Eastern Daylight Time)
remove unused imports
commit 8f8bf3dd
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:56:39 GMT-0400 (Eastern Daylight Time)
restrict access to the share logs
commit 9137da9b
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:28:32 GMT-0400 (Eastern Daylight Time)
share Logs button is available only for dataset Admins and stewards
commit fcb16bd9
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:46:25 GMT-0400 (Eastern Daylight Time)
getShareLogs query
commit 0503a3bb
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:21:25 GMT-0400 (Eastern Daylight Time)
Logs modal in Share View
commit bab2f3e6
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 09:09:18 GMT-0400 (Eastern Daylight Time)
Add confirmation pop-ups for deletion of team roles and groups (#1231)
### Feature or Bugfix
- Feature
### Detail
Pop ups added for:
- deletion team from environment
- deletion of the consumption role
- deletion of group from Organization
### Relates
- #942
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 93ff7725
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 08:00:38 GMT-0400 (Eastern Daylight Time)
Update version.json (#1264)
Release info update
commit e718d861
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 07:29:27 GMT-0400 (Eastern Daylight Time)
fix permission query (#1263)
### Feature or Bugfix
- Bugfix
### Detail
- The filter -- array of permissions' NAMES, so in order to query
policies correctly we need to add join
- The filter 'share_type' and 'share_item_status' must be string
- IMPORTANT: in block "finally" the param session was used, but session
was defined only in "try" block. So, the lock failed to be released.
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 479b8f3f
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Wed May 08 2024 10:29:36 GMT-0400 (Eastern Daylight Time)
Add encryption and tag immutability to ECR repository (#1224)
### Feature or Bugfix
- Bugfix
### Detail
- Currently the ecr repository created do not have encryption and tag
immutability enabled which is identified by checkov scans. This fix is
to enable both.
### Relates
[- <URL or Ticket>](https://github.com/data-dot-all/dataall/issues/1200)
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
N/A
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? N/A
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? N/A
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
No. This is with default encryption
- Are you introducing any new policies/roles/users? N/A
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 2f885773
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Wed May 08 2024 09:22:40 GMT-0400 (Eastern Daylight Time)
Multiple permission roots (#1259)
### Feature or Bugfix
- Bugfix
### Detail
- GET_DATASET_TABLE (FOLDER) permissions are granted to the group only
if they are not granted already
- these permissions are removed if group is not admin|steward and there
are no other shares of this item.
### Relates
- #1174
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit c4cc07ee
Author: Petros Kalos <petrkalos@gmail.com>
Date: Wed May 08 2024 08:54:02 GMT-0400 (Eastern Daylight Time)
explicitly specify dataset_client s3 endpoint_url (#1260)
* AWS requires that the endpoint_url should be explicitly specified for
some regions
* Remove misleading CORS error message, the upload step can fail for
many reason
### Feature or Bugfix
- Bugfix
### Detail
Resolves #778
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 40defe8e
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue May 07 2024 11:52:17 GMT-0400 (Eastern Daylight Time)
Generic dataset module and specific s3_datasets module - part 1 (Rename datasets as s3_datasets) (#1250)
### Feature or Bugfix
- Refactoring
### Detail
- Rename `datasets` module to `s3_datasets` module
This PR is the first step to extract a generic datasets_base module that
implements the undifferentiated concepts of Dataset in data.all.
s3_datasets will use this base module to implement the specific
implementation for S3 datatasets.
### Relates
- #1123
- #955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 74a303cb
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:26:09 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests (#1253)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 2f33320c
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:25:03 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /backend/dataall/base/cdkproxy (#1252)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 0b49633f
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:24:34 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests_new/integration_tests (#1254)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 08862420
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Tue May 07 2024 02:15:15 GMT-0400 (Eastern Daylight Time)
Updated lambda_api.py to add encryption for lambda env vars for custo… (#1255)
Feature or Bugfix
Bugfix
Detail
The environment variables for the lambda functions are not encrypted in
cdk which are identified by checkov scans. This fix is to enable kms
encryption for the lambda environment variables.
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based on
[OWASP 10](https://owasp.org/Top10/en/).
Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? N/A
Is the input sanitized? N/A
What precautions are you taking before deserializing the data you
consume? N/A
Is injection prevented by parametrizing queries? N/A
Have you ensured no eval or similar functions are used? N/A
Does this PR introduce any functionality or component that requires
authorization? N/A
How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
Are you logging failed auth attempts? N/A
Are you using or adding any cryptographic features? N/A
Do you use a standard proven implementations? N/A
Are the used keys controlled by the customer? Where are they stored? the
KMS keys are generated by cdk and are used to encrypt the environment
variables for all lambda functions in the lambda-api stack
Are you introducing any new policies/roles/users? - N/A
Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit ed7cc3eb
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon May 06 2024 09:32:30 GMT-0400 (Eastern Daylight Time)
Add order_by for paginated queries (#1249)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- This PR aims to solve the following
- (1) for particular queries (identified as ones that perform
`.outerjoin()` operations and have results paginated with `paginate()`
function - sometimes the returned query results is *less than* the limit
set by the pageSize of the paginate function even when the total count
is greater than the pageSize
- Ex 1: 11 envs total, `query_user_environments()` returning 9 envs on
1st page + 2 on 2nd page
- Ex 2: 10 envs total, `query_user_environments()` returning 9 envs on
1st page + no 2nd page
- Believe this is to be happening due to the way SQLAlchemy is
"uniquing" the records resulted from an outerjoin and then returning
that result back to the frontend
- Adding a `.distinct()` check on the query ensures each distinct record
is returned (tested successfully)
- (2) Currently we often times do not implement an `.order_by()`
condition for the query used in `paginate()` and do not have a stable
way of preserving order of the items returned from a query (i.e. when
navigating through pages of response)
- A generally good practice seems to include an `order_by()` on a column
or set of columns
- For each query used in `paginate()` this PR adds an `order_by()`
condition (full list in comments below)
Can read a bit more context from related issue linked below
### Relates
- https://github.com/data-dot-all/dataall/issues/1241
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 98e67fa8
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 12:21:57 GMT-0400 (Eastern Daylight Time)
fix: DATASET_READ_TABLE read permissions (#1237)
### Feature or Bugfix
- Bugfix
### Detail
- backfill DATASET_READ_TABLE permissions
- delete this permissions, when dataset tables are revoked or deteled
-
### Relates
- #1173
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 18e2f509
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 10:14:52 GMT-0400 (Eastern Daylight Time)
Fix local test groups listing for listGroups query (#1239)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Locally when trying to invite a team to Env or Org we call listGroups
and the returned `LOCAL_TEST_GROUPS` is not returning the proper data
type expected
### Relates
N/A
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit a0be03c4
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 10:12:34 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-5 FINAL DELETE DATASETS_BASE (#1242)
### Feature or Bugfix
- Refactoring
### Detail
After all the previous PRs are merged, there should be no circular
dependencies between `datasets` and `datasets_sharing`. We can now
proceed to:
- move `datasets_base` models, repositories, permissions and enums to
`datasets`
- adjust the `__init__` files to establish the `datasets_sharing`
depends on `datasets`
- adjust the Module interfaces to ensure that all necessary dataset
models... are imported in the interface for sharing
Next steps:
- share_notifications paramter to dataset_sharing in config.json
### Relates
#955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b68b40c1
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 10:12:11 GMT-0400 (Eastern Daylight Time)
bugfix: EnvironmentGroup can remove other groups (#1234)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Now, if the group can't update other group, it also can not remove
them.
-
### Relates
- #1212
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 264539b5
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 05:23:11 GMT-0400 (Eastern Daylight Time)
Fix Alembic Migration: has table checks (#1240)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Fix `has_table()` check to ensure dropping the tables if the exists as
part of alembic migration upgrade
- Fix `DatasetLock nullable=True`
### Relates
- https://github.com/data-dot-all/dataall/issues/1165
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? No
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? No
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? No
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
N/A
- Are you introducing any new policies/roles/users? No
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 42a5f6bd
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 02:24:09 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-4 (#1214)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1213
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Use interface to resolve dataset roles related to datasets shared
and implement logic in the dataset_sharing module
- [X] Extend and clean-up stewards share permissions through interface
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 6d3f2d45
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Thu May 02 2024 10:55:00 GMT-0400 (Eastern Daylight Time)
[After 2.4]Core Refactoring part5 (#1194)
### Feature or Bugfix
- Refactoring
### Detail
- focus on core/environments
- move logic from resolvers to services
- create s3_client in base/aws --> TO BE REFACTORED. Needs to be merged
with dataset_sharind/aws/s3_client
### Relates
- #741
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 2ea24cbb
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Thu May 02 2024 08:22:12 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-3 (#1213)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1187
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Creates an interface to execute checks and clean-ups of data
sharing objects when dataset objects are deleted (initially it was going
to be an db interface, but I think it is better in the service)
- [X] Move listDatasetShares query to dataset_sharing module in
https://github.com/data-dot-all/dataall/pull/1185
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 750a5ec8
Author: Anushka Singh <anushka.singh.2511@gmail.com>
Date: Wed May 01 2024 12:28:18 GMT-0400 (Eastern Daylight Time)
Feature:1221 - Make visibility of auto-approval toggle configurable based on confidentiality (#1223)
### Feature or Bugfix
- Feature
### Detail
- Users should be able to disable visibility of auto-approval toggle
with code. For example, at our company, we require that shares always go
through approval process if their confidentiality classification is
Secret. We dont even want to give the option to users to be able to set
autoApproval enabled to ensure they dont do so by mistake and end up
over sharing.
Video demo:
https://github.com/data-dot-all/dataall/issues/1221#issuecomment-2077412044
### Relates
- https://github.com/data-dot-all/dataall/issues/1221
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 82044689
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Wed May 01 2024 12:26:42 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-2 (#1187)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1185
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- Split the getDatasetAssumeRole API into 2 APIs, one for dataset owners
role (in datasets module) and another one for share requester roles (in
datasets_sharing module)
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 5173419f
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed May 01 2024 12:24:42 GMT-0400 (Eastern Daylight Time)
Fix so listValidEnvironments called only once (#1238)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When request access to a share on data.all the query to
`listValidEnvironments` used to be called twice which (depending on how
long for query results to return) could cause the environment initially
selected to disappear
### Relates
- Continuation of https://github.com/data-dot-all/dataall/issues/916
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 7656ea86
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Apr 30 2024 07:13:01 GMT-0400 (Eastern Daylight Time)
Add integration tests on a real API client and integrate the tests in CICD (#1219)
### Feature or Bugfix
- Feature
### Detail
Add integration tests that use a real Client to execute different
validation actions.
- Define the Client and the way API calls are posted to API Gateway in
the conftest
- Define the Cognito users and the different fixtures needed for all
tests
- Write tests for the Organization core module as example
- Add feature flag in `cdk.json` called `with_approval_tests` that can
be defined at the deployment environment level. If set to True, a
CodeBuild stage running the tests is created.
### Relates
- https://github.com/data-dot-all/dataall/issues/1220
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
c…
noah-paige
added a commit
that referenced
this pull request
Jun 25, 2024
commit 1617953c
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 16:48:17 GMT-0400 (Eastern Daylight Time)
Add open dependency matrix
commit d8497c55
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:19:35 GMT-0400 (Eastern Daylight Time)
fix
commit 199ab505
Author: Admin/noahpaig-Isengard <Admin/noahpaig-Isengard>
Date: Mon May 20 2024 15:16:14 GMT-0400 (Eastern Daylight Time)
Conflicts resolved in the console.
commit ad415575
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:13:06 GMT-0400 (Eastern Daylight Time)
Merge branch 'chatbot-test' into noah-main-2
commit 2893efc7
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:10:20 GMT-0400 (Eastern Daylight Time)
Merge branch 'chatbot-test' into noah-main-2
commit caad12e1
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:12:47 GMT-0400 (Eastern Daylight Time)
ruff
commit a73b7110
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:09:19 GMT-0400 (Eastern Daylight Time)
Remove hardcoding
commit 7b32a8f7
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:07:29 GMT-0400 (Eastern Daylight Time)
Chatbot POC
commit e25e5815
Author: Noah Paige <noahpaig@amazon.com>
Date: Mon May 20 2024 15:08:32 GMT-0400 (Eastern Daylight Time)
Merge branch '1215-share-logs' into noah-main-2
commit a06c8cba
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:37:05 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit a5626670
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Fri May 17 2024 17:19:25 GMT-0400 (Eastern Daylight Time)
make ruff happy
commit aee98cf7
Author: Noah Paige <noahpaig@amazon.com>
Date: Fri May 17 2024 16:34:24 GMT-0400 (Eastern Daylight Time)
Merge share logs PR
commit c3ee2f7c
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Fri May 17 2024 17:11:00 GMT-0400 (Eastern Daylight Time)
PR comments
commit 5ca55303
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:57:41 GMT-0400 (Eastern Daylight Time)
remove unused imports
commit 8f8bf3dd
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:56:39 GMT-0400 (Eastern Daylight Time)
restrict access to the share logs
commit 9137da9b
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 11:28:32 GMT-0400 (Eastern Daylight Time)
share Logs button is available only for dataset Admins and stewards
commit fcb16bd9
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:46:25 GMT-0400 (Eastern Daylight Time)
getShareLogs query
commit 0503a3bb
Author: Sofia Sazonova <sazonova@amazon.co.uk>
Date: Wed May 15 2024 10:21:25 GMT-0400 (Eastern Daylight Time)
Logs modal in Share View
commit bab2f3e6
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 09:09:18 GMT-0400 (Eastern Daylight Time)
Add confirmation pop-ups for deletion of team roles and groups (#1231)
### Feature or Bugfix
- Feature
### Detail
Pop ups added for:
- deletion team from environment
- deletion of the consumption role
- deletion of group from Organization
### Relates
- #942
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 93ff7725
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 08:00:38 GMT-0400 (Eastern Daylight Time)
Update version.json (#1264)
Release info update
commit e718d861
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Mon May 13 2024 07:29:27 GMT-0400 (Eastern Daylight Time)
fix permission query (#1263)
### Feature or Bugfix
- Bugfix
### Detail
- The filter -- array of permissions' NAMES, so in order to query
policies correctly we need to add join
- The filter 'share_type' and 'share_item_status' must be string
- IMPORTANT: in block "finally" the param session was used, but session
was defined only in "try" block. So, the lock failed to be released.
### Relates
- <URL or Ticket>
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 479b8f3f
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Wed May 08 2024 10:29:36 GMT-0400 (Eastern Daylight Time)
Add encryption and tag immutability to ECR repository (#1224)
### Feature or Bugfix
- Bugfix
### Detail
- Currently the ecr repository created do not have encryption and tag
immutability enabled which is identified by checkov scans. This fix is
to enable both.
### Relates
[- <URL or Ticket>](https://github.com/data-dot-all/dataall/issues/1200)
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
N/A
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? N/A
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? N/A
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
No. This is with default encryption
- Are you introducing any new policies/roles/users? N/A
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 2f885773
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Wed May 08 2024 09:22:40 GMT-0400 (Eastern Daylight Time)
Multiple permission roots (#1259)
### Feature or Bugfix
- Bugfix
### Detail
- GET_DATASET_TABLE (FOLDER) permissions are granted to the group only
if they are not granted already
- these permissions are removed if group is not admin|steward and there
are no other shares of this item.
### Relates
- #1174
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit c4cc07ee
Author: Petros Kalos <petrkalos@gmail.com>
Date: Wed May 08 2024 08:54:02 GMT-0400 (Eastern Daylight Time)
explicitly specify dataset_client s3 endpoint_url (#1260)
* AWS requires that the endpoint_url should be explicitly specified for
some regions
* Remove misleading CORS error message, the upload step can fail for
many reason
### Feature or Bugfix
- Bugfix
### Detail
Resolves #778
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 40defe8e
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue May 07 2024 11:52:17 GMT-0400 (Eastern Daylight Time)
Generic dataset module and specific s3_datasets module - part 1 (Rename datasets as s3_datasets) (#1250)
### Feature or Bugfix
- Refactoring
### Detail
- Rename `datasets` module to `s3_datasets` module
This PR is the first step to extract a generic datasets_base module that
implements the undifferentiated concepts of Dataset in data.all.
s3_datasets will use this base module to implement the specific
implementation for S3 datatasets.
### Relates
- #1123
- #955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 74a303cb
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:26:09 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests (#1253)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 2f33320c
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:25:03 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /backend/dataall/base/cdkproxy (#1252)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 0b49633f
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date: Tue May 07 2024 02:24:34 GMT-0400 (Eastern Daylight Time)
Bump werkzeug from 3.0.1 to 3.0.3 in /tests_new/integration_tests (#1254)
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 3.0.1 to
3.0.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
<li>Better TLS cert format with <code>adhoc</code> dev certs. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2891">#2891</a></li>
<li>Inform Python < 3.12 how to handle <code>itms-services</code>
URIs correctly, rather than using an overly-broad workaround in Werkzeug
that caused some redirect URIs to be passed on without encoding. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2828">#2828</a></li>
<li>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is <code>Any</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2836">#2836</a></li>
</ul>
<h2>3.0.2</h2>
<p>This is a fix release for the 3.0.x feature branch.</p>
<ul>
<li>Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-2</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>
<p>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger
UI makes requests using the full URL rather than only the path.
:ghsa:<code>2g68-c3qc-8985</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
<li>
<p>Better TLS cert format with <code>adhoc</code> dev certs.
:pr:<code>2891</code></p>
</li>
<li>
<p>Inform Python < 3.12 how to handle <code>itms-services</code> URIs
correctly, rather
than using an overly-broad workaround in Werkzeug that caused some
redirect
URIs to be passed on without encoding. :issue:<code>2828</code></p>
</li>
<li>
<p>Type annotation for <code>Rule.endpoint</code> and other uses of
<code>endpoint</code> is
<code>Any</code>. :issue:<code>2836</code></p>
</li>
<li>
<p>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. :pr:<code>2823</code></p>
</li>
</ul>
<h2>Version 3.0.2</h2>
<p>Released 2024-04-01</p>
<ul>
<li>Ensure setting <code>merge_slashes</code> to <code>False</code>
results in <code>NotFound</code> for
repeated-slash requests against single slash routes.
:issue:<code>2834</code></li>
<li>Fix handling of <code>TypeError</code> in
<code>TypeConversionDict.get()</code> to match
<code>ValueError</code>. :issue:<code>2843</code></li>
<li>Fix <code>response_wrapper</code> type check in test client.
:issue:<code>2831</code></li>
<li>Make the return type of <code>MultiPartParser.parse</code> more
precise.
:issue:<code>2840</code></li>
<li>Raise an error if converter arguments cannot be parsed.
:issue:<code>2822</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pallets/werkzeug/commit/f9995e967979eb694d6b31536cc65314fd7e9c8c"><code>f9995e9</code></a>
release version 3.0.3</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/3386395b24c7371db11a5b8eaac0c91da5362692"><code>3386395</code></a>
Merge pull request from GHSA-2g68-c3qc-8985</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/890b6b62634fa61224222aee31081c61b054ff01"><code>890b6b6</code></a>
only require trusted host for evalex</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/71b69dfb7df3d912e66bab87fbb1f21f83504967"><code>71b69df</code></a>
restrict debugger trusted hosts</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/d2d3869525a4ffb2c41dfb2c0e39d94dab2d870c"><code>d2d3869</code></a>
endpoint type is Any (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2895">#2895</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7080b55acd48b68afdda65ee6c7f99e9afafb0ba"><code>7080b55</code></a>
endpoint type is Any</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/7555eff296fbdf12f2e576b6bbb0b506df8417ed"><code>7555eff</code></a>
remove iri_to_uri redirect workaround (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2894">#2894</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/97fb2f722297ae4e12e36dab024e0acf8477b3c8"><code>97fb2f7</code></a>
remove _invalid_iri_to_uri workaround</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/249527ff981e7aa22cd714825c5637cc92df7761"><code>249527f</code></a>
make cn field a valid single hostname, and use wildcard in SANs field.
(<a
href="https://redirect.github.com/pallets/werkzeug/issues/2892">#2892</a>)</li>
<li><a
href="https://github.com/pallets/werkzeug/commit/793be472c9d145eb9be7d4200672d1806289d84a"><code>793be47</code></a>
update adhoc tls dev cert format</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/3.0.1...3.0.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/data-dot-all/dataall/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
commit 08862420
Author: mourya-33 <134511711+mourya-33@users.noreply.github.com>
Date: Tue May 07 2024 02:15:15 GMT-0400 (Eastern Daylight Time)
Updated lambda_api.py to add encryption for lambda env vars for custo… (#1255)
Feature or Bugfix
Bugfix
Detail
The environment variables for the lambda functions are not encrypted in
cdk which are identified by checkov scans. This fix is to enable kms
encryption for the lambda environment variables.
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based on
[OWASP 10](https://owasp.org/Top10/en/).
Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? N/A
Is the input sanitized? N/A
What precautions are you taking before deserializing the data you
consume? N/A
Is injection prevented by parametrizing queries? N/A
Have you ensured no eval or similar functions are used? N/A
Does this PR introduce any functionality or component that requires
authorization? N/A
How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
Are you logging failed auth attempts? N/A
Are you using or adding any cryptographic features? N/A
Do you use a standard proven implementations? N/A
Are the used keys controlled by the customer? Where are they stored? the
KMS keys are generated by cdk and are used to encrypt the environment
variables for all lambda functions in the lambda-api stack
Are you introducing any new policies/roles/users? - N/A
Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit ed7cc3eb
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Mon May 06 2024 09:32:30 GMT-0400 (Eastern Daylight Time)
Add order_by for paginated queries (#1249)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- This PR aims to solve the following
- (1) for particular queries (identified as ones that perform
`.outerjoin()` operations and have results paginated with `paginate()`
function - sometimes the returned query results is *less than* the limit
set by the pageSize of the paginate function even when the total count
is greater than the pageSize
- Ex 1: 11 envs total, `query_user_environments()` returning 9 envs on
1st page + 2 on 2nd page
- Ex 2: 10 envs total, `query_user_environments()` returning 9 envs on
1st page + no 2nd page
- Believe this is to be happening due to the way SQLAlchemy is
"uniquing" the records resulted from an outerjoin and then returning
that result back to the frontend
- Adding a `.distinct()` check on the query ensures each distinct record
is returned (tested successfully)
- (2) Currently we often times do not implement an `.order_by()`
condition for the query used in `paginate()` and do not have a stable
way of preserving order of the items returned from a query (i.e. when
navigating through pages of response)
- A generally good practice seems to include an `order_by()` on a column
or set of columns
- For each query used in `paginate()` this PR adds an `order_by()`
condition (full list in comments below)
Can read a bit more context from related issue linked below
### Relates
- https://github.com/data-dot-all/dataall/issues/1241
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 98e67fa8
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 12:21:57 GMT-0400 (Eastern Daylight Time)
fix: DATASET_READ_TABLE read permissions (#1237)
### Feature or Bugfix
- Bugfix
### Detail
- backfill DATASET_READ_TABLE permissions
- delete this permissions, when dataset tables are revoked or deteled
-
### Relates
- #1173
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 18e2f509
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 10:14:52 GMT-0400 (Eastern Daylight Time)
Fix local test groups listing for listGroups query (#1239)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Locally when trying to invite a team to Env or Org we call listGroups
and the returned `LOCAL_TEST_GROUPS` is not returning the proper data
type expected
### Relates
N/A
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit a0be03c4
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 10:12:34 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-5 FINAL DELETE DATASETS_BASE (#1242)
### Feature or Bugfix
- Refactoring
### Detail
After all the previous PRs are merged, there should be no circular
dependencies between `datasets` and `datasets_sharing`. We can now
proceed to:
- move `datasets_base` models, repositories, permissions and enums to
`datasets`
- adjust the `__init__` files to establish the `datasets_sharing`
depends on `datasets`
- adjust the Module interfaces to ensure that all necessary dataset
models... are imported in the interface for sharing
Next steps:
- share_notifications paramter to dataset_sharing in config.json
### Relates
#955
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit b68b40c1
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Fri May 03 2024 10:12:11 GMT-0400 (Eastern Daylight Time)
bugfix: EnvironmentGroup can remove other groups (#1234)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Now, if the group can't update other group, it also can not remove
them.
-
### Relates
- #1212
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 264539b5
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Fri May 03 2024 05:23:11 GMT-0400 (Eastern Daylight Time)
Fix Alembic Migration: has table checks (#1240)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- Fix `has_table()` check to ensure dropping the tables if the exists as
part of alembic migration upgrade
- Fix `DatasetLock nullable=True`
### Relates
- https://github.com/data-dot-all/dataall/issues/1165
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)? No
- Is the input sanitized? N/A
- What precautions are you taking before deserializing the data you
consume? N/A
- Is injection prevented by parametrizing queries? N/A
- Have you ensured no `eval` or similar functions are used? N/A
- Does this PR introduce any functionality or component that requires
authorization? No
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
N/A
- Are you logging failed auth attempts? N/A
- Are you using or adding any cryptographic features? No
- Do you use a standard proven implementations? N/A
- Are the used keys controlled by the customer? Where are they stored?
N/A
- Are you introducing any new policies/roles/users? No
- Have you used the least-privilege principle? How? N/A
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 42a5f6bd
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Fri May 03 2024 02:24:09 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-4 (#1214)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1213
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Use interface to resolve dataset roles related to datasets shared
and implement logic in the dataset_sharing module
- [X] Extend and clean-up stewards share permissions through interface
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 6d3f2d45
Author: Sofia Sazonova <sofia-s@304.ru>
Date: Thu May 02 2024 10:55:00 GMT-0400 (Eastern Daylight Time)
[After 2.4]Core Refactoring part5 (#1194)
### Feature or Bugfix
- Refactoring
### Detail
- focus on core/environments
- move logic from resolvers to services
- create s3_client in base/aws --> TO BE REFACTORED. Needs to be merged
with dataset_sharind/aws/s3_client
### Relates
- #741
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
---------
Co-authored-by: Sofia Sazonova <sazonova@amazon.co.uk>
commit 2ea24cbb
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Thu May 02 2024 08:22:12 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-3 (#1213)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1187
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- [X] Creates an interface to execute checks and clean-ups of data
sharing objects when dataset objects are deleted (initially it was going
to be an db interface, but I think it is better in the service)
- [X] Move listDatasetShares query to dataset_sharing module in
https://github.com/data-dot-all/dataall/pull/1185
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 750a5ec8
Author: Anushka Singh <anushka.singh.2511@gmail.com>
Date: Wed May 01 2024 12:28:18 GMT-0400 (Eastern Daylight Time)
Feature:1221 - Make visibility of auto-approval toggle configurable based on confidentiality (#1223)
### Feature or Bugfix
- Feature
### Detail
- Users should be able to disable visibility of auto-approval toggle
with code. For example, at our company, we require that shares always go
through approval process if their confidentiality classification is
Secret. We dont even want to give the option to users to be able to set
autoApproval enabled to ensure they dont do so by mistake and end up
over sharing.
Video demo:
https://github.com/data-dot-all/dataall/issues/1221#issuecomment-2077412044
### Relates
- https://github.com/data-dot-all/dataall/issues/1221
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 82044689
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Wed May 01 2024 12:26:42 GMT-0400 (Eastern Daylight Time)
Refactor: uncouple datasets and dataset_sharing modules - part 2-2 (#1187)
### Feature or Bugfix
- Refactoring
⚠️ MERGE AFTER https://github.com/data-dot-all/dataall/pull/1185
### Detail
This is needed as explained in full PR [AFTER 2.4] Refactor: uncouple
datasets and dataset_sharing modules #1179
- Split the getDatasetAssumeRole API into 2 APIs, one for dataset owners
role (in datasets module) and another one for share requester roles (in
datasets_sharing module)
### Relates
- #1179
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 5173419f
Author: Noah Paige <69586985+noah-paige@users.noreply.github.com>
Date: Wed May 01 2024 12:24:42 GMT-0400 (Eastern Daylight Time)
Fix so listValidEnvironments called only once (#1238)
### Feature or Bugfix
<!-- please choose -->
- Bugfix
### Detail
- When request access to a share on data.all the query to
`listValidEnvironments` used to be called twice which (depending on how
long for query results to return) could cause the environment initially
selected to disappear
### Relates
- Continuation of https://github.com/data-dot-all/dataall/issues/916
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage outside the application (e.g. a database, an
S3 bucket)?
- Is the input sanitized?
- What precautions are you taking before deserializing the data you
consume?
- Is injection prevented by parametrizing queries?
- Have you ensured no `eval` or similar functions are used?
- Does this PR introduce any functionality or component that requires
authorization?
- How have you ensured it respects the existing AuthN/AuthZ mechanisms?
- Are you logging failed auth attempts?
- Are you using or adding any cryptographic features?
- Do you use a standard proven implementations?
- Are the used keys controlled by the customer? Where are they stored?
- Are you introducing any new policies/roles/users?
- Have you used the least-privilege principle? How?
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
commit 7656ea86
Author: dlpzx <71252798+dlpzx@users.noreply.github.com>
Date: Tue Apr 30 2024 07:13:01 GMT-0400 (Eastern Daylight Time)
Add integration tests on a real API client and integrate the tests in CICD (#1219)
### Feature or Bugfix
- Feature
### Detail
Add integration tests that use a real Client to execute different
validation actions.
- Define the Client and the way API calls are posted to API Gateway in
the conftest
- Define the Cognito users and the different fixtures needed for all
tests
- Write tests for the Organization core module as example
- Add feature flag in `cdk.json` called `with_approval_tests` that can
be defined at the deployment environment level. If set to True, a
CodeBuild stage running the tests is created.
### Relates
- https://github.com/data-dot-all/dataall/issues/1220
### Security
Please answer the questions below briefly where applicable, or write
`N/A`. Based on
[OWASP 10](https://owasp.org/Top10/en/).
- Does this PR introduce or modify any input fields or queries - this
includes
fetching data from storage …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feature or Bugfix
Detail
Relates
Security
Please answer the questions below briefly where applicable, or write
N/A. Based onOWASP 10.
fetching data from storage outside the application (e.g. a database, an S3 bucket)?
evalor similar functions are used?By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.