Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Upgrade Bigtable to V2 #7317

Merged
merged 38 commits into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b32fb19
Added changes to BigtableClient
saranshdhingra May 9, 2024
152702b
feat(Bigtable): Replaced V1 GAPIC client surface with the V2 GAPIC su…
saranshdhingra May 13, 2024
deb138d
Modified tests for ChunkFormatter
saranshdhingra May 15, 2024
6aabbf4
Modified BigtableClientTest
saranshdhingra May 15, 2024
9d26162
Modified TableTest
saranshdhingra May 16, 2024
590d1a6
Modified SmartRetriesTest
saranshdhingra May 16, 2024
2009315
Modified Table resource class
saranshdhingra May 16, 2024
82254ee
Disable GAX retries in ResumableStream
saranshdhingra May 16, 2024
a91e895
Modified ChunkFormatter
saranshdhingra May 16, 2024
bde2f7f
Removed Todos
saranshdhingra May 16, 2024
6eacc25
Bypassed 'final' keyword for final classes in unit tests
saranshdhingra May 17, 2024
018f348
Fix snippet tests
saranshdhingra May 21, 2024
107e146
Changed the system tests to use the V2 GAPIC surface
saranshdhingra May 21, 2024
c527875
Removed deprecated classes
saranshdhingra May 21, 2024
69674d9
Modified retries param to retrySettings.maxRetries
saranshdhingra May 22, 2024
913c247
Fix snippet tests
saranshdhingra May 22, 2024
f2e83c0
Added a migrating guide
saranshdhingra May 22, 2024
ad0a412
Undo changes in V2/BigtableClient and mark it deprecated
saranshdhingra May 22, 2024
15714ac
Lint fixes
saranshdhingra May 22, 2024
cf3381b
Merge branch 'main' into bigtable-v2
saranshdhingra May 22, 2024
4d09967
Added 'final' class bypassing in Core's bootstrap file
saranshdhingra May 23, 2024
d643732
Added the bypass-finals package in root composer.json
saranshdhingra May 23, 2024
6198f82
Added 'final' class bypassing in Core's snippet-bootstrap file
saranshdhingra May 23, 2024
2df887b
Fix for the package tests
saranshdhingra May 23, 2024
476b811
Bump cloud-core version in composer.json
saranshdhingra May 23, 2024
24765e6
Change bootstrap file for snippet tests
saranshdhingra May 24, 2024
ae41042
Removed V1 GAPIC surface clients
saranshdhingra May 24, 2024
e4f33bd
Removed deprecated classes from the Admin namespace
saranshdhingra May 24, 2024
f4a052d
Removed V1 GAPIC tests
saranshdhingra May 24, 2024
837dfc6
Addressed PR comments
saranshdhingra May 24, 2024
e8e9715
Updated migrating guide
saranshdhingra May 24, 2024
2b1a0ca
chore(Bigtable): generator changes from owlbot for V2 (#7352)
bshaffer May 31, 2024
1baac98
Merge branch 'main' into bigtable-v2
bshaffer Jun 3, 2024
daa44f9
Update Bigtable/composer.json
bshaffer Jun 3, 2024
cb3108a
Update composer.json
bshaffer Jun 3, 2024
d9c4fd0
fix filters
bshaffer Jun 3, 2024
26e9c46
update bigtable versions to 2.0.0-RC1
bshaffer Jun 3, 2024
303b5ef
fix bigtable version in composer
bshaffer Jun 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions Bigtable/MIGRATING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Migrating Google Cloud Bigtable from V1 to V2

## How to upgrade

Update your `google/cloud-bigtable` dependency to `^2.0`:

```
{
"require": {
"google/cloud-bigtable": "^2.0"
}
}
```

## Changes

### Retry Options changes

The `retries` parameter in the following RPC calls is changed to `retrySettings.maxRetries`:
- Google\Cloud\Bigtable\Table::mutateRows
- Google\Cloud\Bigtable\Table::upsert
- Google\Cloud\Bigtable\Table::readRows

For example:
```php
$table->readRows([
'retrySettings' => [
'maxRetries' => 3
]
]);
```

OR

```php
$retrySettings = RetrySettings::constructDefault();
$retrySettings = $retrySettings->with(['maxRetries' => 3])
$table->readRows([
'retrySettings' => $retrySettings
]);
```

This is done in order to be consistent across other RPC calls which use GAX's retrySettings and to be consistent across other products which mostly use the [RetrySettings](https://github.com/googleapis/gax-php/blob/main/src/RetrySettings.php) from GAX already.

Only the maxRetries parameter is supported for now and not the other options in the `RetrySettings`.

### Removed deprecated classes

The following deprecated classes are now removed completely. Use the specified alternatives.
- `Google\Cloud\Bigtable\V2\Gapic\BigtableGapicClient`, use `Google\Cloud\Bigtable\V2\Client\BigtableClient` instead.
- `Google\Cloud\Bigtable\V2\BigtableClient`, use `Google\Cloud\Bigtable\V2\Client\BigtableClient` instead.
- `Google\Cloud\Bigtable\Admin\V2\Gapic\BigtableInstanceAdminGapicClient`, use `Google\Cloud\Bigtable\Admin\V2\Client\BigtableInstanceAdminClient` instead.
- `Google\Cloud\Bigtable\Admin\V2\Gapic\BigtableTableAdminGapicClient`, use `Google\Cloud\Bigtable\Admin\V2\Client\BigtableTableAdminClient` instead.
- `Google\Cloud\Bigtable\V2\MutateRowsRequest_Entry`, use `Google\Cloud\Bigtable\V2\MutateRowsRequest\Entry` instead.
- `Google\Cloud\Bigtable\V2\MutateRowsResponse_Entry`, use `Google\Cloud\Bigtable\V2\MutateRowsResponse\Entry` instead.
- `Google\Cloud\Bigtable\V2\Mutation_DeleteFromFamily`, use `Google\Cloud\Bigtable\V2\Mutation\DeleteFromFamily` instead.
- `Google\Cloud\Bigtable\V2\Mutation_DeleteFromColumn`, use `Google\Cloud\Bigtable\V2\Mutation\DeleteFromColumn` instead.
- `Google\Cloud\Bigtable\V2\Mutation_DeleteFromRow`, use `Google\Cloud\Bigtable\V2\Mutation\DeleteFromRow` instead.
- `Google\Cloud\Bigtable\V2\Mutation_SetCell`, use `Google\Cloud\Bigtable\V2\Mutation\SetCell` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_CloseStream`, use Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\CloseStream`` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_DataChange_Type`, use `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\DataChange\Type` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_DataChange`, use `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\DataChange` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_Heartbeat`, use `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\Heartbeat` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_MutationChunk_ChunkInfo`, use `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\MutationChunk\ChunkInfo` instead.
- `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse_MutationChunk`, use `Google\Cloud\Bigtable\V2\ReadChangeStreamResponse\MutationChunk` instead.
- `Google\Cloud\Bigtable\V2\ReadRowsRequest_RequestStatsView`, use `Google\Cloud\Bigtable\V2\ReadRowsRequest\RequestStatsView` instead.
- `Google\Cloud\Bigtable\V2\RowFilter_Chain`, use `Google\Cloud\Bigtable\V2\RowFilter\Chain` instead.
- `Google\Cloud\Bigtable\V2\RowFilter_Condition`, use `Google\Cloud\Bigtable\V2\RowFilter\Condition` instead.
- `Google\Cloud\Bigtable\V2\RowFilter_Interleave`, use `Google\Cloud\Bigtable\V2\RowFilter\Interleave` instead.
- `Google\Cloud\Bigtable\V2\ReadRowsResponse_CellChunk`, use `Google\Cloud\Bigtable\V2\ReadRowsResponse\CellChunk` instead.
saranshdhingra marked this conversation as resolved.
Show resolved Hide resolved
- `Google\Cloud\Bigtable\Admin\V2\AppProfile_MultiClusterRoutingUseAny`, use `Google\Cloud\Bigtable\Admin\V2\AppProfile\MultiClusterRoutingUseAny` instead.
- `Google\Cloud\Bigtable\Admin\V2\AppProfile_SingleClusterRouting`, use `Google\Cloud\Bigtable\Admin\V2\AppProfile\SingleClusterRouting` instead.
- `Google\Cloud\Bigtable\Admin\V2\Backup_State`, use `Google\Cloud\Bigtable\Admin\V2\Backup\State` instead.
- `Google\Cloud\Bigtable\Admin\V2\Cluster_ClusterAutoscalingConfig`, use `Google\Cloud\Bigtable\Admin\V2\Cluster\ClusterAutoscalingConfig` instead.
- `Google\Cloud\Bigtable\Admin\V2\Cluster_ClusterConfig`, use `Google\Cloud\Bigtable\Admin\V2\Cluster\ClusterConfig` instead.
- `Google\Cloud\Bigtable\Admin\V2\Cluster_EncryptionConfig`, use `Google\Cloud\Bigtable\Admin\V2\Cluster\EncryptionConfig` instead.
- `Google\Cloud\Bigtable\Admin\V2\Cluster_State`, use `Google\Cloud\Bigtable\Admin\V2\Cluster\State` instead.
- `Google\Cloud\Bigtable\Admin\V2\CreateClusterMetadata_TableProgress`, use `Google\Cloud\Bigtable\Admin\V2\CreateClusterMetadata\TableProgress` instead.
- `Google\Cloud\Bigtable\Admin\V2\CreateClusterMetadata_TableProgress_State`, use `Google\Cloud\Bigtable\Admin\V2\CreateClusterMetadata\TableProgress\State` instead.
- `Google\Cloud\Bigtable\Admin\V2\CreateTableRequest_Split`, use `Google\Cloud\Bigtable\Admin\V2\CreateTableRequest\Split` instead.
- `Google\Cloud\Bigtable\Admin\V2\EncryptionInfo_EncryptionType`, use `Google\Cloud\Bigtable\Admin\V2\EncryptionInfo\EncryptionType` instead.
- `Google\Cloud\Bigtable\Admin\V2\GcRule_Intersection`, use `Google\Cloud\Bigtable\Admin\V2\GcRule\Intersection` instead.
- `Google\Cloud\Bigtable\Admin\V2\GcRule_Union`, use `Google\Cloud\Bigtable\Admin\V2\GcRule\Union` instead.
- `Google\Cloud\Bigtable\Admin\V2\Instance_State`, use `Google\Cloud\Bigtable\Admin\V2\Instance\State` instead.
- `Google\Cloud\Bigtable\Admin\V2\Instance_Type`, use `Google\Cloud\Bigtable\Admin\V2\Instance\Type` instead.
- `Google\Cloud\Bigtable\Admin\V2\ModifyColumnFamiliesRequest_Modification`, use `Google\Cloud\Bigtable\Admin\V2\ModifyColumnFamiliesRequest\Modification` instead.
- `Google\Cloud\Bigtable\Admin\V2\Snapshot_State`, use `Google\Cloud\Bigtable\Admin\V2\Snapshot\State` instead.
- `Google\Cloud\Bigtable\Admin\V2\Table_ClusterState`, use `Google\Cloud\Bigtable\Admin\V2\Table\ClusterState` instead.
- `Google\Cloud\Bigtable\Admin\V2\Table_ClusterState_ReplicationState`, use `Google\Cloud\Bigtable\Admin\V2\Table\ClusterState\ReplicationState` instead.
- `Google\Cloud\Bigtable\Admin\V2\Table_TimestampGranularity`, use `Google\Cloud\Bigtable\Admin\V2\Table\TimestampGranularity` instead.
- `Google\Cloud\Bigtable\Admin\V2\Table_View`, use `Google\Cloud\Bigtable\Admin\V2\Table\View` instead.
2 changes: 2 additions & 0 deletions Bigtable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ foreach ($rows as $row) {
This component is considered GA (generally available). As such, it will not introduce backwards-incompatible changes in
any minor or patch releases. We will address issues and requests with the highest priority.

Please see the [migration guide](./MIGRATING.md) to upgrade from **V1** of the library to **V2**.

### Next Steps

Take a look at and understand the [official documentation](https://cloud.google.com/bigtable/docs).
2 changes: 1 addition & 1 deletion Bigtable/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.32.1
2.0.0-RC1
7 changes: 4 additions & 3 deletions Bigtable/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@
"minimum-stability": "stable",
"require": {
"php": "^8.0",
"google/gax": "^1.34.0",
"google/cloud-core": "^1.52.7"
"google/gax": "^1.34",
"google/cloud-core": "^1.55"
},
"require-dev": {
"phpunit/phpunit": "^9.0",
"erusev/parsedown": "^1.6",
"phpdocumentor/reflection": "^5.3.3",
"phpdocumentor/reflection-docblock": "^5.3",
"phpspec/prophecy-phpunit": "^2.0"
"phpspec/prophecy-phpunit": "^2.0",
"dg/bypass-finals": "^1.7"
},
"suggest": {
"ext-protobuf": "Provides a significant increase in throughput over the pure PHP protobuf implementation. See https://cloud.google.com/php/grpc for installation instructions."
Expand Down
80 changes: 5 additions & 75 deletions Bigtable/owlbot.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,88 +38,18 @@
src=src,
dest=dest,
copy_excludes=[
src / "*/src/V2/BigtableClient.php"
src / "**/[A-Z]*_*.php",
]
)

# First copy the Bigtable Admin
admin_library = Path(f"../{php.STAGING_DIR}/Bigtable/v2/Admin").resolve()

# copy all src except handwritten partial veneers
s.move(admin_library / f'src/V2/Gapic', 'src/Admin/V2/Gapic', merge=preserve_copyright_year)
s.move(admin_library / f'src/V2/Client', 'src/Admin/V2/Client', merge=preserve_copyright_year)
s.move(admin_library / f'src/V2/resources', f'src/Admin/V2/resources', merge=preserve_copyright_year)

# copy proto files to src also
s.move(admin_library / f'proto/src/Google/Cloud/Bigtable', f'src/', merge=preserve_copyright_year)
# copy gapic src and tests
s.move(admin_library / f'src', 'src/Admin', merge=preserve_copyright_year)
s.move(admin_library / f'tests/Unit', 'tests/Unit/Admin', merge=preserve_copyright_year)

# copy GPBMetadata file to metadata
# copy proto and metadata files
s.move(admin_library / f'proto/src/Google/Cloud/Bigtable', f'src/', merge=preserve_copyright_year)
s.move(admin_library / f'proto/src/GPBMetadata/Google/Bigtable', f'metadata/', merge=preserve_copyright_year)


# fix unit test namespace
s.replace(
'tests/Unit/Admin/*/*.php',
r'namespace Google\\Cloud\\Bigtable\\Admin\\Tests\\Unit',
r'namespace Google\\Cloud\\Bigtable\\Tests\\Unit\\Admin')

# fix test group
s.replace(
'tests/**/Admin/**/*Test.php',
'@group admin',
'@group bigtable' + '\n' + ' * bigtable-admin')

# Fix class references in gapic samples
for version in ['V2']:
pathExpr = [
'src/' + version + '/Gapic/BigtableGapicClient.php',
'src/Admin/' + version + '/Gapic/*GapicClient.php'
]

types = {
'new BigtableClient': r'new Google\\Cloud\\Bigtable\\' + version + r'\\BigtableClient',
'new BigtableInstanceAdminClient': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\BigtableInstanceAdminClient',
r'\$instance = new Instance': r'$instance = new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Instance',
'= Type::': r'= Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Instance\\Type::',
'new FieldMask': r'new Google\\Protobuf\\FieldMask',
r'\$cluster = new Cluster': r'$cluster = new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Cluster',
'new AppProfile': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\AppProfile',
'new Policy': r'new Google\\Cloud\\Iam\\V1\\Policy',
'new BigtableTableAdminClient': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\BigtableTableAdminClient',
'new Table': r'new Google\\Cloud\\Bigtable\\Admin\\' + version + r'\\Table',
}

for search, replace in types.items():
s.replace(
pathExpr,
search,
replace
)

### [START] protoc backwards compatibility fixes

# roll back to private properties.
s.replace(
"src/**/V*/**/*.php",
r"Generated from protobuf field ([^\n]{0,})\n\s{5}\*/\n\s{4}protected \$",
r"""Generated from protobuf field \1
*/
private $""")

# Replace "Unwrapped" with "Value" for method names.
s.replace(
"src/**/V*/**/*.php",
r"public function ([s|g]\w{3,})Unwrapped",
r"public function \1Value"
)

### [END] protoc backwards compatibility fixes

# fix relative cloud.google.com links
s.replace(
"src/**/V*/**/*.php",
r"(.{0,})\]\((/.{0,})\)",
r"\1](https://cloud.google.com\2)"
)

2 changes: 1 addition & 1 deletion Bigtable/phpunit-snippets.xml.dist
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" bootstrap="./vendor/google/cloud-core/snippet-bootstrap.php" colors="true" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" bootstrap="../Core/snippet-bootstrap.php" colors="true" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
<coverage>
<include>
<directory suffix=".php">src</directory>
Expand Down
2 changes: 1 addition & 1 deletion Bigtable/phpunit.xml.dist
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" colors="true" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" bootstrap="./tests/Unit/bootstrap.php" colors="true" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
<coverage>
<include>
<directory suffix=".php">src</directory>
Expand Down
6 changes: 3 additions & 3 deletions Bigtable/src/Admin/V2/AppProfile.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Bigtable/src/Admin/V2/AppProfile/SingleClusterRouting.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Bigtable/src/Admin/V2/AppProfile/StandardIsolation.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 0 additions & 16 deletions Bigtable/src/Admin/V2/AppProfile_MultiClusterRoutingUseAny.php

This file was deleted.

16 changes: 0 additions & 16 deletions Bigtable/src/Admin/V2/AppProfile_SingleClusterRouting.php

This file was deleted.

6 changes: 3 additions & 3 deletions Bigtable/src/Admin/V2/AuthorizedView.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Bigtable/src/Admin/V2/AutoscalingLimits.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Bigtable/src/Admin/V2/AutoscalingTargets.php

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading