Continue localstack test harness development #111

sjperkins · 2023-09-05T07:40:51Z

Follows on from these comments in #91

laramiel · 2023-09-05T07:53:13Z

Heh. I just submitted a few updates which conflict with this--it separates out the localstack tests from the other tests, so your changes will need to target the new file.

laramiel · 2023-09-05T07:54:41Z

tensorstore/kvstore/s3/s3_key_value_store_test.cc

@@ -59,15 +59,15 @@ using ::tensorstore::internal_http::HttpResponse;
 using ::tensorstore::internal_kvstore_s3::S3Credentials;
 using ::tensorstore::internal_kvstore_s3::S3RequestBuilder;

-ABSL_FLAG(std::string, localstack_binary, "", "Path to the localstack");
+ABSL_FLAG(std::string, localstack_binary, "", "Path to the localstack binary");
+ABSL_FLAG(std::string, localstack_endpoint, "http://localhost:4566", "Localstack endpoint");


Is there a command line flag in localstack to set the port, or does it require a config file?

Looks like its configurable via environment variables

https://docs.localstack.cloud/references/configuration/

GATEWAY_LISTEN

LOCALSTACK_HOST

I guess its possible to try PickUnusedPortOrDie as discussed here: #91 (comment)

Yes, if it's unset (and the env var is unset), that would be nice.

sjperkins · 2023-09-05T08:14:27Z

As mentioned in #91 (comment), this test

tensorstore/tensorstore/kvstore/test_util.cc

Lines 442 to 450 in 78b93c9

    
           ABSL_LOG(INFO) << kSep << "Test unconditional range read [1 .. 1], size 0"; 
        
           { 
        
             kvstore::ReadOptions options; 
        
             options.byte_range.inclusive_min = 1; 
        
             options.byte_range.exclusive_max = 1; 
        
             EXPECT_THAT( 
        
                 kvstore::Read(store, key, options).result(), 
        
                 MatchesKvsReadResult(absl::Cord(), read_result->stamp.generation)); 
        
           }

fails as follows:

I0000 00:00:1693900057.213783      80 s3_key_value_store.cc:515] ReadTask: HttpRequest{GET http://localhost:4566/tensorstore/test/key_read user_agent=, Range: bytes=1-0, host: testbucket.s3.af-south-1.localstack.localhost.com, x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, x-amz-date: 20230905T074737Z, Authorization: AWS4-HMAC-SHA256 Credential=LSIAQAAAAAAVNCBMPNSG/20230905/af-south-1/s3/aws4_request,SignedHeaders=host;range;x-amz-content-sha256;x-amz-date,Signature=777d4530e2970f7e0583f366d3f3212a31efda45bbb9cb0e0d69b1e313a5620a}
I0000 00:00:1693900057.263308      79 s3_key_value_store.cc:528] ReadTask HttpResponse{code=416, headers=<content-type=application/xml, date=Tue, 05 Sep 2023 07:47:37 GMT, server=hypercorn-h2, x-amz-id-2=s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=, x-amz-request-id=ca499343-df93-4a3c-9756-19d5b83aae76>, body=<?xml version='1.0' encoding='utf-8'?>
<Error><Code>InvalidRange</Code><Message>The requested range is not satisfiable</Message><RequestId>ca499343-df93-4a3c-9756-19d5b83aae76</RequestId><ActualObjectSize>15</ActualObjectSize><RangeRequested>bytes=1-0</RangeRequested></Error>}
tensorstore/kvstore/test_util.cc:447: Failure
Value of: kvstore::Read(store, key, options).result()
Expected: value (is an object whose field `state` is equal to <value>) and (is an object whose field `value` is equal to ) and (is an object whose field `stamp` value (is an object whose field `generation` is equal to "\"6f314ce98d753057568807b0863b3bf0\"\x01") and (is an object whose field `time` is anything))
  Actual: OUT_OF_RANGE: HTTP response code: 416 with body: <?xml version='1.0' encoding='utf-8'?>
<Error><Code>InvalidRange</Code><Message>The requested range is not satisfiable</Message><RequestId>ca499343-df93-4a3c-9756-19d5b83aae76</RequestId><ActualObjectSize>15</ActualObjectSize><RangeRequested>bytes=1-0</RangeRequested></Error> [source locations='tensorstore/kvstore/s3/s3_key_value_store.cc:540'], which is not engaged
Google Test trace:
tensorstore/kvstore/test_util.cc:400: TestKeyValueStoreReadOps
Stack trace:
  0x7eff6074dd34: tensorstore::internal::TestKeyValueStoreReadOps()
  0x7eff60751d82: tensorstore::internal::TestKeyValueReadWriteOps()
  0x7eff60751933: tensorstore::internal::TestKeyValueReadWriteOps()
  0x562b45ea4f8f: (anonymous namespace)::LocalStackFixture_Basic_Test::TestBody()
  0x7eff5f519fd1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
  0x7eff5f513e7b: testing::internal::HandleExceptionsInMethodIfSupported<>()
  0x7eff5f4f5ad4: testing::Test::Run()
  0x7eff5f4f6562: testing::TestInfo::Run()
... Google Test internal frames ...

I think this is due to Range: bytes=1-0 producing a 416 due to the following logic in localstack:

https://github.com/localstack/localstack/blob/279aea173dc56ab58fcae7f25d5a0e515c3aff6d/localstack/services/s3/utils.py#L217

Its not clear to me whether Range: bytes=1-0 is an invalid range. Common sense suggests that it is but:

This test succeeds on AWS (and presumably on GCS) so maybe these services are handling this case gracefully to produce the 206.
I guess a flexible reading https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range#directives doesn't discount end < start.

I'll maybe take a look at how nginx handles this, but this will probably require a localstack change to align it's behaviour with s3/gcs. Do you know of any compelling arguments here?

sjperkins · 2023-09-05T08:27:46Z

Setting options.byte_range.exclusive_max = 2; here:

tensorstore/tensorstore/kvstore/test_util.cc

Lines 442 to 450 in 78b93c9

    
           ABSL_LOG(INFO) << kSep << "Test unconditional range read [1 .. 1], size 0"; 
        
           { 
        
             kvstore::ReadOptions options; 
        
             options.byte_range.inclusive_min = 1; 
        
             options.byte_range.exclusive_max = 1; 
        
             EXPECT_THAT( 
        
                 kvstore::Read(store, key, options).result(), 
        
                 MatchesKvsReadResult(absl::Cord(), read_result->stamp.generation)); 
        
           }

to produce a Range: bytes=1-1 produces the 206 but still fails the assert:

I0000 00:00:1693902191.704033      81 s3_key_value_store.cc:515] ReadTask: HttpRequest{GET http://localhost:4566/tensorstore/test/key_read user_agent=, Range: bytes=1-1, host: testbucket.s3.af-south-1.localstack.localhost.com, x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, x-amz-date: 20230905T082311Z, Authorization: AWS4-HMAC-SHA256 Credential=LSIAQAAAAAAVNCBMPNSG/20230905/af-south-1/s3/aws4_request,SignedHeaders=host;range;x-amz-content-sha256;x-amz-date,Signature=535c37221d7c17ea32453c98a87f896bef2e0518f55f141c312df08f8d7ad31a}
I0000 00:00:1693902191.763336      80 s3_key_value_store.cc:528] ReadTask HttpResponse{code=206, headers=<accept-ranges=bytes, content-range=bytes 1-1/15, content-type=application/octet-stream, date=Tue, 05 Sep 2023 08:23:11 GMT, etag="6f314ce98d753057568807b0863b3bf0", last-modified=Tue, 05 Sep 2023 08:23:11 GMT, server=hypercorn-h2, x-amz-id-2=s9lzHYrFp76ZVxRcpX9+5cjAnEH2ROuNkd2BHfIa6UkFVdtjf5mKR3/eTPFvsiP/XV/VLi31234=, x-amz-request-id=33042dbb-1e20-44e1-ae8b-d2f7092d2482>, body=k}
tensorstore/kvstore/test_util.cc:447: Failure
Value of: kvstore::Read(store, key, options).result()
Expected: value (is an object whose field `state` is equal to <value>) and (is an object whose field `value` is equal to ) and (is an object whose field `stamp` value (is an object whose field `generation` is equal to "\"6f314ce98d753057568807b0863b3bf0\"\x01") and (is an object whose field `time` is anything))
  Actual: Result{
{value="k", stamp={generation="\"6f314ce98d753057568807b0863b3bf0\"\x01", time=2023-09-05T08:23:11.703827803+00:00}}},
whose value
{value="k", stamp={generation="\"6f314ce98d753057568807b0863b3bf0\"\x01", time=2023-09-05T08:23:11.703827803+00:00}} doesn't match,
whose field `value` is k (of type absl::Cord)
Google Test trace:
tensorstore/kvstore/test_util.cc:400: TestKeyValueStoreReadOps
Stack trace:
  0x7f9e3bbc1d34: tensorstore::internal::TestKeyValueStoreReadOps()
  0x7f9e3bbc5d82: tensorstore::internal::TestKeyValueReadWriteOps()
  0x7f9e3bbc5933: tensorstore::internal::TestKeyValueReadWriteOps()
  0x563c60abcf8f: (anonymous namespace)::LocalStackFixture_Basic_Test::TestBody()
  0x7f9e3a98dfd1: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
  0x7f9e3a987e7b: testing::internal::HandleExceptionsInMethodIfSupported<>()
  0x7f9e3a969ad4: testing::Test::Run()
  0x7f9e3a96a562: testing::TestInfo::Run()
... Google Test internal frames ...

sjperkins · 2023-09-05T07:43:39Z

tensorstore/kvstore/s3/BUILD

@@ -173,12 +173,16 @@ tensorstore_cc_test(
    name = "s3_key_value_store_test",
    size = "small",
    srcs = ["s3_key_value_store_test.cc"],
+    # args = ["--localstack_binary=$(location @pypa_localstack//:localstack)"],


I think this is the idea, but I don't know enough bazel to reference the localstack script install in the python environment's bin directory. Perhaps I need to declare py_binary that depends on @pypa_localstack?

To do that you'd need a workspace for localstack with a py_binary, similar to what we have for storagebench:

https://github.com/google/tensorstore/blob/master/third_party/com_google_storagetestbench/workspace.bzl

Then reference it like here, similar to what the gcs_grpc_testbench_test:

tensorstore/tensorstore/kvstore/gcs_grpc/BUILD

Line 129 in 97bd317

args = ["--testbench_binary=$(location @com_google_storagetestbench//:rest_server)"],

I'd call the directory com_github_localstack or perhaps cloud_localstack.

sjperkins · 2023-09-05T08:34:56Z

tensorstore/kvstore/s3/s3_key_value_store_test.cc

@@ -59,15 +59,15 @@ using ::tensorstore::internal_http::HttpResponse;
 using ::tensorstore::internal_kvstore_s3::S3Credentials;
 using ::tensorstore::internal_kvstore_s3::S3RequestBuilder;

-ABSL_FLAG(std::string, localstack_binary, "", "Path to the localstack");
+ABSL_FLAG(std::string, localstack_binary, "", "Path to the localstack binary");
+ABSL_FLAG(std::string, localstack_endpoint, "http://localhost:4566", "Localstack endpoint");


Looks like its configurable via environment variables

https://docs.localstack.cloud/references/configuration/

GATEWAY_LISTEN

LOCALSTACK_HOST

I guess its possible to try PickUnusedPortOrDie as discussed here: #91 (comment)

sjperkins · 2023-09-05T09:11:46Z

Heh. I just submitted a few updates which conflict with this--it separates out the localstack tests from the other tests, so your changes will need to target the new file.

:-) We should probably coordinate more -- apologies I was out on PTO mid-August. Thanks for merging #91 and adding the docs.

I can devote a day or so a week to working on tensorstore so the velocity is not as high as it could be. I'd imagine the context switches aren't ideal for either of us. If it's desirable to develop the remaining S3 functionality more speedily, I'd be fine with stepping back and letting others take the helm. Alternatively, I can continue at the current pace, aiming to get the localstack testharness into good shape. Then, the remaining work should be easier for others to deal with piecemeal in smaller PRs.

Also, In terms of communication, I think it would be valuable if I converted the TODO in #91 into an umbrella issue detailing the remaining S3 work.

laramiel · 2023-09-05T17:21:14Z

tensorstore/kvstore/s3/validate_test.cc

@@ -33,14 +33,17 @@ TEST(ValidateTest, ClassifyBucketName) {
  EXPECT_EQ(ClassifyBucketName("sthree-configurator.bucket"),
            BucketNameType::kInvalid);

-  // Standard bucket names are < 63 characters
+    // Standard bucket names are < 63 characters


Split this test change into a separate pull request, please.

laramiel · 2023-09-05T17:36:51Z

There are a few cleanups that I was going to do, mostly related to using Context for AWS auth, and perhaps a very minimal mock based test (similar to the gcs driver). But since it's submitted, the s3 kvstore will also be included in our continuous build and will see updates that affect it in that way, which might include updates which affect all of our drivers.

But other than that, I don't have immediate plans to add features and am happy let you handle those improvements. Hopefully we won't step on each other's toes too much.

laramiel · 2023-09-05T17:40:35Z

w.r.t the range issue, I had to fix a few of those in the gcs testbench, it seems very likely that range requests are not well tested in localstack as well. If AWS respond correctly, it seems like a pull request for localstack is worthwhile.

You could see what happens running the localstack test against minio instead of localstack.

As for what's the right thing to do, according to this: https://www.rfc-editor.org/rfc/rfc9110.html#name-range-requests

bytes=0-0 should work (in fact, it's an example), and by extension, bytes=1-1, but we may want to return inclusive_min == exclusive_max as an invalid range.

laramiel · 2023-09-05T23:28:00Z

Just a note about localstack_test vs other possibilities.

With the addition of "s3" to tensorstore/kvstore/BUILD

Then the following will work to access an actual s3 bucket:

./bazelisk.py build --//tensorstore/kvstore/s3:debug //tensorstore/kvstore:live_kvstore_test

AWS_ACCESS_KEY_ID=XXX AWS_SECRET_ACCESS_KEY=... \
bazel-bin/tensorstore/kvstore/live_kvstore_test --kvstore_spec='{"driver": "s3", "bucket": "MYBUCKET"}'

With that, I see the following sequence for the basic test (some entries elided), where it appears to fail to write the initial value:

[aws_credential_provider.cc:103] Using Environment Variable AWS_ACCESS_KEY_ID
[s3_resource.cc:120] Using default AdmissionQueue with limit 32
[s3_key_value_store.cc:1341] S3 driver using endpoint [https://MYBUCKET.s3.us-east-1.amazonaws.com]

[s3_key_value_store.cc:950] DeleteTask: HttpRequest{
  DELETE https://MYBUCKET.s3.us-east-1.amazonaws.com/key_missing user_agent=, 
  host: https://MYBUCKET.s3.us-east-1.amazonaws.com, 
  x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, 
  x-amz-date: 20230905T232126Z,
  Authorization: AWS4-HMAC-SHA256 Credential=XXX/20230905/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=e2264829ce8b69ab7f85064ec1c76bf9604948e4355be6fb08a505aee8c10a71
}

[s3_key_value_store.cc:964] DeleteTask HttpResponse{
  code=400, 
  headers=<
    connection=close, 
    date=Tue, 05 Sep 2023 23:21:27 GMT, 
    server=AmazonS3, 
    transfer-encoding=chunked, 
    x-amz-id-2=SV//UJMSX7vUnGJmkPrpCrT9m2owstUqaaN1TXnqc3eGcO+WV/e4UXmoFA+3OUsAtn3xgTZ+5r8=, 
    x-amz-request-id=ZZKHV454NFJWJ540>, 
  body=
}

[s3_key_value_store.cc:768] WriteTask: HttpRequest{
     PUT https://MYBUCKET.s3.us-east-1.amazonaws.com/key_read
     user_agent=, 
     Content-Type: application/octet-stream, 
Content-Length: 15, 
host: https://MYBUCKET.s3.us-east-1.amazonaws.com, 
x-amz-content-sha256: b34aa317ec09ecaaf0c849b0cd2a0864ad707a7691e31140d044e7c04128a31a, 
x-amz-date: 20230905T232127Z, 
Authorization: AWS4-HMAC-SHA256 Credential=XXX/20230905/us-east-1/s3/aws4_request,SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date,Signature=f4971bb33f62861863f657e040cf91cd232e5817705dae340a71d3bdd2fc4615
} 
  size=15
}


[s3_key_value_store.cc:782] WriteTask HttpResponse{
  code=400, headers=<
 connection=close, 
  date=Tue, 05 Sep 2023 23:21:26 GMT, 
  server=AmazonS3, 
  transfer-encoding=chunked, 
  x-amz-id-2=ROMbRjGjYZT4fZfqy+mDnjB6O8PHn0IPDF1Lo0X9hIhD+aa4kizLJ6IAgp+51n3aof4HT92EYPE=, 
  x-amz-request-id=ZZKMNYWN5KPZP0DB>,
 body=
}

tensorstore/kvstore/test_util.cc:625: Failure
Value of: write_result
Expected: value (is an object whose field `generation` is regular storage generation) and (is an object whose field `time` is anything)
  Actual: INVALID_ARGUMENT: HTTP response code: 400 with body:  [source locations='tensorstore/kvstore/s3/s3_key_value_store.cc:787'], which is not engaged
Stack trace:
  0x1053ac618: tensorstore::internal::TestKeyValueReadWriteOps()
  0x1053ac22c: tensorstore::internal::TestKeyValueReadWriteOps()
  0x1049b25a4: (anonymous namespace)::LiveKvStoreTest_Basic_Test::TestBody()
  0x10680baec: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
  0x1067d8454: testing::internal::HandleExceptionsInMethodIfSupported<>()
  0x1067d83a4: testing::Test::Run()
  0x1067d93a4: testing::TestInfo::Run()
... Google Test internal frames ...

[  FAILED  ] LiveKvStoreTest.Basic (1098 ms)

... so I may have mixed up some of the endpoint and host stuff while getting this to be async. Stay tuned for a followup cl or 2.

sjperkins · 2023-09-06T07:36:08Z

As for what's the right thing to do, according to this: https://www.rfc-editor.org/rfc/rfc9110.html#name-range-requests

Thanks, I'll need to take the time to read this carefully.

bytes=0-0 should work (in fact, it's an example), and by extension, bytes=1-1, but we may want to return inclusive_min == exclusive_max as an invalid range.

I think the examples above are fine, but for the OptionalByteRangeRequest{1, 1} case, the following is transmitted in the header "Range: bytes=1-0". I contrasted the behaviour on AWS vs localstack here:

bug: S3 range requests where end < last return a 416, instead of returning the entire content. localstack/localstack#9076

Briefly, it looks like AWS behaviourly treats 1-0 as a request for the entire content.

Actually, skimming the RFC above, it looks like this might technically be invalid:

An int-range is invalid if the last-pos value is present and less than the first-pos.

Maybe a clamp is needed here for the inclusive_min=1, exclusive_max=1 case?

tensorstore/tensorstore/internal/http/http_request.cc

Lines 37 to 39 in c01efdc

    
           if (byte_range.IsRange()) { 
        
             return absl::StrFormat("Range: bytes=%d-%d", byte_range.inclusive_min, 
        
                                    byte_range.exclusive_max - 1);

400 on write

That concerns me as from memory it was working in July, I'll have to set it up with my AWS bucket to test, but I need to switch to other tasks for the rest of the week.

sjperkins · 2023-09-06T07:53:34Z

Briefly, it looks like AWS behaviourly treats 1-0 as a request for the entire content.

How does GCS respond to this?

laramiel · 2023-09-06T18:09:41Z

For localstack, the most important thing is that it mimics AWS responses, and AWS responds like this:

Test unconditional range read [1 .. 1], size 0

ReadTask: HttpRequest{
    GET https://BUCKET.s3.us-east-1.amazonaws.com/key_read
    user_agent=,
    headers<
       Range: bytes=1-0,
      host: BUCKET.s3.us-east-1.amazonaws.com,
      x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,
      x-amz-date: 20230906T175317Z,
      Authorization: AWS4-HMAC-SHA256 Credential=AKIAVTCZAT74DPVHV2BF/20230906/us-east-1/s3/aws4_request,SignedHeaders=host;range;x-amz-content-sha256;x-amz-date,Signature=65fb1444bfb66cf6db550ece96d1e8f8038b5759fee893407b7a2177603d4775
>}

ReadTask HttpResponse{
    code=200, 
    headers=<
      accept-ranges=bytes,
      content-length=15,
      content-type=application/octet-stream,
      date=Wed, 06 Sep 2023 17:53:18 GMT,
      etag="6f314ce98d753057568807b0863b3bf0",
      last-modified=Wed, 06 Sep 2023 17:53:18 GMT,
      server=AmazonS3,
      x-amz-id-2=He7FyCyQ4HX8juB1jJWU+pXWO4Ck4jpwGj+erOzr/ApfAxVNfH7gcGCAIF2U4LbdxsYq4FtubMQ=,
      x-amz-request-id=KKMATVTY6577BT80,
      x-amz-server-side-encryption=AES256>,
    body=_kvstore_value_
}

Which is the entire value.

sjperkins · 2023-09-06T18:24:11Z

For localstack, the most important thing is that it mimics AWS responses, and AWS responds like this:

Which is the entire value.

The localstack devs have indicated on the issue they agree that it should mimic AWS and return the entire response.

laramiel · 2023-09-06T19:12:32Z

I have a pending update for http-style backends to avoid sending invalid Range: headers.

ea7aecb

sjperkins · 2023-09-07T09:53:30Z

I had a closer look at the rfc and it looks like a server may choose to ignore an invalid range request:

A server that supports range requests MAY ignore or reject a Range header field that contains an invalid ranges-specifier (Section 14.1.1), a ranges-specifier with more than two overlapping ranges, or a set of many small ranges that are not listed in ascending order,

sjperkins · 2023-09-12T14:26:43Z

The AWS range issue fix was merged:

bug: S3 range requests where end < last return a 416, instead of returning the entire content. localstack/localstack#9076
fix S3 v3 Range handling localstack/localstack#9082

I guess it will be included in the 2.3 release, which does not seem to have many remaining issues:

https://github.com/localstack/localstack/pulls?q=is%3Apr+milestone%3A2.3+is%3Aopen

I'll likely recreate this PR once 2.3 is released.

sjperkins · 2023-09-21T08:22:45Z

FWIW I've run the localstack test case locally with the localstack/localstack:latest image (c.f. localstack/localstack#9076 (comment)) and tensorstore::internal::TestKeyValueReadWriteOps succeeds.

Looks like localstack 2.3 is targeting a 26th September release:

https://github.com/localstack/localstack/milestone/4

laramiel · 2023-09-21T18:18:19Z

You should be able to use even the older version now since I've updated the range request logic across the board to avoid the bad requests.

Based in part on #111 When provided with --localstack_binary, localstack_test will start localstack in host mode (via package localstack[runtime]). When provided with --localstack_endpoint, localstack_test will connect to a running localstack instance. In order to integrate it into the CI, we need to install localstack[runtime], however there is presently a pypa conflict between apache-beam and localstack. PiperOrigin-RevId: 574269409 Change-Id: I511f6a7c28b9d14d9836030988f6d9bcb68a0549

laramiel · 2023-10-17T22:03:39Z

With my latest commit it's possible to do this (on a mac with python 3.10) (in two terminals):

python3 -m pip install localstack[runtime]
localstack start --host

./bazelisk.py run //tensorstore/kvstore/s3:localstack_test -- \
   --localstack_endpoint=http://127.0.0.1:4566

It's almost possible to do this, however there are two outstanding issues.

The localstack_test issues a create bucket request against the wrong endpoint.
The s3 driver does not correctly identify the host for the bucket when endpoint is provided.

python3 -m pip install localstack[runtime]
./bazelisk.py run //tensorstore/kvstore/s3:localstack_test -- \
   --localstack_binary=$(which localstack)

sjperkins · 2023-10-18T12:25:30Z

With my latest commit it's possible to do this (on a mac with python 3.10) (in two terminals):
python3 -m pip install localstack[runtime]
localstack start --host
./bazelisk.py run //tensorstore/kvstore/s3:localstack_test -- \
   --localstack_endpoint=http://127.0.0.1:4566
It's almost possible to do this, however there are two outstanding issues.

The localstack_test issues a create bucket request against the wrong endpoint.

The s3 driver does not correctly identify the host for the bucket when endpoint is provided.
python3 -m pip install localstack[runtime]
./bazelisk.py run //tensorstore/kvstore/s3:localstack_test -- \
   --localstack_binary=$(which localstack)

I remember trying a localstack[runtime] install. IIRC it wanted sudo for access to low ports so I didn't pursue it further as it seemed to me that the docker-based approach would require similarly elevated privileges. I guess the benefit of localstack[runtime] is that it may be possible to run with bazel remote (as opposed to the docker approach).

Contine localstack test harness development

1ca7962

laramiel reviewed Sep 5, 2023

View reviewed changes

sjperkins commented Sep 5, 2023

View reviewed changes

sjperkins changed the title ~~Contine localstack test harness development~~ Continue localstack test harness development Sep 5, 2023

laramiel reviewed Sep 5, 2023

View reviewed changes

sjperkins closed this Oct 18, 2023

sjperkins deleted the continue-localstack-integration branch October 18, 2023 14:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Continue localstack test harness development #111

Continue localstack test harness development #111

sjperkins commented Sep 5, 2023 •

edited

laramiel commented Sep 5, 2023

laramiel Sep 5, 2023

sjperkins Sep 5, 2023

laramiel Sep 5, 2023

sjperkins commented Sep 5, 2023 •

edited

sjperkins commented Sep 5, 2023

sjperkins Sep 5, 2023

laramiel Sep 5, 2023

sjperkins Sep 5, 2023

sjperkins commented Sep 5, 2023 •

edited

laramiel Sep 5, 2023

laramiel commented Sep 5, 2023

laramiel commented Sep 5, 2023 •

edited

laramiel commented Sep 5, 2023 •

edited

sjperkins commented Sep 6, 2023 •

edited

sjperkins commented Sep 6, 2023

laramiel commented Sep 6, 2023 •

edited

sjperkins commented Sep 6, 2023 •

edited

laramiel commented Sep 6, 2023 •

edited

sjperkins commented Sep 7, 2023

sjperkins commented Sep 12, 2023

sjperkins commented Sep 21, 2023

laramiel commented Sep 21, 2023

laramiel commented Oct 17, 2023 •

edited

sjperkins commented Oct 18, 2023

Continue localstack test harness development #111

Continue localstack test harness development #111

Conversation

sjperkins commented Sep 5, 2023 • edited

laramiel commented Sep 5, 2023

laramiel Sep 5, 2023

Choose a reason for hiding this comment

sjperkins Sep 5, 2023

Choose a reason for hiding this comment

laramiel Sep 5, 2023

Choose a reason for hiding this comment

sjperkins commented Sep 5, 2023 • edited

sjperkins commented Sep 5, 2023

sjperkins Sep 5, 2023

Choose a reason for hiding this comment

laramiel Sep 5, 2023

Choose a reason for hiding this comment

sjperkins Sep 5, 2023

Choose a reason for hiding this comment

sjperkins commented Sep 5, 2023 • edited

laramiel Sep 5, 2023

Choose a reason for hiding this comment

laramiel commented Sep 5, 2023

laramiel commented Sep 5, 2023 • edited

laramiel commented Sep 5, 2023 • edited

sjperkins commented Sep 6, 2023 • edited

sjperkins commented Sep 6, 2023

laramiel commented Sep 6, 2023 • edited

sjperkins commented Sep 6, 2023 • edited

laramiel commented Sep 6, 2023 • edited

sjperkins commented Sep 7, 2023

sjperkins commented Sep 12, 2023

sjperkins commented Sep 21, 2023

laramiel commented Sep 21, 2023

laramiel commented Oct 17, 2023 • edited

sjperkins commented Oct 18, 2023

sjperkins commented Sep 5, 2023 •

edited

sjperkins commented Sep 5, 2023 •

edited

sjperkins commented Sep 5, 2023 •

edited

laramiel commented Sep 5, 2023 •

edited

laramiel commented Sep 5, 2023 •

edited

sjperkins commented Sep 6, 2023 •

edited

laramiel commented Sep 6, 2023 •

edited

sjperkins commented Sep 6, 2023 •

edited

laramiel commented Sep 6, 2023 •

edited

laramiel commented Oct 17, 2023 •

edited