You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Client: Docker Engine - Community
Version: 26.0.0
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.13.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.4.1
Path: /home/jeroencorsius/.docker/cli-plugins/docker-compose
scan: Docker Scan (Docker Inc.)
Version: v0.23.0
Path: /usr/libexec/docker/cli-plugins/docker-scan
Server:
Containers: 92
Running: 9
Paused: 0
Stopped: 83
Images: 1577
Server Version: 26.0.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 5.10.16.3-microsoft-standard-WSL2
Operating System: Ubuntu 20.04.6 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 24.84GiB
Name: EINGRMW2791-D
ID: dfbe5910-3e30-4a9d-8d1c-d81f3ffe03fd
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
[DEPRECATION NOTICE]: API is accessible on http://0.0.0.0:2375 without encryption.
Access to the remote API is equivalent to root access on the host. Refer
to the 'Docker daemon attack surface' section in the documentation for
more information: https://docs.docker.com/go/attack-surface/
In future versions this will be a hard failure preventing the daemon from starting! Learn more at: https://docs.docker.com/go/api-security/
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
What happened?
Trying to upload a CSV file fails.
Example code:
using System.Net;
using System.Text;
using Google.Cloud.BigQuery.V2;
using Testcontainers.BigQuery;
using Xunit;
namespace Plugin.Google.Test.Integration.Export;
public class ExampleTest {
[Fact]
public async Task Example() {
var projectName = "some_project";
var datasetName = "some_dataset";
var tableName = "some_table";
// Create/start container
var bigQueryContainer = new BigQueryBuilder().WithProject(projectName).WithReuse(true).Build();
await bigQueryContainer.StartAsync();
// Create client
var bigQueryClient = await new BigQueryClientBuilder {
BaseUri = bigQueryContainer.GetEmulatorEndpoint(),
ProjectId = projectName,
}.BuildAsync();
// Create dataset
await bigQueryClient.CreateDatasetAsync(datasetName);
// Create table
var createTableQuery = $"CREATE TABLE IF NOT EXISTS {datasetName}.{tableName} (id INT64, name STRING);";
await bigQueryClient.ExecuteQueryAsync(createTableQuery, null);
// Create CSV memory stream
var csvData = "1,John\n2,Jane\n3,Michael";
var csvMemoryStream = new MemoryStream(Encoding.UTF8.GetBytes(csvData));
// Upload CSV
var uploadCsvJob = await bigQueryClient.UploadCsvAsync(datasetName, tableName, null, csvMemoryStream);
await uploadCsvJob.PollUntilCompletedAsync();
}
}
Relevant log output
System.Net.Http.HttpRequestException: IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Pa...
System.Net.Http.HttpRequestException
IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Parameter 'hostName') (0.0.0.0:9050)
at Google.Cloud.BigQuery.V2.BigQueryClientImpl.UploadDataAsync(JobConfigurationLoad loadConfiguration, Stream input, String contentType, JobCreationOptions options, CancellationToken cancellationToken)
at Google.Cloud.BigQuery.V2.BigQueryClientImpl.UploadCsvAsync(TableReference tableReference, TableSchema schema, Stream input, UploadCsvOptions options, CancellationToken cancellationToken)
at Plugin.Google.Test.Integration.Export.ExampleTest.Example() in C:\Projects\badger\services\Plugin.Google.Test\Integration\Export\TemptTest.cs:line 37
at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass46_0.<<InvokeTestMethodAsync>b__1>d.MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 253
--- End of stack trace from previous location ---
at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48 at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90System.ArgumentExceptionIPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Parameter 'hostName') at System.Net.Dns.GetHostEntryOrAddressesCoreAsync(String hostName, Boolean justReturnParsedIp, Boolean throwOnIIPAny, Boolean justAddresses, AddressFamily family, CancellationToken cancellationToken) at System.Net.Dns.GetHostAddressesAsync(String hostNameOrAddress, AddressFamily family, CancellationToken cancellationToken) at System.Net.Sockets.SocketAsyncEventArgs.DnsConnectAsync(DnsEndPoint endPoint, SocketType socketType, ProtocolType protocolType) at System.Net.Sockets.Socket.ConnectAsync(SocketAsyncEventArgs e, Boolean userSocket, Boolean saeaCancelable) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ConnectAsync(Socket socket) at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
Additional information
Workaround
If I replace the "Create client" part of the provided example with the code below, I'm able to upload CSV files:
// Create client
var mappedPublicPort = bigQueryContainer.GetMappedPublicPort(BigQueryBuilder.BigQueryPort);
var exposedContainerAddress = $"localhost:{mappedPublicPort}";
var bigQueryClient = await new BigQueryClientBuilder {
BaseUri = bigQueryContainer.GetEmulatorEndpoint(),
ProjectId = projectName,
HttpClientFactory = HttpClientFactory.ForProxy(new WebProxy(exposedContainerAddress)),
}.BuildAsync();
The text was updated successfully, but these errors were encountered:
I am not very familiar with BigQuery, but while debugging the issue, I noticed that the client initiates the resumable upload session and returns the following URI (for the session URI or upload URI UploadUri property):
Which I believe is incorrect. I assume the IP should be the Docker host (127.0.0.1), and the port should be the randomly assigned host port. I expect the same problem when running the container from the CLI (without Testcontainers). Although, I am unsure if configurations are missing (or if there are any configurations available at all) or if your workaround is the correct approach and should be documented.
Perhaps the best is to create an upstream issue (probably the initial issue goccy/bigquery-emulator#311 was right) and get more details about how the URI gets created and whether any configurations are available or necessary. I have not looked at the documentation yet, so that is probably something we should do too.
As mentioned in my previous comment, I do not think this is an issue with Testcontainers. If you have any further information about the issue, specifically regarding Testcontainers, please do not hesitate to reopen the issue. In the meantime, I will close it.
Testcontainers version
3.8.0
Using the latest Testcontainers version?
Yes
Host OS
Windows
Host arch
x64
.NET version
8.0.100
Docker version
Docker info
What happened?
Trying to upload a CSV file fails.
Example code:
Relevant log output
Additional information
Workaround
If I replace the "Create client" part of the provided example with the code below, I'm able to upload CSV files:
The text was updated successfully, but these errors were encountered: