Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use UploadCsvAsync #311

Closed
jeroen-corsius-choreograph opened this issue May 10, 2024 · 1 comment
Closed

Unable to use UploadCsvAsync #311

jeroen-corsius-choreograph opened this issue May 10, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@jeroen-corsius-choreograph
Copy link

What happened?

Trying to upload a CSV file fails.

System.Net.Http.HttpRequestException: IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Pa...

System.Net.Http.HttpRequestException
IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Parameter 'hostName') (0.0.0.0:9050)
   at Google.Cloud.BigQuery.V2.BigQueryClientImpl.UploadDataAsync(JobConfigurationLoad loadConfiguration, Stream input, String contentType, JobCreationOptions options, CancellationToken cancellationToken)
   at Google.Cloud.BigQuery.V2.BigQueryClientImpl.UploadCsvAsync(TableReference tableReference, TableSchema schema, Stream input, UploadCsvOptions options, CancellationToken cancellationToken)
   at Plugin.Google.Test.Integration.Export.ExampleTest.Example() in C:\Projects\badger\services\Plugin.Google.Test\Integration\Export\TemptTest.cs:line 37
   at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass46_0.<<InvokeTestMethodAsync>b__1>d.MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 253
--- End of stack trace from previous location ---
   at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90

System.ArgumentException
IPv4 address 0.0.0.0 and IPv6 address ::0 are unspecified addresses that cannot be used as a target address. (Parameter 'hostName')
   at System.Net.Dns.GetHostEntryOrAddressesCoreAsync(String hostName, Boolean justReturnParsedIp, Boolean throwOnIIPAny, Boolean justAddresses, AddressFamily family, CancellationToken cancellationToken)
   at System.Net.Dns.GetHostAddressesAsync(String hostNameOrAddress, AddressFamily family, CancellationToken cancellationToken)
   at System.Net.Sockets.SocketAsyncEventArgs.DnsConnectAsync(DnsEndPoint endPoint, SocketType socketType, ProtocolType protocolType)
   at System.Net.Sockets.Socket.ConnectAsync(SocketAsyncEventArgs e, Boolean userSocket, Boolean saeaCancelable)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ConnectAsync(Socket socket)
   at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

What did you expect to happen?

The action succeeding

How can we reproduce it (as minimally and precisely as possible)?

using System.Net;
using System.Text;
using Google.Cloud.BigQuery.V2;
using Testcontainers.BigQuery;
using Xunit;

namespace Plugin.Google.Test.Integration.Export;

public class ExampleTest {
  [Fact]
  public async Task Example() {
    var projectName = "some_project";
    var datasetName = "some_dataset";
    var tableName = "some_table";

    // Create/start container
    var bigQueryContainer = new BigQueryBuilder().WithProject(projectName).WithReuse(true).Build();
    await bigQueryContainer.StartAsync();
    
    // Create client
    var bigQueryClient = await new BigQueryClientBuilder {
      BaseUri = bigQueryContainer.GetEmulatorEndpoint(),
      ProjectId = projectName,
    }.BuildAsync();

    // Create dataset
    await bigQueryClient.CreateDatasetAsync(datasetName);
    
    // Create table
    var createTableQuery = $"CREATE TABLE IF NOT EXISTS {datasetName}.{tableName} (id INT64, name STRING);";
    await bigQueryClient.ExecuteQueryAsync(createTableQuery, null);
    
    // Create CSV memory stream
    var csvData = "1,John\n2,Jane\n3,Michael";
    var csvMemoryStream = new MemoryStream(Encoding.UTF8.GetBytes(csvData));
    
    // Upload CSV
    var uploadCsvJob = await bigQueryClient.UploadCsvAsync(datasetName, tableName, null, csvMemoryStream);
    await uploadCsvJob.PollUntilCompletedAsync();
  }
}

Anything else we need to know?

NuGet package version: 3.8.0

Workaround:
If I replace the "Create client" part of the provided example with the code below, I'm able to upload CSV files:

    // Create client
    var mappedPublicPort = bigQueryContainer.GetMappedPublicPort(BigQueryBuilder.BigQueryPort);
    var exposedContainerAddress = $"localhost:{mappedPublicPort}";
    var bigQueryClient = await new BigQueryClientBuilder {
      BaseUri = bigQueryContainer.GetEmulatorEndpoint(),
      ProjectId = projectName,
      HttpClientFactory = HttpClientFactory.ForProxy(new WebProxy(exposedContainerAddress)),
    }.BuildAsync();
@jeroen-corsius-choreograph jeroen-corsius-choreograph added the bug Something isn't working label May 10, 2024
@jeroen-corsius-choreograph
Copy link
Author

I'm sorry, I've created the issue on the wrong repository (should have been: testcontainers/testcontainers-dotnet#1175).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant