Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow schema of destination table to be specified in CsvDataReader constructor #1435

Merged
merged 1 commit into from Mar 23, 2021

Conversation

mattosaurus
Copy link
Contributor

IDataReader maps between the input source and the destination table using the schema details provided by GetSchemaTable(). Currently in CsvDataReader all columns are defaulted to strings which causes conversion issues when using as a source for SqlBulkCopy.

To remedy this I have added an additional constructor that accepts a DataTable along with the required CsvReader, this DataTable should contain the schema of the destination table and can be generated like so.

public static DataTable GetDataTableSchema(string schema, string tableName)
{
	DataTable dataTable = new DataTable();

	using (SqlConnection _sqlConnection = new SqlConnection(_sqlConnectionString))
	using (SqlCommand command = new SqlCommand("select top 0 * from " + schema + "." + tableName))
	{
		command.Connection = _sqlConnection;
		_sqlConnection.Open();

		using (SqlDataReader reader = command.ExecuteReader(CommandBehavior.SchemaOnly))
		{
			dataTable = reader.GetSchemaTable();
		}

		_sqlConnection.Close();
	}

	return dataTable;
}

This will ensure that the correctly typed Get method will be used within the CsvDataReader.

A complete example can be seen here.

…g the destination table schema to be used instead of GetSchemaTable when provided.
@lucastheisen
Copy link

This is a really old patch that seems to have never been merged. It seems to provide the desired ability to stream data into SqlBulkCopy without reading the entire dataset into memory. Is there anything preventing this from happening? I am about to just grab @mattosaurus fork and see if it works for me, but would be nice to have this upstream.

@JoshClose JoshClose merged commit 7b5167c into JoshClose:master Mar 23, 2021
@JoshClose
Copy link
Owner

This is in 26.1.0 on NuGet. https://www.nuget.org/packages/CsvHelper/26.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants