Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: fixed a bug that all TIMESTAMP fileds become 00 secconds. #148

Merged
merged 3 commits into from
Mar 13, 2018
Merged

BUG: fixed a bug that all TIMESTAMP fileds become 00 secconds. #148

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bfcef95
BUG: fix bug that all TIMESTAMP fileds become 00 secconds.
rhoboro Mar 12, 2018
69147dd
add a test for extact timestamp
rhoboro Mar 13, 2018
05ac27b
use fixed value instead of now()
rhoboro Mar 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion pandas_gbq/_load.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def encode_chunk(dataframe):
csv_buffer = six.StringIO()
dataframe.to_csv(
csv_buffer, index=False, header=False, encoding='utf-8',
date_format='%Y-%m-%d %H:%M')
date_format='%Y-%m-%d %H:%M:%S.%f')

# Convert to a BytesIO buffer so that unicode text is properly handled.
# See: https://github.com/pydata/pandas-gbq/issues/106
Expand Down
24 changes: 24 additions & 0 deletions pandas_gbq/tests/test_gbq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1389,6 +1389,30 @@ def test_upload_data_with_missing_schema_fields_raises_error(self):
private_key=self.credentials,
table_schema=test_schema)

def test_upload_data_with_timestamp(self):
test_id = "21"
test_size = 6
df = DataFrame(np.random.randn(test_size, 4), index=range(test_size),
columns=list('ABCD'))
test_timestamp = datetime.now(pytz.timezone('UTC'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this would be a value rather than .now, and that's harder to reproduce, but not a huge issue

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your review. I fixed this.
05ac27b

df['times'] = test_timestamp

gbq.to_gbq(
df, self.destination_table + test_id,
_get_project_id(),
private_key=self.credentials)

result_df = gbq.read_gbq("SELECT * FROM {0}".format(
self.destination_table + test_id),
project_id=_get_project_id(),
private_key=self.credentials)

assert len(result_df) == test_size

expected = df['times'].astype('M8[ns]').sort_values()
result = result_df['times'].astype('M8[ns]').sort_values()
tm.assert_numpy_array_equal(expected.values, result.values)

def test_list_dataset(self):
dataset_id = self.dataset_prefix + "1"
assert dataset_id in self.dataset.datasets()
Expand Down