Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitbase returns commit_author_when that can't be parsed by pymysql #805

Closed
smacker opened this issue Apr 19, 2019 · 2 comments · Fixed by #810
Closed

gitbase returns commit_author_when that can't be parsed by pymysql #805

smacker opened this issue Apr 19, 2019 · 2 comments · Fixed by #810
Assignees
Labels
bug Something isn't working

Comments

@smacker
Copy link

smacker commented Apr 19, 2019

Reproduce:

git clone gitlab.com/gitlab-org/gitlab-test
<run gitbase with this repo>

query:

select commit_author_when from commits where repository_id = 'gitlab-test';

returns

...
292278994-08-17 07:12:55
...

If you try to query it with pymysql it would return error:

ValueError: invalid literal for int() with base 10: '4-'

full traceback:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-80cb0324a498> in <module>
      8 #     i += 1
      9 
---> 10 query_gitbase("select commit_author_when from commits")

/home/jovyan/work/gitbase.py in query_gitbase(sql_query, host, port, user, password, db)
     50             # might be broken.
     51             _kill_conn(id, host, port, user, password, db)
---> 52         raise e
     53     finally:
     54         connection.close()

/home/jovyan/work/gitbase.py in query_gitbase(sql_query, host, port, user, password, db)
     43 
     44         with connection.cursor() as cursor:
---> 45             cursor.execute(sql_query)
     46             return cursor.fetchall()
     47     except (Exception, KeyboardInterrupt) as e:

/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py in execute(self, query, args)
    168         query = self.mogrify(query, args)
    169 
--> 170         result = self._query(query)
    171         self._executed = query
    172         return result

/usr/local/lib/python3.6/dist-packages/pymysql/cursors.py in _query(self, q)
    326         self._last_executed = q
    327         self._clear_result()
--> 328         conn.query(q)
    329         self._do_get_result()
    330         return self.rowcount

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in query(self, sql, unbuffered)
    515                 sql = sql.encode(self.encoding, 'surrogateescape')
    516         self._execute_command(COMMAND.COM_QUERY, sql)
--> 517         self._affected_rows = self._read_query_result(unbuffered=unbuffered)
    518         return self._affected_rows
    519 

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in _read_query_result(self, unbuffered)
    730         else:
    731             result = MySQLResult(self)
--> 732             result.read()
    733         self._result = result
    734         if result.server_status is not None:

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in read(self)
   1080                 self._read_load_local_packet(first_packet)
   1081             else:
-> 1082                 self._read_result_packet(first_packet)
   1083         finally:
   1084             self.connection = None

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in _read_result_packet(self, first_packet)
   1150         self.field_count = first_packet.read_length_encoded_integer()
   1151         self._get_descriptions()
-> 1152         self._read_rowdata_packet()
   1153 
   1154     def _read_rowdata_packet_unbuffered(self):

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in _read_rowdata_packet(self)
   1188                 self.connection = None  # release reference to kill cyclic reference.
   1189                 break
-> 1190             rows.append(self._read_row_from_packet(packet))
   1191 
   1192         self.affected_rows = len(rows)

/usr/local/lib/python3.6/dist-packages/pymysql/connections.py in _read_row_from_packet(self, packet)
   1207                 if DEBUG: print("DEBUG: DATA = ", data)
   1208                 if converter is not None:
-> 1209                     data = converter(data)
   1210             row.append(data)
   1211         return tuple(row)

/usr/local/lib/python3.6/dist-packages/pymysql/converters.py in convert_mysql_timestamp(timestamp)
    330     year, month, day, hour, minute, second = \
    331         int(timestamp[:4]), int(timestamp[4:6]), int(timestamp[6:8]), \
--> 332         int(timestamp[8:10]), int(timestamp[10:12]), int(timestamp[12:14])
    333     try:
    334         return datetime.datetime(year, month, day, hour, minute, second)

most probably some other drivers would also fail to parse such datetime.

@mcuadros
Copy link
Contributor

mcuadros commented Apr 22, 2019

The DATETIME type is used for values that contain both date and time parts. MySQL retrieves and displays DATETIME values in 'YYYY-MM-DD HH:MM:SS' format. The supported range is '1000-01-01 00:00:00' to '9999-12-31 23:59:59'.

So we should limit the range of dates from '1000-01-01 00:00:00' to '9999-12-31 23:59:59'

@erizocosmico erizocosmico self-assigned this Apr 24, 2019
@erizocosmico erizocosmico transferred this issue from src-d/gitbase Apr 24, 2019
@erizocosmico erizocosmico transferred this issue from src-d/go-mysql-server Apr 24, 2019
@erizocosmico erizocosmico added the bug Something isn't working label Apr 24, 2019
@erizocosmico
Copy link
Contributor

Needs src-d/go-mysql-server#694 merged and updated before applying the fix here. Will leave blocked until then.

@erizocosmico erizocosmico added the blocked Some other issue is blocking this label Apr 24, 2019
@erizocosmico erizocosmico removed the blocked Some other issue is blocking this label May 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants