New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jumpserver 指令审计中对emoji表情不支持抛出错误 #1747

Open
harvey-zang opened this Issue Aug 27, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@harvey-zang

harvey-zang commented Aug 27, 2018

[简述你的问题]

使用版本

jumpserver 1.3

问题复现步骤
  1. 开发人员tail -f 查看后台日志时,日志中会有用户输入的标题,含有emoji表情。
  2. jumpserver 保存指令输入输出时,因为默认编码为UTF-8,不支持emoji,造成jumpserver.log中大量报错出现。
具体表现[截图可能会更好些,最好能截全]

File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\xAA\x8C\x81 ...' for column 'output' at row 1")

其他

[注:] 完成后请关闭 issue

@harvey-zang

This comment has been minimized.

Show comment
Hide comment
@harvey-zang

harvey-zang Aug 28, 2018

而且还是循环的插入,终端echo了两个emoji表情,日志开始循环的报错。这在生产环境中,大量的查看含有emoji表情的日志,会导致大量死循环报错,最终导致jumpserver反应变慢。

harvey-zang commented Aug 28, 2018

而且还是循环的插入,终端echo了两个emoji表情,日志开始循环的报错。这在生产环境中,大量的查看含有emoji表情的日志,会导致大量死循环报错,最终导致jumpserver反应变慢。

@printff

This comment has been minimized.

Show comment
Hide comment
@printff

printff Sep 8, 2018

我的也是,狂刷这个错误日志,怎样把这条错误数据删掉?

2018-09-08 21:29:34 [exception ERROR] Internal Server Error: /api/terminal/v1/command/
Traceback (most recent call last):
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 101, in execute
return self.cursor.execute(query, args)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
rowcount = self._do_query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
db.query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
_mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x90\x8E\xE5\xA8...' for column 'output' at row 1")

The above exception was the direct cause of the following exception:
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.6/contextlib.py", line 53, in inner
return func(*args, **kwds)
File "/opt/py3/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
return view_func(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/viewsets.py", line 90, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 489, in dispatch
response = self.handle_exception(exc)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 449, in handle_exception
self.raise_uncaught_exception(exc)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 486, in dispatch
response = handler(request, *args, **kwargs)
File "/opt/jumpserver/apps/terminal/api.py", line 242, in create
ok = self.command_store.bulk_save(serializer.validated_data)
File "/opt/jumpserver/apps/terminal/backends/command/db.py", line 38, in bulk_save
return self.model.objects.bulk_create(_commands)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 439, in bulk_create
self._batched_insert(objs_with_pk, fields, batch_size)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1083, in _batched_insert
self._insert(item, fields=fields, using=self.db)
File "/opt/jumpserver/apps/terminal/backends/command/db.py", line 38, in bulk_save
return self.model.objects.bulk_create(_commands)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 439, in bulk_create
self._batched_insert(objs_with_pk, fields, batch_size)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1083, in _batched_insert
self._insert(item, fields=fields, using=self.db)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1060, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql
cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/py3/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 101, in execute
return self.cursor.execute(query, args)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
rowcount = self._do_query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
db.query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x90\x8E\xE5\xA8...' for column 'output' at row 1")

printff commented Sep 8, 2018

我的也是,狂刷这个错误日志,怎样把这条错误数据删掉?

2018-09-08 21:29:34 [exception ERROR] Internal Server Error: /api/terminal/v1/command/
Traceback (most recent call last):
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 101, in execute
return self.cursor.execute(query, args)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
rowcount = self._do_query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
db.query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
_mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x90\x8E\xE5\xA8...' for column 'output' at row 1")

The above exception was the direct cause of the following exception:
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
response = get_response(request)
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/py3/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.6/contextlib.py", line 53, in inner
return func(*args, **kwds)
File "/opt/py3/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 58, in wrapped_view
return view_func(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/viewsets.py", line 90, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 489, in dispatch
response = self.handle_exception(exc)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 449, in handle_exception
self.raise_uncaught_exception(exc)
File "/opt/py3/lib/python3.6/site-packages/rest_framework/views.py", line 486, in dispatch
response = handler(request, *args, **kwargs)
File "/opt/jumpserver/apps/terminal/api.py", line 242, in create
ok = self.command_store.bulk_save(serializer.validated_data)
File "/opt/jumpserver/apps/terminal/backends/command/db.py", line 38, in bulk_save
return self.model.objects.bulk_create(_commands)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 439, in bulk_create
self._batched_insert(objs_with_pk, fields, batch_size)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1083, in _batched_insert
self._insert(item, fields=fields, using=self.db)
File "/opt/jumpserver/apps/terminal/backends/command/db.py", line 38, in bulk_save
return self.model.objects.bulk_create(_commands)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 439, in bulk_create
self._batched_insert(objs_with_pk, fields, batch_size)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1083, in _batched_insert
self._insert(item, fields=fields, using=self.db)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/query.py", line 1060, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/opt/py3/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1099, in execute_sql
cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/py3/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/opt/py3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 101, in execute
return self.cursor.execute(query, args)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 411, in _query
rowcount = self._do_query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/cursors.py", line 374, in _do_query
db.query(q)
File "/opt/py3/lib/python3.6/site-packages/MySQLdb/connections.py", line 277, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x90\x8E\xE5\xA8...' for column 'output' at row 1")

@harvey-zang

This comment has been minimized.

Show comment
Hide comment
@harvey-zang

harvey-zang Sep 8, 2018

harvey-zang commented Sep 8, 2018

@wojiushixiaobai

This comment has been minimized.

Show comment
Hide comment
@wojiushixiaobai
Collaborator

wojiushixiaobai commented Sep 8, 2018

@printff

This comment has been minimized.

Show comment
Hide comment
@printff

printff Sep 8, 2018

可以了,谢谢啦

printff commented Sep 8, 2018

可以了,谢谢啦

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment