Made "refactor_social_login_keys" patch SQL-based #5116

gaurav-naik · 2018-03-05T09:05:50Z

Replaced ORM calls with SQL code to reduce execution time.

Testing required!

revant · 2018-03-05T13:35:56Z

revant@revant-laptop:~/frappe-bench$ bench --site test.local.com migrate
Migrating test.local.com
Executing frappe.patches.v10_0.refactor_social_login_keys in test.local.com (40a783fd8f382214)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 94, in <module>
    main()
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/site.py", line 222, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website)
  File "/home/revant/frappe-bench/apps/frappe/frappe/migrate.py", line 39, in migrate
    frappe.modules.patch_handler.run_all()
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 29, in run_all
    if not run_single(patchmodule = patch):
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 63, in run_single
    return execute_patch(patchmodule, method, methodargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 83, in execute_patch
    frappe.get_attr(patchmodule.split()[0] + ".execute")()
  File "/home/revant/frappe-bench/apps/frappe/frappe/patches/v10_0/refactor_social_login_keys.py", line 33, in execute
    frappe.db.sql(query)
  File "/home/revant/frappe-bench/apps/frappe/frappe/database.py", line 183, in sql
    self._cursor.execute(query)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 165, in execute
    result = self._query(query)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 321, in _query
    conn.query(q)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 860, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1061, in _read_query_result
    result.read()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1349, in read
    first_packet = self.connection._read_packet()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1018, in _read_packet
    packet.check_error()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1241, u'Operand should contain 1 column(s)')

revant · 2018-03-05T14:07:42Z

pymysql.err.ProgrammingError: (1064, u"You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'and `tabUser`.`userid` is not null' at line 5")

…ocial Login Keys

revant · 2018-03-06T11:49:48Z

revant@revant-laptop:~/frappe-bench$ bench --site test.local.com migrate 
Migrating test.local.com
Executing frappe.patches.v10_0.refactor_social_login_keys in test.local.com (40a783fd8f382214)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 94, in <module>
    main()
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/site.py", line 222, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website)
  File "/home/revant/frappe-bench/apps/frappe/frappe/migrate.py", line 39, in migrate
    frappe.modules.patch_handler.run_all()
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 29, in run_all
    if not run_single(patchmodule = patch):
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 63, in run_single
    return execute_patch(patchmodule, method, methodargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 83, in execute_patch
    frappe.get_attr(patchmodule.split()[0] + ".execute")()
  File "/home/revant/frappe-bench/apps/frappe/frappe/patches/v10_0/refactor_social_login_keys.py", line 58, in execute
    frappe.db.sql(query)
  File "/home/revant/frappe-bench/apps/frappe/frappe/database.py", line 183, in sql
    self._cursor.execute(query)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 165, in execute
    result = self._query(query)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 321, in _query
    conn.query(q)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 860, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1061, in _read_query_result
    result.read()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1349, in read
    first_packet = self.connection._read_packet()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1018, in _read_packet
    packet.check_error()
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1136, u"Column count doesn't match value count at row 1")

…l_login_keys_patch_sql

rmehta · 2018-03-13T06:27:08Z

@revant is this good to go? Can you test and merge?

revant · 2018-03-13T15:09:45Z

It still has errors.
I'll test and merge.

revant · 2018-03-13T16:26:09Z

Migrating test.local.com
Executing execute:frappe.reload_doc('core', 'doctype', 'docfield', force=True) #2018-02-20 in test.local.com (a8020c826ec60a10)
Success
Executing frappe.patches.v10_0.refactor_social_login_keys in test.local.com (a8020c826ec60a10)
Traceback (most recent call last):
  File "/opt/pypy3/lib-python/3/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/pypy3/lib-python/3/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 94, in <module>
    main()
  File "/home/revant/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name='bench')
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/revant/frappe-bench/env/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/revant/frappe-bench/env/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 25, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/commands/site.py", line 222, in migrate
    migrate(context.verbose, rebuild_website=rebuild_website)
  File "/home/revant/frappe-bench/apps/frappe/frappe/migrate.py", line 39, in migrate
    frappe.modules.patch_handler.run_all()
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 29, in run_all
    if not run_single(patchmodule = patch):
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 63, in run_single
    return execute_patch(patchmodule, method, methodargs)
  File "/home/revant/frappe-bench/apps/frappe/frappe/modules/patch_handler.py", line 83, in execute_patch
    frappe.get_attr(patchmodule.split()[0] + ".execute")()
  File "/home/revant/frappe-bench/apps/frappe/frappe/patches/v10_0/refactor_social_login_keys.py", line 6, in execute
    run_patch()
  File "/home/revant/frappe-bench/apps/frappe/frappe/patches/v10_0/refactor_social_login_keys.py", line 69, in run_patch
    insert_user_social_login(user.name, user.modified_by, 'frappe', idx, userid=user.frappe_userid)
  File "/home/revant/frappe-bench/apps/frappe/frappe/patches/v10_0/refactor_social_login_keys.py", line 113, in insert_user_social_login
    values= "'" + "', '".join(values) + "'"
TypeError: sequence item 8: expected str, int found

Gaurav Naik added 2 commits March 5, 2018 14:28

Made refactor_social_login_keys SQL based

c096e10

Fixed syntax

73daa67

revant self-requested a review March 5, 2018 13:34

revant added the waiting-for-fixes label Mar 5, 2018

Syntax fixes frappe#2

8be805c

Gaurav Naik added 4 commits March 5, 2018 19:47

Syntax fixes frappe#3

3c8e821

OR instead of AND in null condition

68c2d1d

Refactored query to include standard columns. Reintroduced code for S…

146755e

…ocial Login Keys

Removed print statement

a5babf1

Gaurav Naik added 4 commits March 6, 2018 18:44

Added provider to standard cols

c05e27d

Merge branch 'develop' of https://github.com/frappe/frappe into socia…

e5a6c58

…l_login_keys_patch_sql

Complete refactor

e7f4833

Corrections to columns and values

4470770

Corrected insert_user_social_login and position of idx increment

8b540b4

Gaurav Naik added 2 commits March 13, 2018 22:02

cstr in idx

ac36440

Corrected target column list

f5d5ea7

revant approved these changes Mar 13, 2018

View reviewed changes

revant removed the waiting-for-fixes label Mar 13, 2018

revant requested a review from nabinhait March 13, 2018 17:23

revant added 2 commits March 15, 2018 10:27

Codacy fixes

52408d6

Fix Codacy Errors

774e022

rmehta merged commit fdc9de8 into frappe:develop Apr 6, 2018

github-actions bot locked as resolved and limited conversation to collaborators May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Made "refactor_social_login_keys" patch SQL-based #5116

Made "refactor_social_login_keys" patch SQL-based #5116

gaurav-naik commented Mar 5, 2018 •

edited

Loading

revant commented Mar 5, 2018

revant commented Mar 5, 2018

revant commented Mar 6, 2018

rmehta commented Mar 13, 2018

revant commented Mar 13, 2018

revant commented Mar 13, 2018

Made "refactor_social_login_keys" patch SQL-based #5116

Made "refactor_social_login_keys" patch SQL-based #5116

Conversation

gaurav-naik commented Mar 5, 2018 • edited Loading

revant commented Mar 5, 2018

revant commented Mar 5, 2018

revant commented Mar 6, 2018

rmehta commented Mar 13, 2018

revant commented Mar 13, 2018

revant commented Mar 13, 2018

gaurav-naik commented Mar 5, 2018 •

edited

Loading