eox-tagging not working with tutor

[angetresca-mercadolibre]

Problem: Adding eox-tagging to a fresh instance of tutor (using Maple as default) it raises an ImportError.

Maybe we are missing some configuration to use this lib with tutor?

Steps we are using to add eox-tagging to tutor instance:

1. Install a new fresh instance of tutor (locally) following this steps. After this the local tutor instance is running ok.
2. Add to the tutor configs in the file cat "$(tutor config printroot)/config.yml" this lines that installs the lib:

OPENEDX_EXTRA_PIP_REQUIREMENTS:
- git+https://github.com/eduNEXT/eox-tagging.git@v4.0.0#egg=eox_tagging==4.0.0

3. Create a new tutor plugin in a yaml file following this steps. The yaml file contains the following config to be added in the openedx common settings:

name: eox-tagging-plugin
version: 0.1.0
patches:
  openedx-common-settings: |
    EOX_TAGGING_GET_ENROLLMENT_OBJECT: "eox_tagging.edxapp_wrappers.backends.enrollment_l_v1"

4. Install the plugin doing tutor plugins enable eox-tagging-plugin and then tutor config save.
5. Build the openedx image doing tutor images build openedx
6. Start the tutor instance with the new config doing tutor local quickstart
   Here it raises the error: ImportError: cannot import name 'generate_password' from 'openedx.core.djangoapps.user_authn.utils' (/openedx/edx-platform/openedx/core/djangoapps/user_authn/utils.py)

Complete log:

➜  ~ tutor local quickstart
==================================================
        Interactive platform configuration
==================================================
Are you configuring a production platform? Type 'n' if you are just testing Tutor on your local computer [y/N] n
As you are not running this platform in production, we automatically set the following configuration values:
    LMS_HOST = local.overhang.io
    CMS_HOST = studio.local.overhang.io
    ENABLE_HTTPS = False
Your platform name/title [open edx test]
Your public contact email address [contact@local.overhang.io]
The default language code for the platform [en]
Configuration saved to /Users/atresca/tutor/config.yml
Environment generated in /Users/atresca/tutor/env
==============================================
        Stopping any existing platform
==============================================
docker-compose -f /Users/atresca/tutor/env/local/docker-compose.yml -f /Users/atresca/tutor/env/local/docker-compose.prod.yml --project-name tutor_local stop
Stopping tutor_local_cms-worker_1    ... done
Stopping tutor_local_lms-worker_1    ... done
Stopping tutor_local_cms_1           ... done
Stopping tutor_local_lms_1           ... done
Stopping tutor_local_elasticsearch_1 ... done
Stopping tutor_local_mongodb_1       ... done
Stopping tutor_local_redis_1         ... done
Stopping tutor_local_mysql_1         ... done
Stopping tutor_local_smtp_1          ... done
Stopping tutor_local_caddy_1         ... done
======================================================
        Starting the platform in detached mode
======================================================
docker-compose -f /Users/atresca/tutor/env/local/docker-compose.yml -f /Users/atresca/tutor/env/local/docker-compose.prod.yml --project-name tutor_local up --remove-orphans --build -d
Starting tutor_local_lms-permissions_1           ... done
Starting tutor_local_caddy_1                     ... done
Starting tutor_local_elasticsearch-permissions_1 ... done
Starting tutor_local_mongodb-permissions_1       ... done
Starting tutor_local_mysql-permissions_1         ... done
Starting tutor_local_smtp_1                      ... done
Starting tutor_local_redis-permissions_1         ... done
Starting tutor_local_mysql_1                     ... done
Starting tutor_local_cms-permissions_1           ... done
Starting tutor_local_mongodb_1                   ... done
Starting tutor_local_elasticsearch_1             ... done
Starting tutor_local_redis_1                     ... done
Recreating tutor_local_lms_1                     ... done
Recreating tutor_local_cms_1                     ... done
Recreating tutor_local_lms-worker_1              ... done
Recreating tutor_local_cms-worker_1              ... done
================================================
        Database creation and migrations
================================================
Initialising all services...
Initialising mysql...
docker-compose -f /Users/atresca/tutor/env/local/docker-compose.yml -f /Users/atresca/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /Users/atresca/tutor/env/local/docker-compose.jobs.yml run --rm mysql-job sh -e -c echo "Initialising MySQL..."
mysql_connection_max_attempts=10
mysql_connection_attempt=0
until mysql -u root --password="imUV4Dq8" --host "mysql" --port 3306 -e 'exit'
do
    mysql_connection_attempt=$(expr $mysql_connection_attempt + 1)
    echo "    [$mysql_connection_attempt/$mysql_connection_max_attempts] Waiting for MySQL service (this may take a while)..."
    if [ $mysql_connection_attempt -eq $mysql_connection_max_attempts ]
    then
      echo "MySQL initialisation error" 1>&2
      exit 1
    fi
    sleep 10
done
echo "MySQL is up and running"

# edx-platform database
mysql -u root --password="imUV4Dq8" --host "mysql" --port 3306 -e 'CREATE DATABASE IF NOT EXISTS openedx;'
mysql -u root --password="imUV4Dq8" --host "mysql" --port 3306 -e 'GRANT ALL ON openedx.* TO "openedx"@"%" IDENTIFIED BY "Qt8pwZ8h";'
Creating tutor_local_mysql-job_run ... done
Initialising MySQL...
mysql: [Warning] Using a password on the command line interface can be insecure.
MySQL is up and running
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
Initialising lms...
docker-compose -f /Users/atresca/tutor/env/local/docker-compose.yml -f /Users/atresca/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /Users/atresca/tutor/env/local/docker-compose.jobs.yml run --rm lms-job sh -e -c dockerize -wait tcp://mysql:3306 -timeout 20s

echo "Loading settings $DJANGO_SETTINGS_MODULE"

./manage.py lms migrate

# Create oauth2 apps for CMS SSO
# https://github.com/openedx/edx-platform/blob/master/docs/guides/studio_oauth.rst
./manage.py lms manage_user cms cms@openedx --unusable-password
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.local.overhang.io/complete/edx-oauth2/" \
  --client-id cms-sso \
  --client-secret 4oZHJ9MY9HBh090pMnm0VIil \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso cms
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.local.overhang.io:8001/complete/edx-oauth2/" \
  --client-id cms-sso-dev \
  --client-secret 4oZHJ9MY9HBh090pMnm0VIil \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso-dev cms


# Fix incorrect uploaded file path
if [ -d /openedx/data/uploads/ ]; then
  if [ -n "$(ls -A /openedx/data/uploads/)" ]; then
    echo "Migrating LMS uploaded files to shared directory"
    mv /openedx/data/uploads/* /openedx/media/
    rm -rf /openedx/data/uploads/
  fi
fi

# Create waffle switches to enable some features, if they have not been explicitly defined before
# Completion tracking: add green ticks to every completed unit
(./manage.py lms waffle_switch --list | grep completion.enable_completion_tracking) || ./manage.py lms waffle_switch --create completion.enable_completion_tracking on
Creating tutor_local_lms-job_run ... done
2022/04/21 16:08:07 Ready: tcp://mysql:3306.
Loading settings lms.envs.tutor.production
2022-04-21 16:08:12,932 WARNING 12 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

2022-04-21 16:08:13,116 WARNING 12 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  from collections import Mapping, MutableMapping

2022-04-21 16:08:15,944 WARNING 12 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/edx-platform/openedx/core/types/admin.py:49: DeprecationWarning: Django 3.2+ available: the _admin_display method and the AdminMethodclass should be removed from openedx.core.types
  warnings.warn(

Traceback (most recent call last):
  File "./manage.py", line 106, in <module>
    execute_from_command_line([sys.argv[0]] + django_args)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 75, in handle
    self.check(databases=[database])
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/openedx/venv/lib/python3.8/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/openedx/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 416, in check
    for pattern in self.url_patterns:
  File "/openedx/venv/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/openedx/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 602, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/openedx/venv/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/openedx/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 595, in urlconf_module
    return import_module(self.urlconf_name)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/urls.py", line 984, in <module>
    urlpatterns.extend(get_plugin_url_patterns(ProjectType.LMS))
  File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/plugin_urls.py", line 34, in get_plugin_url_patterns
    return [
  File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/plugin_urls.py", line 35, in <listcomp>
    _get_url(url_module_path, url_config)
  File "/openedx/venv/lib/python3.8/site-packages/edx_django_utils/plugins/plugin_urls.py", line 24, in _get_url
    return url(regex, include((url_module_path, app_name), namespace=namespace))
  File "/openedx/venv/lib/python3.8/site-packages/django/urls/conf.py", line 34, in include
    urlconf_module = import_module(urlconf_module)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/urls.py", line 6, in <module>
    from eox_core.api_schema import docs_ui_view
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/api_schema.py", line 12, in <module>
    from eox_core.api.v1 import views
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/api/v1/views.py", line 22, in <module>
    from eox_core.api.v1.serializers import (
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/api/v1/serializers.py", line 22, in <module>
    from eox_core.utils import (
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/utils.py", line 15, in <module>
    UserProfile = get_user_profile()
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/edxapp_wrapper/users.py", line 88, in get_user_profile
    backend = import_module(backend_function)
  File "/opt/pyenv/versions/3.8.12/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/openedx/venv/lib/python3.8/site-packages/eox_core/edxapp_wrapper/backends/users_l_v1.py", line 36, in <module>
    from openedx.core.djangoapps.user_authn.utils import generate_password  # pylint: disable=import-error,unused-import
ImportError: cannot import name 'generate_password' from 'openedx.core.djangoapps.user_authn.utils' (/openedx/edx-platform/openedx/core/djangoapps/user_authn/utils.py)
ERROR: 1
Error: Command failed with status 1: docker-compose -f /Users/atresca/tutor/env/local/docker-compose.yml -f /Users/atresca/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /Users/atresca/tutor/env/local/docker-compose.jobs.yml run --rm lms-job sh -e -c dockerize -wait tcp://mysql:3306 -timeout 20s

echo "Loading settings $DJANGO_SETTINGS_MODULE"

./manage.py lms migrate

# Create oauth2 apps for CMS SSO
# https://github.com/openedx/edx-platform/blob/master/docs/guides/studio_oauth.rst
./manage.py lms manage_user cms cms@openedx --unusable-password
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.local.overhang.io/complete/edx-oauth2/" \
  --client-id cms-sso \
  --client-secret 4oZHJ9MY9HBh090pMnm0VIil \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso cms
./manage.py lms create_dot_application \
  --grant-type authorization-code \
  --redirect-uris "http://studio.local.overhang.io:8001/complete/edx-oauth2/" \
  --client-id cms-sso-dev \
  --client-secret 4oZHJ9MY9HBh090pMnm0VIil \
  --scopes user_id \
  --skip-authorization \
  --update cms-sso-dev cms


# Fix incorrect uploaded file path
if [ -d /openedx/data/uploads/ ]; then
  if [ -n "$(ls -A /openedx/data/uploads/)" ]; then
    echo "Migrating LMS uploaded files to shared directory"
    mv /openedx/data/uploads/* /openedx/media/
    rm -rf /openedx/data/uploads/
  fi
fi

# Create waffle switches to enable some features, if they have not been explicitly defined before
# Completion tracking: add green ticks to every completed unit
(./manage.py lms waffle_switch --list | grep completion.enable_completion_tracking) || ./manage.py lms waffle_switch --create completion.enable_completion_tracking on

[mariajgrimaldi]

Hi! Thank you so much for the thorough explanation.

The error comes from the eox-core dependency: eox_core/edxapp_wrapper/backends/users_l_v1.py, as it should have because that file doesn't work with the maple release.

Can you test adding:

name: eox-tagging-plugin
version: 0.1.0
patches:
  openedx-common-settings: |
    EOX_TAGGING_GET_ENROLLMENT_OBJECT: "eox_tagging.edxapp_wrappers.backends.enrollment_l_v1"
    EOX_CORE_USERS_BACKEND: "eox_core.edxapp_wrapper.backends.users_m_v1"

If that works I'll be opening a PR in eox-core plugin.

[angetresca-mercadolibre]

Thanks @mariajgrimaldi for your quickly answer! I'm testing this right now, I'll let you know as soon as I have some news

[angetresca-mercadolibre]

@mariajgrimaldi I think the plugin is not working because i'm facing the same issue in eox_core/edxapp_wrapper/backends/users_l_v1.py:

  File "/openedx/venv/lib/python3.8/site-packages/eox_core/edxapp_wrapper/backends/users_l_v1.py", line 36, in <module>
    from openedx.core.djangoapps.user_authn.utils import generate_password  # pylint: disable=import-error,unused-import
ImportError: cannot import name 'generate_password' from 'openedx.core.djangoapps.user_authn.utils' (/openedx/edx-platform/openedx/core/djangoapps/user_authn/utils.py)

Any ideas on how we can configure that setting in tutor?

[mariajgrimaldi]

Last attempt:
can you install eox-core as you're installing eox-tagging? And add that setting:

name: eox-core-plugin
version: 0.1.0
patches:
  openedx-common-settings: |
    EOX_CORE_USERS_BACKEND: "eox_core.edxapp_wrapper.backends.users_m_v1"

Either way, I'll try to replicate this error in my local environment. Thanks again for reporting!

[angetresca-mercadolibre]

Still facing the same issue 😢 @mariajgrimaldi let me know if you can reproduce it

[mariajgrimaldi]

This worked for me 🥳:

name: eox-plugins
version: 0.1.0
patches:
  openedx-common-settings: |
    EOX_TAGGING_GET_ENROLLMENT_OBJECT = "eox_tagging.edxapp_wrappers.backends.enrollment_l_v1"
    EOX_CORE_USERS_BACKEND = "eox_core.edxapp_wrapper.backends.users_m_v1"

Using = instead of : since those settings are copied in the tutor/env/production.py

[angetresca-mercadolibre]

OMG, I can't believe that was the problem! Thanks a lot for your time @mariajgrimaldi ❤️ Now it's working