2.12 upgrade issues

[stefk]

Hi,

I'm having a hard time upgrading from 2.11 to 2.12. Since there's a few things in this process I'm not familiar with, I might be doing noob mistakes, but here are the problems I encountered so far:

1. Usual docker workflow suggests to replace the current container with an up-to-date version and then perform upgrade commands, however that's not possible in this case due to the Postgres 9.4/9.6 bump (BTW was there a particular reason to make that upgrade?). This isn't strictly related to weblate but I thought it would be worth mentionning because there's no way to know beforehand that a Postgres upgrade is required and because that operation is quite painful (Upgrading between major versions? docker-library/postgres#37).

2. If I use the 2.12-2 version of this repo and just keep Postgres version to 9.4 (by overriding the corresponding line in docker-compose.yml), I can get the containers up and running, but any attempt to access the weblate container with docker-compose exec makes it crash. In this case, it seems to be necessary to run the commands from outside, like the cron tasks described in https://docs.weblate.org/en/latest/admin/deployments.html#maintenance-tasks:

   docker-compose run --rm weblate migrate

   (This could be a good addition to the weblate/docker doc)

3. The command above seems to be successful:

   Postgres is up
   Operations to perform:
     Apply all migrations: accounts, admin, auth, authtoken, contenttypes, gitexport, lang, screenshots, sessions, sites, social_django, trans
   Running migrations:
     Applying accounts.0025_auto_20170211_1609... OK
     Applying accounts.0026_profile_special_chars... OK
     Applying trans.0072_auto_20170209_1234... OK
     Applying trans.0073_auto_20170209_1359... OK
     Applying trans.0074_auto_20170209_1412... OK
     Applying screenshots.0001_initial... OK
     Applying screenshots.0002_auto_20170215_0849... OK
     Applying screenshots.0003_auto_20170215_1633... OK
     Applying trans.0075_auto_20170215_1750... OK
   

   However, the app is still not accessible and I can see this exception in the logs:

   weblate_1   |   File "/usr/local/bin/weblate", line 11, in <module>
   weblate_1   |     sys.exit(main())
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/weblate/runner.py", line 33, in main
   weblate_1   |     execute_from_command_line(argv)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
   weblate_1   |     utility.execute()
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 359, in execute
   weblate_1   |     self.fetch_command(subcommand).run_from_argv(self.argv)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 294, in run_from_argv
   weblate_1   |     self.execute(*args, **cmd_options)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 345, in execute
   weblate_1   |     output = self.handle(*args, **options)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 83, in handle
   weblate_1   |     executor = MigrationExecutor(connection, self.migration_progress_callback)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 20, in __init__
   weblate_1   |     self.loader = MigrationLoader(self.connection)
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 52, in __init__
   weblate_1   |     self.build_graph()
   weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 268, in build_graph
   weblate_1   |     raise exc
   weblate_1   | django.db.migrations.exceptions.NodeNotFoundError: Migration trans.0075_auto_20170215_1750 dependencies reference nonexistent parent node (u'screenshots', u'0002_auto_20170215_0849')
   

4. From this point I'm not sure what I'm supposed to do, but running weblate migrate once again yields the following output, that you might find interesting:

   No migrations to apply.
   Your models have changes that are not yet reflected in a migration, and so won't be applied.
   Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
   

   Executing the suggested makemigrations command also ends up with an error:

    Migrations for 'trans':
    usr/local/lib/python2.7/dist-packages/weblate/trans/migrations/0076_auto_20170313_1125.py:
       - Alter field check on check
       - Alter field commit_message on subproject
       - Alter field vcs on subproject
    Traceback (most recent call last):
      File "/usr/local/bin/weblate", line 11, in <module>
        sys.exit(main())
      File "/usr/local/lib/python2.7/dist-packages/weblate/runner.py", line 33, in main
        execute_from_command_line(argv)
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
        utility.execute()
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 359, in execute
        self.fetch_command(subcommand).run_from_argv(self.argv)
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 294, in run_from_argv
        self.execute(*args, **cmd_options)
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/makemigrations.py", line 192, in handle
    self.write_migration_files(changes)
      File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/makemigrations.py", line 228, in write_migration_files
    with open(writer.path, "wb") as fh:
    IOError: [Errno 13] Permission denied: u'/usr/local/lib/python2.7/dist-packages/weblate/trans/migrations/0076_auto_20170313_1125.py'
   

Any help would be greatly appreciated!

[nijel]

The motivation for PostgreSQL upgrade was better performance in some cases. Certainly it's not pushing for existing users to upgrade (especially if the installation is not really big), but new users should start on this version. I have no clue how to better express this....

As for migrations:

• You can ignore the warning, see Why do I get warning about not reflected changes on database migration?
• The error seems to be from some older migration run, it seems as if screenshot app would not be installed. However all migrations did run okay, so I assume this is really not a problem.

What error do you get if you enable debug mode (setting WEBLATE_DEBUG=1)?

[stefk]

What error do you get if you enable debug mode (setting WEBLATE_DEBUG=1)?

From the web interface, I always get a 502 response. The error in the logs is the same whatever the value of WEBLATE_DEBUG is:

weblate_1   | Traceback (most recent call last):
weblate_1   |   File "/usr/local/bin/weblate", line 11, in <module>
weblate_1   |     sys.exit(main())
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/weblate/runner.py", line 33, in main
weblate_1   |     execute_from_command_line(argv)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
weblate_1   |     utility.execute()
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 359, in execute
weblate_1   |     self.fetch_command(subcommand).run_from_argv(self.argv)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 294, in run_from_argv
weblate_1   |     self.execute(*args, **cmd_options)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 345, in execute
weblate_1   |     output = self.handle(*args, **options)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 83, in handle
weblate_1   |     executor = MigrationExecutor(connection, self.migration_progress_callback)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 20, in __init__
weblate_1   |     self.loader = MigrationLoader(self.connection)
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 52, in __init__
weblate_1   |     self.build_graph()
weblate_1   |   File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/loader.py", line 268, in build_graph
weblate_1   |     raise exc
weblate_1   | django.db.migrations.exceptions.NodeNotFoundError: Migration trans.0075_auto_20170215_1750 dependencies reference nonexistent parent node (u'screenshots', u'0002_auto_20170215_0849')

[nijel]

This seems to happen when there are multiple versions of the weblate container running. To fix that, shutdown all (especially the run) containers by docker-compose down.

I've now improved documentation to cover upgrading (the manual migrate step is not needed).

Besides that, the docker-compose has been just changed in order to use prebuilt image from docker hub to avoid need to build it...

[stefk]

It's all good now! Again, thanks a lot for your support 👍