Doesn't work out of the box

[nicorikken]

Reading the readme, trying to get the Docker setup to run, I was presented with the following error:

$ make publish-commit
Pulling db (postgres:latest)...
latest: Pulling from library/postgres
386a066cd84a: Pull complete
e6dd80b38d38: Pull complete
9cd706823821: Pull complete
40c17ac202a9: Pull complete
7380b383ba3d: Pull complete
538e418b46ce: Pull complete
c3b9d41b7758: Pull complete
dd4f9522dd30: Pull complete
920e548f9635: Pull complete
628af7ef2ee5: Pull complete
211678575a06: Pull complete
Digest: sha256:3da198a1846d1fa6cf55978c8326d5c7e801155843c469ce9213cdbb25b5ae33
Status: Downloaded newer image for postgres:latest
Creating freedomvote_db_1

ERROR: for db  Cannot start service db: driver failed programming external connectivity on endpoint freedomvote_db_1 (47b8bde95c64195ff573811f6da543766119b26c8cd3b7c0e6c1e1f2387dbafa): Error starting userland proxy: Bind for 0.0.0.0:5432: unexpected error Uwt.Uwt_error(Uwt.EADDRINUSE, "listen", "")
ERROR: Encountered errors while bringing up the project.
make: *** [docker-init] Error 1

I tried running the docker-compose up directy, but after building the application, it too fails on the db:

$ docker-compose up
Creating freedomvote_db_1

ERROR: for db  Cannot start service db: driver failed programming external connectivity on endpoint freedomvote_db_1 (8be7aa292667245456c55f91183a9c62c55b61fd026b5549103b961a86cd5e8e): Error starting userland proxy: Bind for 0.0.0.0:5432 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.

I guess the db Docker image has changed. I'll look into it. But if somebody now the fix by heart, please let me know.

[anehx]

Since this works perfectly on my own machine, it could be a problem of your machine. Could it be that the postgres port is already in use? Please post the output of sudo netstat -tulpn | grep 5432.

[nicorikken]

Thanks, I figured this one out now, I indeed had Postgres running already. I now upped it one number in both the docker-compose "5433:5432" and in the settings.py: 'PORT' : '5433. Now it is working so far.

Now I get another stacktrace, about the sanitizer module:

$ make docker-init docker
Starting freedomvote_db_1
CREATE TABLE
ALTER TABLE
COPY 0
ALTER TABLE
CREATE INDEX
Traceback (most recent call last):
  File "app/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute
    django.setup()
  File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/site-packages/djangocms_text_ckeditor/models.py", line 13, in <module>
    from .html import clean_html, extract_images
  File "/usr/local/lib/python2.7/site-packages/djangocms_text_ckeditor/html.py", line 8, in <module>
    from html5lib import sanitizer, serializer, treebuilders, treewalkers
ImportError: cannot import name sanitizer
make: *** [docker-init] Error 1

I'll see if I can find the solution to this issue.

[nicorikken]

Trying to trace the root of the issue, it seems that html5lib has broken their API, and therefore some of the later versions of djangocms_text_ckeditor have either explicitly pinned the html5lib version or have change the inner working.
Related issues:

• cannot import name 'sanitizer' html5lib/html5lib-python#277
• html5lib version 0.999999999 (latest) does not work with djangocms-text-ckeditor django-cms/djangocms-text-ckeditor#344
• sanitizer import error due to changes in html5lib django-cms/djangocms-text-ckeditor#336
• cannot import name 'sanitizer' html5lib/html5lib-python#277
• Sanitizing filter broken in 0.90 html5lib/html5lib-python#72

It seems a version update like tanzquotient/website@718a92a is needed.

[anehx]

Please pull and rebuild the image, since this issue is already solved by a61ab6a

[nicorikken]

Still issues here. The 5 second sleep was not long enough on my Macbook. I assume Docker4Mac is resulting in a performance penalty. Lengthening the sleep to 10s helps. A neater solution would be to add the commonly used wait-for-it.sh script inside the Docker container, to wait for the other service to become available.

Unfortunately the Django container is stopping directly, so that the latter commands will not execute:

$ make docker-init
Creating freedomvote_db_1
Creating freedomvote_web_1
CREATE TABLE
ALTER TABLE
COPY 0
ALTER TABLE
CREATE INDEX
Error response from daemon: Container 05d0b9ac1790f76828306915da66fb19f5377ad743a98106c7e22aff0e3f2f5e is not running
make[1]: *** [docker-migrate] Error 1
make: *** [docker-init] Error 2

I have reverted the previous Makefile version and added the needed sleep fixes. Than I'm back to square one, despite the updated requirements.txt:

$ make docker-init
Creating freedomvote_db_1
CREATE TABLE
ALTER TABLE
COPY 0
ALTER TABLE
CREATE INDEX
Traceback (most recent call last):
  File "app/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 312, in execute
    django.setup()
  File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 198, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/site-packages/djangocms_text_ckeditor/models.py", line 13, in <module>
    from .html import clean_html, extract_images
  File "/usr/local/lib/python2.7/site-packages/djangocms_text_ckeditor/html.py", line 8, in <module>
    from html5lib import sanitizer, serializer, treebuilders, treewalkers
ImportError: cannot import name sanitizer
make: *** [docker-init] Error 1

Going back to the latest state, I tried to keep the container running. Normally when running a Docker container I would at the -t flag, to mean tty: false. Now with Docker Compose I tried adding the tty: false option in the docker-compose.yml. Then however make docker-migrate fails, without any info:

$ make docker-init
Creating freedomvote_db_1
Creating freedomvote_web_1
CREATE TABLE
ALTER TABLE
COPY 0
ALTER TABLE
CREATE INDEX
make[1]: *** [docker-migrate] Error 137
make: *** [docker-init] Error 2

(in all of this I did reintroduce my Postgres port remap, including for the psql command).

Maybe I'll ditch the Docker hassle altogether and just try running it natively.

[nicorikken]

Alright, got it working natively. I was first presented with some error on the backend because the /de?edit url or something couldn't be parsed properly. Navigating to the /admin URL solved that. I'll take a look to see how it might look with some sample data.

[nicorikken]

The Docker issue still stands for me. Without Docker it is working fine now. I've already done a rough translation to Dutch, and I have some small improvements in mind. Expect a PR soon.

[nicorikken]

I just pushed my attempt to fix Docker on my fork/clone: nicorikken@7576a0a

[nicorikken]

I'm now in the process of getting it to run on my newly acquired Libreboot laptop with Debian testing. I had to read the Debian manual to find out how best to create the user and database, but that is all working now.

The Pillow dependency for version 3.1.0 would not want to install (a friend had the same issue on CentOS), so I'm now trying to continue with version 3.4.2 which was installed correctly (perhaps because I also installed the corresponding Debian package).

Having provided a Dutch (NL) translation, Django was already trying to write stuff in the database, but without any success:

Applying core.add_core_statistic_indexes... OK
  Applying core.rename_core_politician_unique_url... OK
  Applying core.0002_auto_20150401_0929... OK
  Applying core.0003_auto_20150521_0856... OK
  Applying core.0004_politician_user... OK
  Applying core.migrate_party_user...Traceback (most recent call last):
  File "app/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/nico/.local/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/home/nico/.local/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/nico/.local/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/nico/.local/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
    output = self.handle(*args, **options)
  File "/home/nico/.local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 221, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 147, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 181, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/nico/Software/freedomvote/app/core/migrations/migrate_party_user.py", line 18, in add_user_for_parties
    for party in parties:
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/models/query.py", line 238, in iterator
    results = compiler.execute_sql()
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 829, in execute_sql
    cursor.execute(sql, params)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/nico/.local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column core_party.name_nl does not exist
LINE 1: ..., "core_party"."name_fr", "core_party"."name_it", "core_part...

Perhaps my language file is not complete. I looked at the tools/docker/cache_table.sql but that didn't seem to contain the initalization of the colum itself, so I assume the generation somewere is failing. I did manage to get it working by disabling the nl language for now.

I was then presented with the same error page I wrote about earlier:

Somehow the initial redirect page does not exist. After moving to http://localhost:8000/admin/ everything is working as expected.

Do I need to split this out into some seperate issues to tackle and track?

[nicorikken]

After the initial boot, doing the steps to compile and migrate the messages did result in a Dutch feature on the website.
The language selection in the top menu bar however, does not seem to work, neither in the admin panel, nor on the public facing pages. As if there is nothing actually changing.

[anehx]

I tried to fix this as generically as I could. The docker containers are now using wait-for-it.sh which should erase the "sleep problem" (see a8a8cbb).

If you map your DB to another port, pass DB_PORT=1234 as argument for the docker-init command (e.g $ docker-init DB_PORT=1234).

Also I forgot to mention how to configure the DB cleaner with an .ini file instead of changing settings.py (see https://github.com/adfinis-sygroup/freedomvote#the-hard-way).

I was able to fix the translation issue by fixing a wrong migration (see 8e507d9).

You can import politicians with a custom management command. E.g ./manage.py politician_import /tmp/somecsvfile.csv. The order of the attributes in the CSV file is:

• first name
• last name
• email
• state (name)
• party (shortname)
• user (username)
• is the politician is already a member of the parliament (1 for yes, 0 for no)

User, party and state have to exist in the DB in order to import the politician.

The problem you had with the non-existing startpage is, because you need to add it first, since it is a page of the CMS. You can add the "Home" site at this URL.

Thanks for your effort! I hope everything works as expected. Otherwise, let me know!

[nicorikken]

Thanks for picking up on this. Nice improvements and I like the hints on the csv import. Does this imply I can also do a user_import, party_import and so on? I was thinking of proposing a set of dummy-data to showcase the app. I guess I could do an SQL import as well.
Tonight I'll be hacking on this again, and I'll see if I can try out your improvements.

[anehx]

There is a user_import management command but none for parties.

The user_import expects the csv as following:

• username
• password

[nicorikken]

I've now seen the importing tools in app/core/management/commands/. Also I've written down a couple of concrete issues which could improve the current state of Freedomvote. Maybe I can make some improvements, but tthis stack is new for me, so I have quite some catching up to do.

[nicorikken]

Most of the issues here have been addressed, so I'll close this issue.