Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

setup Hitchwiki with ansible #164

Closed
wants to merge 230 commits into from
Closed

setup Hitchwiki with ansible #164

wants to merge 230 commits into from

Conversation

traumschule
Copy link
Contributor

@traumschule traumschule commented Oct 17, 2017 •
edited

The ansible playbook (hitchwiki.yml) is a major step to make Hitchwiki v3 production ready. This branch aims for a stable and easy installation process. For future development see #172.

Build Status

Test it!

git clone https://github.com/traumschule/hitchwiki -b ansible
./scripts/vagrant/install.sh
./scripts/deploy_remote.sh HOST

For details see INSTALL.md and ansible/README.md.

Changelog

Nov 18 Install Mediawiki without errors

Run ansible-playbook hitchwiki.yml until Import Mediawiki content, check debug output and fix the errors (See mw_import.yml)

  • Run Mediawiki install script (LocalSettings.php must be absent, or Cannot modify header information - headers already sent is risen) how to find BOMs backtrace
  • Setup database for several MW extensions (SemanticMediaWiki, AntiSpoof etc.)
  • Pre-populate the AntiSpoof extension's table
  • Create MediaWiki users
  • Import Semantic templates and other MediaWiki special pages
  • the MW update role has several errors, sometimes. automatic error correction would be great.
  • add security release: mediawiki-1.29.2.tar.gz (November 2017)

Nov 13 secure production setup (fixes #142)

Oct 28 deploy beta.hitchwiki.org with ansible

  • Monit: apache, mysql, parsoid, ssh
  • https #127 https: Certbot / lets_encrypt module
  • Look into unix ACL stuff in roles/mw/tasks/main.yml
  • Move configs to yaml file (instead of ini file) setup Hitchwiki with ansible #164 (comment)
  • Test loading settings with shyaml in configs/mediawiki.php and public/robots.txt.php
    (alternatives: spyc and below)
  • merge roles that won't be distributed to different hosts (eg. web mw dev), outsource groups of task to apps specific files, included conditionally (when clause)
  • auomate apache configuration
  • Fix Apache errors
    • Apache: /var/www/public/.htaccess: Invalid command 'php_value' although mod_php is loaded => comment both lines and add them to /etc/php/7.0/cli/php.ini
    • Apache: Could not reliably determine the server's fully qualified domain name although /etc/apache2/sites-enabled/hitchwiki.confdefines it => add ServerName outside of <VirtualHost
    • Apache: index.php MWException from line 1205 of /home/hitchwiki/hitchwiki/public/wiki/includes/WebRequest.php: Unable to determine IP. => fix: add $_SERVER['REMOTE_ADDR'] = 'YOUR IP ADDRESS'; to Local Settings.php
  • Fix Mediawiki import errors
  • Configure Mediawiki logs path, now: $logDir = '/vagrant/logs' (./configs/mediawiki.php)
  • test TLS for staging (see INSTALL.md)
  • run post install scripts
  • add script to update LocalSettings.php

@simison simison added this to the Hitchwiki v3 production ready milestone Oct 17, 2017
simison
simison reviewed Oct 18, 2017
View reviewed changes
INSTALL-vagrant.md Outdated Show resolved Hide resolved
simison
simison reviewed Oct 18, 2017
View reviewed changes
group_vars/vars Outdated
VAGRANT_WIKIDIR: "{{ VAGRANT_ROOTDIR }}/public/wiki"

# Hitchwiki
hw_owners: "{{ HW__general__webserver_user }}:{{ HW__general__webserver_group }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that HW__general__webserver_user were set with bash from settings.ini, but here they might need to be set just as strings? They could be taken out form settings.ini as this doesn't really need to be configurable. www-data:www-data is the default Apache2 user:group.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: I merged them into group_vars/all.
as soon as i figure out, how to include them from a script, they will depend on _settings.sh, or if we decide to deprecate it, from settings.ini.

Copy link
Member

@simison simison Oct 18, 2017 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a bunch of scripts now relying on settings.ini now, although those can probably all be migrated to Ansible.

settings.sh setting up env vars for Ansible could be one trick, although in general I always feel like they are a bit of a blackbox so I try to avoid them.

Bottom line is that there should be just one place for one type of configs. Install/runtime configs can be separate as long as they don't have same configuration multiple times; that's kinda how it is now with configs/vagrant.yml

simison
simison reviewed Oct 18, 2017
View reviewed changes
roles/db/tasks/main.yml Outdated Show resolved Hide resolved
simison
simison reviewed Oct 18, 2017
View reviewed changes
roles/mw/tasks/main.yml Outdated Show resolved Hide resolved
simison
simison reviewed Oct 18, 2017
View reviewed changes
roles/web/tasks/main.yml Outdated
sed -i "s/error_reporting = .*/error_reporting = E_ALL/" /etc/php/7.0/apache2/php.ini
sed -i "s/display_errors = .*/display_errors = On/" /etc/php/7.0/apache2/php.ini
notify:
- restart apache
Copy link
Member

@simison simison Oct 18, 2017 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be off by default (at least in prod) and we have a flag at config we use to control this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to dev as well now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should still be off by default, so that setting $hwDebug=false will hide them.

@simison
Copy link
Member

simison commented Oct 18, 2017

@guaka or @omelnyk do you know how to read variables into Ansible from external config files (yaml/ini/json/whatnot)?

@guaka
Copy link
Contributor

guaka commented Oct 18, 2017 via email

@simison
Copy link
Member

simison commented Oct 18, 2017

Aight!

@traumschule so I guess this line in Mediawiki config needs to use PHP YAML then.

As for bash files, might be we won't have those left after Ansible but if they need access to settings, there's this: https://github.com/0k/shyaml

@simison
Copy link
Member

simison commented Oct 18, 2017

Two other yaml readers for php that we can install via Composer:
https://symfony.com/doc/current/components/yaml.html
https://github.com/mustangostang/spyc/

traumschule added 5 commits October 19, 2017 21:06
ansible: improve roles
4151fa2
Replace settings.ini with settings.yml
d2c139e 
group_vars/all includes
 - configs/settings.yml (link/copy from settings-example.yml)
 - configs/vagrant.yaml

This superseeds and removes the following files:
 - configs/settings-example.ini
 - scripts/_path_resolve.sh
 - scripts/_settings.sh

The following files are replaced by ansible:
 - scripts/server_install.sh
 - scripts/install_funcs.sh
 - scripts/install_parsoid.sh
 - scripts/create_users.sh
 - scripts/install_local.sh
 - scripts/vendor/bash_ini_parser
ansible: improve roles
741d6f3
New settings.yml replaces settings.ini
83efa6e 
    First copy configs/settings-example.yml to settings.yml

    This commit superseeds the following files:
     - configs/settings-example.ini
     - configs/vagrant-example.yaml
     - configs/vagrant.yaml
     - scripts/server_install.sh

   install_local.sh still depends on:
     - scripts/_path_resolve.sh
     - scripts/_settings.sh
     - scripts/vendor/bash_ini_parse
     - scripts/install_funcs.sh
       - scripts/install_parsoid.sh
       - scripts/create_users.sh

    composer.json references
     - scripts/vagrant/install.sh
     - scripts/install_local.sh
     - scripts/clean.sh

    Links
     - group_vars/all -> group_vars/hitchwiki
     - group_vars/hitchwiki -> configs/settigs.yml
     - configs/vagrant-example.yaml -> configs/settings-example.yml
     - configs/settings-example.ini -> configs/settings-example.yml
Add scripts/status.sh
09f0b08
@simison
Copy link
Member

simison commented Oct 20, 2017

Use lets_encrypt module (and replace scripts/cert_selfsigned.sh)

cert_selfsigned.sh is still needed for dev environment, because lets encrypt works only for domains which are publicly accessible.

@simison
Copy link
Member

simison commented Oct 20, 2017

Update migrations: scripts/bot/*

I think it's okay to let these read settings.ini, we'll just know to do that once we need to run the migration bot. Once migration is done, this whole directory is going to get wiped out anyway.

@simison simison moved this from To do to In Progress in New Hitchwiki to Production - Roadmap Nov 15, 2017
@guaka guaka mentioned this pull request Jul 21, 2019
Closed
22 tasks
@guaka
Copy link
Contributor

guaka commented Jul 17, 2020

This week I put together some ansible code to deploy hitchwiki-private to a new server, it's almost done.

@guaka
Copy link
Contributor

guaka commented Jul 17, 2020

it's unfortunate but I don't think this will be (re)used - but please step up if you do want to (re)use some of this work

@guaka guaka closed this Jul 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simison simison simison left review comments

@omelnyk omelnyk Awaiting requested review from omelnyk

Assignees

@traumschule traumschule

Labels
None yet
Projects
No open projects
New Hitchwiki to Production - Roadmap
  
In Progress
Milestone
Hitchwiki v3 production ready
Development

Successfully merging this pull request may close these issues.

production setup
3 participants
@traumschule @simison @guaka