Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/mediawiki: init service to replace httpd subservice #62748

Merged
merged 3 commits into from Aug 1, 2019

Conversation

@aanderse
Copy link
Contributor

commented Jun 6, 2019

Motivation for this change

Continue to remove httpd.extraSubservices.

See part 1 and part 2 for prior work on the topic. I still need to write release notes for 19.09, but waiting for module review before going ahead with that.

A big thanks to @tstarling of the Wikimedia foundation for all his help on this module over IRC.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 8, 2019

@GrahamcOfBorg test mediawiki

@aanderse aanderse force-pushed the aanderse:mediawiki branch from 716110b to 7c57c87 Jun 9, 2019

@aanderse aanderse marked this pull request as ready for review Jun 9, 2019

@aanderse aanderse requested a review from Infinisil as a code owner Jun 9, 2019

@aanderse aanderse requested a review from samueldr Jun 9, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 9, 2019

@redvers Do you use the mediawiki apache subservice on nixos? Would you mind giving this PR a review?

@nixos-discourse

This comment has been minimized.

Copy link

commented Jun 11, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review-may-2019/3032/15

@redvers

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2019

@redvers Do you use the mediawiki apache subservice on nixos? Would you mind giving this PR a review?

My contribution to this package was all about fixing a security issue - the task of bringing it into a separate module was above my pay grade. I'm sure I'll learn a lot from reading your PR - Thanks for doing it!

There is documentation on nixos.wiki here: https://nixos.wiki/wiki/Mediawiki which we will need to update (and I'm happy to do so).

At first glance it looks like the configuration you generate in this new module isn't compatible with existing configurations so I don't think it will be a seamless migration.

Are you planning on adding a deprecation message to the old subsystem such that it will throw a warning for existing users like: "subSystem is going away, please move to native mediawiki" or somesuch?

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 12, 2019

@redvers My plan is to add an entry to the release notes and write a quick migration guide. If I was able to migrate all httpd subservices into modules before 19.09 I'd try to remove the extraSubservices option but maybe that is being too aggressive so I'll add a deprecation message in 19.09 and entirely remove hopefully for 20.03 - the end result being that either way the user will see a message about it.

Thanks for your offer to write the migration guide! I'll ask again if you are using a mediawiki subservice on NixOS because it would be super valuable to have an actual user of the module for testing and feedback.

Thanks!

@aanderse aanderse force-pushed the aanderse:mediawiki branch from 7c57c87 to c9036b6 Jun 18, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 18, 2019

@GrahamcOfBorg test mediawiki

@aanderse aanderse force-pushed the aanderse:mediawiki branch from c9036b6 to ee700b6 Jun 18, 2019

@redvers

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2019

I'm testing a new installation now - I'll follow up shortly.

@redvers

This comment has been minimized.

Copy link
Contributor

commented Jun 23, 2019

I'm testing a new installation now - I'll follow up shortly.

Looks great from here - thanks for doing this integration - will make life easier in the long run.

@aanderse aanderse force-pushed the aanderse:mediawiki branch 2 times, most recently from 779f14c to c16d081 Jun 23, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 23, 2019

@samueldr Seeing as @redvers is likely the only person currently using this module and they have approved it I'll leave the ball in your court for final review/approval.

@redvers

This comment has been minimized.

Copy link
Contributor

commented Jun 23, 2019

Looks great from here - thanks for doing this integration - will make life easier in the long run.

Just one issue (which is probably just something that should be documented...)

I don't seem to be able to use the maintenance scripts. Maybe I need to specify more than just php and mediawiki in order for it to resolve the include in the default LocalSettings.php.

It contains:

<?php
  return require(getenv('MEDIAWIKI_CONFIG'));
?>
[root@evil:/etc/nixos]# nix-shell -p php mediawiki -I nixpkgs=/root/projects/nixpkgs/
these paths will be fetched (1.30 MiB download, 8.64 MiB unpacked):
  /nix/store/0iiazg67ib2v51h8j1ri279zrscs2ff4-php-7.2.19-dev
  /nix/store/23r0nak96l96ziyilisf4sll8c4bff2r-brotli-1.0.7
  /nix/store/agwiw8h1czv3jdmqcl6q559i48a709ip-openldap-2.4.47-dev
  /nix/store/aza0g3xx6mz0cf7ck0azck0ayfavjfin-apr-1.6.5-dev
  /nix/store/bvhqvwd0744apm57pdbrkix2nhwn22qg-db-5.3.28-dev
  /nix/store/c1cr6g3775d7xv1hkn8q67sw6wfbgnj4-glibc-iconv-2.27
  /nix/store/inxvy25xzk85sqpb709vbpq9c3hvq6fl-apache-httpd-2.4.39-dev
  /nix/store/jhipckj56csw6zpyddrj7m8wg6j4hz28-db-5.3.28-bin
  /nix/store/kkgaacwwijvyslqgchvf72im58rm4f18-apr-util-1.6.1-dev
  /nix/store/lz479anfpnn8jq9q3z9g23mzlrc32ffi-pcre-8.43-bin
  /nix/store/qbvfwwjkaacaqb28ff0rlb9wks9d534g-expat-2.2.6-dev
  /nix/store/vhbm21ns6drmg9xgcpaqlfw88gxhw2rg-pcre-8.43-dev
copying path '/nix/store/aza0g3xx6mz0cf7ck0azck0ayfavjfin-apr-1.6.5-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/23r0nak96l96ziyilisf4sll8c4bff2r-brotli-1.0.7' from 'https://cache.nixos.org'...
copying path '/nix/store/jhipckj56csw6zpyddrj7m8wg6j4hz28-db-5.3.28-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/qbvfwwjkaacaqb28ff0rlb9wks9d534g-expat-2.2.6-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/bvhqvwd0744apm57pdbrkix2nhwn22qg-db-5.3.28-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/c1cr6g3775d7xv1hkn8q67sw6wfbgnj4-glibc-iconv-2.27' from 'https://cache.nixos.org'...
copying path '/nix/store/agwiw8h1czv3jdmqcl6q559i48a709ip-openldap-2.4.47-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/lz479anfpnn8jq9q3z9g23mzlrc32ffi-pcre-8.43-bin' from 'https://cache.nixos.org'...
copying path '/nix/store/kkgaacwwijvyslqgchvf72im58rm4f18-apr-util-1.6.1-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/vhbm21ns6drmg9xgcpaqlfw88gxhw2rg-pcre-8.43-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/inxvy25xzk85sqpb709vbpq9c3hvq6fl-apache-httpd-2.4.39-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/0iiazg67ib2v51h8j1ri279zrscs2ff4-php-7.2.19-dev' from 'https://cache.nixos.org'...

[nix-shell:/etc/nixos]# php /nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/maintenance/removeUnusedAccounts.php
Warning: require(): Filename cannot be empty in /nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/LocalSettings.php on line 2
Fatal error: require(): Failed opening required '' (include_path='/nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pear/console_getopt:/nix/store/6bsmy
npxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pear/mail:/nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pear/mail_mime:/nix
/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pear/net_smtp:/nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pea
r/net_socket:/nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/mediawiki/vendor/pear/pear-core-minimal/src:/nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.3
2.2/share/mediawiki/vendor/pear/pear_exception:.:/nix/store/b4bq1q3gq4dgyhdccbfmvm6a9jzk0rzg-php-7.2.19/lib/php') in /nix/store/6bsmynpxhaml27bgrg2850856hrywdw1-mediawiki-full-1.32.2/share/me
diawiki/LocalSettings.php on line 2
@redvers

This comment has been minimized.

Copy link
Contributor

commented Jun 23, 2019

So, if we can export MEDIAWIKI_CONFIG into a shell automatically somehow, it will work.

I can get it to function like so:

[root@evil:/nix/store]# nix-shell -p php

[nix-shell:/nix/store]# sudo -u mediawiki MEDIAWIKI_CONFIG=/nix/store/wbsb7plrqqgbsa38b73dc5icxy3n0ifw-LocalSettings.php php /nix/store/1hlhp60kh726g0frqh9pvjjy3mp17kjn-mediawiki-1.32.2/share/mediawiki/maintenance/removeUnusedAccounts.php
Remove unused accounts

Checking for unused user accounts...
...found 0.

@aanderse aanderse force-pushed the aanderse:mediawiki branch from c16d081 to 59720c2 Jun 24, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 24, 2019

So, if we can export MEDIAWIKI_CONFIG into a shell automatically somehow, it will work.

    # for easy access to maintenance commands
    environment.variables.MEDIAWIKI_CONFIG = mediawikiConfig;

@redvers Please test and let me know if any other issues come up. Thanks for reporting!

@aanderse aanderse force-pushed the aanderse:mediawiki branch from 59720c2 to ac63643 Jun 28, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jun 29, 2019

@redvers what would be the most common commands used for maintenance (ie. what php files under maintenance/) with mediawiki?

On a side note there has been some changes going on with phpfpm and mysql in master recently that are causing problems for this module, so it probably shouldn't be merged until those are resolved.

@aanderse aanderse force-pushed the aanderse:mediawiki branch 2 times, most recently from 7cf14a7 to 28e4c8b Jul 3, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jul 11, 2019

ping @redvers

@aanderse aanderse referenced this pull request Jul 12, 2019
3 of 10 tasks complete

@aanderse aanderse force-pushed the aanderse:mediawiki branch 2 times, most recently from b9aa8a4 to 2cff55e Jul 23, 2019

@aanderse aanderse force-pushed the aanderse:mediawiki branch from 2cff55e to ebd9067 Jul 24, 2019

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2019

@GrahamcOfBorg test mediawiki

@redvers I have gone back to the way the current mediawiki httpd subservice handles running commands. Subsequent PRs can add more commands if/when needed. As far as I can tell this PR is now ready for merging.

@aanderse

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2019

Merging based on previous input from @redvers and sufficient testing on my own part.

@aanderse aanderse merged commit a1f738b into NixOS:master Aug 1, 2019

18 checks passed

Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
mediawiki on aarch64-linux Success
Details
mediawiki on x86_64-darwin Success
Details
mediawiki on x86_64-linux Success
Details
tests.mediawiki on aarch64-linux Success
Details
tests.mediawiki on x86_64-linux Success
Details

@aanderse aanderse deleted the aanderse:mediawiki branch Aug 1, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.