Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generalised caching, more documentation, and other set-up things #29

Merged
merged 10 commits into from Oct 24, 2016
+2,365 −47
Merged

Generalised caching, more documentation, and other set-up things #29

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0574bef
Generalised caching, more documentation, and other set-up things
samwilson Oct 13, 2016
2b02b92
Coding standards compliance
samwilson Oct 13, 2016
0573a9a
Redis isn't a hard requirement now
samwilson Oct 13, 2016
42865ee
No local DB, composer.lock added, and cache info updated
samwilson Oct 13, 2016
14047dc
Number fix.
samwilson Oct 14, 2016
e0f1d74
Fix errors in example Oauth environment variables
samwilson Oct 21, 2016
8be5bf6
Merge remote-tracking branch 'origin/master' into local-dev
samwilson Oct 21, 2016
0b33d96
Merge remote-tracking branch 'origin/master' into local-dev
samwilson Oct 21, 2016
8acac8b
Don't require intl extension, but check for it before use
samwilson Oct 22, 2016
5e28336
Correct test for existance
samwilson Oct 22, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 16 additions & 0 deletions .env_example
View file
Open in desktop
@@ -0,0 +1,16 @@
; Oauth parameters.
OAUTH_CONSUMER_TOKEN =
OAUTH_SECRET_TOKEN =
OAUTH_ENDPOINT = 'https://meta.wikimedia.org/w/index.php?title=Special:OAuth'
OAUTH_REDIR = 'https://meta.wikimedia.org/wiki/Special:OAuth/authenticate?'
OAUTH_CALLBACK = 'oob'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is oob?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'Out Of Band', which isn't really correct because we do actually use a callback. (OOB usually means e.g. a desktop application is requesting the auth.) In our case, the callback is fixed by the consumer registration on Meta, so actually this will be ignored; it does have to be something though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Yeah for us it looks like we need a full URL to /copypatrol/oauth/callback. On my local I have the app running at localhost/copypatrol so I use http://localhost/copypatrol/oauth/callback and this works. We might suggest something similar here


; Database connection details. The same username and password is used for both data sources.
DB_DSN_ENWIKI = "mysql:host=127.0.0.1;port=4711;dbname=enwiki_p"
DB_DSN_PLAGIABOT = "mysql:host=127.0.0.1;port=4711;dbname=s51306__copyright_p"
DB_USER =
DB_PASS =

; Caching system. If a Redis host is not defined, a local Filesystem cache will be used instead.
REDIS_HOST = "tools-redis"
REDIS_PORT = 6379
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to add SLIM_MODE = 'development' here, or at least I have on my local. Looking at the core Slim code I'm not convinced this does anything in particular by itself, but we might (or should) use it in other parts of the app, e.g. https://github.com/wikimedia/CopyPatrol/blob/master/src/Controllers/CopyPatrol.php#L364

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. At the moment we're determining staging with $slim->view->set( 'staging', strpos( $rootUri, 'plagiabot' ) ); which could maybe be changed to a mode member of the container that uses SLIM_MODE (or APP_ENV? That seems a common name in some PHHP circles...).

And there's LOG_LEVEL and LOG_FILE too, which are good to set for development, to maybe include in this file.

3 changes: 2 additions & 1 deletion .gitignore
View file
Open in desktop
@@ -1,5 +1,6 @@
.DS_Store
.lighttpd.conf
.htaccess
.viminfo
.my.cnf
.env
Expand All @@ -8,6 +9,7 @@
*.log
tmp/
/vendor/
/cache/*
src/Less/cache/*
fill_patroller/
flask-mwoauth/
Expand All @@ -21,5 +23,4 @@ replica.my.cnf
service.manifest
restart_webservice.sh
deploy.sh
composer.lock
!.gitkeep
39 changes: 25 additions & 14 deletions README.md
View file
Open in desktop
Expand Up @@ -3,23 +3,34 @@ This is a web interface for [Plagiabot's Copyright RC feed](https://en.wikipedia

[![Build Status](https://travis-ci.org/wikimedia/CopyPatrol.svg?branch=master)](https://travis-ci.org/wikimedia/CopyPatrol)

#### To test locally:
1. Run `composer install` and `composer update` after cloning the repository
2. Setup your .env file with the following params:
* OAUTH_CONSUMER_TOKEN
* OAUTH_SECRET_TOKEN
* OAUTH_ENDPOINT
* OAUTH_REDIR
* OAUTH_CALLBACK
* DB_DSN_PLAGIABOT
* DB_DSN_ENWIKI
* DB_USER
* DB_PASS
3. Rewrite your routing locally, if needed
## To install locally
1. Clone the repository and run `composer install`.
2. Edit the `.env` file that was created by composer.
1. Get Oauth tokens by registering a new consumer on Meta
at [Special:OAuthConsumerRegistration](https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration).
2. To use Redis caching, also add `REDIS_HOST` and `REDIS_PORT`;
without these, a local filesystem cache will be used.
3. Make the `cache/` directory writable by the web server.
4. Rewrite your routing, if needed.
For Lighttpd, use this in your `.lighttpd.conf`:
```
url.rewrite-if-not-file += ( "(.*)" => "/copypatrol/index.php/$0" )
```
Or for Apache, this (in `.htaccess` at the root of the project):
```
DirectorySlash Off
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^public_html(.*)$ public_html/index.php$1 [L]
```
5. Open up an SSH tunnel to access the databases on Tool Labs (substitute your own username).
```
$ ssh -L 4711:enwiki.labsdb:3306 YOU@tools-login.wmflabs.org -N
```

This application makes of use the [Wikimedia-slimapp](https://github.com/wikimedia/wikimedia-slimapp) library and uses Twig as its templating engine.

##### To add a new translation message:
## To add a new translation message:
1. Add it to en.json
2. Update the qqq.json documentation accordingly
3. Call it in Twig as `{{ '<message-key>'|message }}`. If the message contains any HTML, you'll need to append the `|raw` filter after `message`.
Expand Down
0 src/Less/cache/.gitkeep → cache/.gitkeep
View file
Open in desktop
File renamed without changes.
12 changes: 11 additions & 1 deletion composer.json
View file
Open in desktop
@@ -1,11 +1,13 @@
{
"license": "GPL-3.0+",
"require": {
"ext-intl": "*",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we need the intl extension for?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's for the number formatting I think. I did not know we could require PHP extensions through composer, so I ended up manually installing/enabling it my php.ini

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's for NumberFormatter. Composer can't install extensions but it does alert people to the fact that they are required.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of requiring intl (which might require someone to spend half an hour upgrading their PHP), let's just check for the existence of NumberFormatter. After all, number formatting isn't critical to the tool's functionality:

if ( class_exists( 'NumberFormatter' ) ) {
    $formatter = new \NumberFormatter( $locale, \NumberFormatter::DECIMAL );
    // Get separator symbols (include decimal as it is required and we might use it at some point)
    $decimal = $formatter->getSymbol( \NumberFormatter::DECIMAL_SEPARATOR_SYMBOL );
    $thousands = $formatter->getSymbol( \NumberFormatter::GROUPING_SEPARATOR_SYMBOL );
} else {
    $decimal = '.';
    $thousands = ',';
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! And we can make it simpler even, because the Twig defaults are the same as you've set there (dot and comma).

"oyejorge/less.php": "~1.5",
"mediawiki/oauthclient": "~0.1",
"wikimedia/slimapp": "dev-master",
"addwiki/mediawiki-api-base": "~2.1",
"wikimedia/simplei18n": "~1.0"
"wikimedia/simplei18n": "~1.0",
"tedivm/stash": "^0.14"
},
"autoload": {
"psr-4": {
Expand All @@ -23,5 +25,13 @@
"parallel-lint . --exclude vendor",
"phpcs -p"
]
},
"script": {
"post-install-cmd": [
"php -r \"file_exists('.env') || copy('.env_example', '.env');\""
],
"post-update-cmd": [
"php -r \"file_exists('.env') || copy('.env_example', '.env');\""
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super cool!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just a shame that there's no post-update-or-install-cmd! :-)

}
}