Showing with 21 additions and 414 deletions.
  1. +1 −1 .travis.yml
  2. +1 −216 README-compose.md
  3. +19 −39 README.md
  4. +0 −158 docker-compose.yml
2 .travis.yml
View file
@@ -40,7 +40,7 @@ stages:
notifications:
irc:
channels:
- "chat.freenode.net#wikidata-feed"
- "libera.chat#wikidata-feed"
template:
- "%{repository_slug} (%{branch} - %{commit} %{commit_message} : %{author}): %{message}"
- "Change view : %{compare_url}"
217 README-compose.md
View file
@@ -1,216 +1 @@
## Installing a stand-alone Wikibase and most used services

This repository contains an EXAMPLE docker-compose file that can be used with the wikibase docker images on https://hub.docker.com/r/wikibase
Following these instructions, you'll be able to install:
- A stand-alone instance of Wikibase – the collaborative structured data engine behind [Wikidata](https://wikidata.org/)
- A fully functional SPARQL endpoint and query service, complete with a data visualization frontend and query helper (similar to the [Wikidata Query Service](https://query.wikidata.org/)).
- ElasticSearch, for improved serach
- Quickstatements, a bulk editing tool

**RAM Required:** This example requires 4GB+ of RAM. If you are running Docker for Windows or Mac and have containers run within a VM (the default) make sure you set your VM memory allocation to 4GB or more. If you don't do this everything will probably fail.

### 1) Installing Docker and docker-compose

This is the first step and often the hardest, as every operating system has a different way of installing Docker.

Most setups will come with docker-compose.
If yours doesn't then you can get it at https://docs.docker.com/compose/install/

#### Linux

1. Download and Install Docker Engine-Community using the appropriate instructions for your distro from https://docs.docker.com/install/
2. Make sure you read the instructions regarding using sudo/root or creating a group for docker.

#### Windows

##### Pro, with Hyper-V virtualization

1. Download and Install - Docker Desktop https://www.docker.com/products/docker-desktop
2. Turn on virtualization in BIOS settings (if not already done)
3. Open Docker advanced settings and increase the memory limit for the Virtual machine to more than 4GB

##### Not Pro, or without Hyper-V virtualization

1. Download and Install- Docker Toolbox https://docs.docker.com/toolbox/toolbox_install_windows/
2. Increase your virtual machine memmory allocation
3. Note: You may have to use the IP address of your VM machine once setup, not localhost

#### Mac

1. Download and Install - Docker Desktop https://www.docker.com/products/docker-desktop
2. Open Docker advanced settings and increase the memory limit for the Virtual machine to more than 4GB

### 2) Download the docker-compose.yml file example

This file is only meant as an example, your modification might be needed.

#### If you have the wget command

```
wget https://raw.githubusercontent.com/wmde/wikibase-docker/master/docker-compose.yml
```

#### If you do not

If you don't have wget then just download the file from the web and save it into your working directory with the name docker-compose.yml

### 3) Downloading images and starting the containers

**Pulling / updating the images**

The below command will download the versions of the docker images used in the docker-compose example.

```
docker-compose pull
```

**Starting the containers**

The below command will start several containers but hide the output. If you want to see what is happening see the section on viewing logs below.

```
docker-compose up -d
```

**Viewing container output**

The below command will output a stream of logs from all containers

```
docker-compose logs -f
```

### 4) Wait and check

The setup will take a short while and includes automatic steps such as:
- Initializing Blazegraph storage
- Initializing MySql storage
- Initializing Elasticsearch storage
- Installing Mediawiki and Wikibase database tables
- Creating oauth details for Quickstatements

You can check to see if all containers are running with the below command:

```
docker-compose ps
```

If any appear to not be running then something has likely gone wrong.
Take a look at the troubleshooting section at the bottom of this readme.

Once setup has complete you should be able to naviagte to the services.

Note: Docker Toolbox users, you may have to use the IP address of your virtual machine instead of localhost.

Access the following hosts:
- [Wikibase @ http://localhost:8181](http://localhost:8181)
- [Query Service UI @ http://localhost:8282](http://localhost:8282)
- [Query Service Backend (Behind a proxy) @ http://localhost:8989/bigdata/](http://localhost:8989/bigdata/)
- [Quickstatements @ http://localhost:9191](http://localhost:9191)

If you want to access the balzegraph SPARQL endpoint directly for writing you need to bypass the read only proxy.
You will need to make a change similar to this: https://github.com/wmde/wikibase-docker/pull/16/files
You will then be able to access the query service on localhost for writing.
- [Query Service Backend (Direct) @ http://localhost:8999/bigdata/](http://localhost:8999/bigdata/)

### 4) Other commands

**Stopping the containers (keeps data and containers)**

```
docker-compose stop
```

**Removing the containers (keeps data, removes containers)**

This will keep all data stored by MySQL, Mediawiki and the Query Service in Docker volumes.

```
docker-compose down
```

**Removing the containers and data**

**WARNING:** this will remove ALL of the data you had added to MediaWiki, Wikibase, the QueryService and ElasticSearch.

```
docker-compose down --volumes
```

### 5) Interacting with Wikibase

A default user is created on setup. Username "WikibaseAdmin", Password "WikibaseDockerAdminPass".

This is highly insecure, so if your instance is public please change the password in your docker-compose.yml before setup, or after setting up using MediaWiki.

#### Creating content and querying it

You can start creating items and properties to start populating your Wikibase instance via these special pages
- [Create a new item](http://localhost:8181/wiki/Special:NewItem)
- [Create a new property](http://localhost:8181/wiki/Special:NewProperty)

Once you've created your first items and statements, you'll be able to query them and visualize the results through the [Query Service UI](http://localhost:8282).

#### Exporting data as a JSON or RDF dump

Get a JSON dump from wikibase:

```docker-compose exec wikibase php ./extensions/Wikibase/repo/maintenance/dumpJson.php```
Get an RDF dump from wikibase:
```docker-compose exec wikibase php ./extensions/Wikibase/repo/maintenance/dumpRdf.php```
### 6) Managing data
#### Backing up data from Docker volumes
All data for Wikibase and the Query Service is stored in Docker volumes. You can create compressed copies of these volumes to use as backups or to hand off to other users.
Volume backups will only work if you use the same image when restoring / using the backup data.
If you are backing up your mysql data you may also want to just take an SQL dump.
You can see all Docker volumes created by using the following command:
```
docker volume ls | grep wikibase-docker
```
You can grab a zip of each volumes by doing the following:
```
docker run -v wikibase-docker_mediawiki-mysql-data:/volume -v /tmp/wikibase-data:/backup --rm loomchild/volume-backup backup mediawiki-mysql-data
```
You, or someone else, can then restore the volume by doing the following:
```
docker run -v wikibase-docker_mediawiki-mysql-data:/volume -v /tmp/wikibase-data:/backup --rm loomchild/volume-backup restore mediawiki-mysql-data
```
#### Backing up data using mysqldump
If using volume-backup for the database does not work because of InnoDB tables
([check here](https://dev.mysql.com/doc/refman/8.0/en/backup-methods.html)),
you can achieve data backup using mysqldump.
##### Backing up with mysqldump
```
docker exec wikibase-docker_mysql_1 mysqldump -u wikiuser -psqlpass my_wiki > backup.sql
```
##### Restoring from mysqldump backup file
```
docker exec wikibase-docker_mysql_1 mysql -u wikiuser -psqlpass my_wiki < backup.sql

```
## Troubleshooting
* [I am on linux and I don't want to run docker as root!](https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo#477554)
* Everything seemed to stop and nothing seems to work
* The docker-compose setup requires more than 4GB of available RAM to start.
* I am using Docker Toolbox and can't access localhost:8181 (or anything else on localhost)
* You probably need to connect to the IP address of the virtual machine
See https://www.mediawiki.org/wiki/Wikibase/Docker
58 README.md
View file
@@ -1,53 +1,33 @@
## wikibase-docker
## LEGACY: wikibase-docker

This repo contains [Docker](https://www.docker.com/) images needed to setup a local Wikibase instance and a query service.
**This repository will no longer be maintained by WMDE.**

Each image contained within this repo has its own README with more detailed information:
Please see the announcement for new WMDE maintained docker images https://lists.wikimedia.org/hyperkitty/list/wikibaseug@lists.wikimedia.org/thread/WW4LZJINT3PIG3DOYKTXIWVP3WAKWXCT/
Along with docs https://www.mediawiki.org/wiki/Wikibase/Docker

Image name | Description | README
------------------------ | ------------- | ----------
[`wikibase/wikibase`](https://hub.docker.com/r/wikibase/wikibase) | MediaWiki with the Wikibase extension| [README](https://github.com/wmde/wikibase-docker/blob/master/wikibase/README.md)
[`wikibase/wdqs`](https://hub.docker.com/r/wikibase/wdqs) | Blazegraph SPARQL query service backend | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs/README.md)
[`wikibase/wdqs-proxy`](https://hub.docker.com/r/wikibase/wdqs-proxy) | Proxy to make the query service READONLY and enforce query timeouts | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs-proxy/README.md)
[`wikibase/wdqs-frontend`](https://hub.docker.com/r/wikibase/wdqs-frontend) | UI for the SPARQL query service | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs-frontend/README.md)
[`wikibase/quickstatements`](https://hub.docker.com/r/wikibase/quickstatements) | UI to add data to Wikibase | [README](https://github.com/wmde/wikibase-docker/blob/master/quickstatements/README.md)

### Docker compose example

This repo contains an EXAMPLE docker compose setup for Wikibase (specified in the [docker-compose.yml](docker-compose.yml) file) that combines the images described above to set up a fully-featured local Wikibase environment. You can use that file as a base to build a custom environment tailored to your needs.

To try it out, make sure you have Docker installed, then just clone this repository, enter its directory and run `docker-compose up -d`. Once that step completes, you should have:
- a MediaWiki wiki fully configured with [Wikibase](https://www.mediawiki.org/wiki/Wikibase), available at http://localhost:8181
- a [Wikidata Query Service](https://www.mediawiki.org/wiki/Wikidata_Query_Service) instance available at http://localhost:8282
- a [QuickStatements](https://www.wikidata.org/wiki/Help:QuickStatements) instance, available at http://localhost:9191

Note that the Wikibase instance has no data; no items or properties. To add some data, use the `Special:NewItem` and `Special:NewProperty` pages in the local wiki; then you can add statements to the added items using the properties you defined.

For more information about this example enviroment, please see the [README-compose.md](https://github.com/wmde/wikibase-docker/blob/master/README-compose.md) file.
-----

### Issue tracking

We use [Phabricator to track
issues](https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wikibase-containers). See the [list of current issues](https://phabricator.wikimedia.org/maniphest/?project=wikibase-containers&statuses=open&group=none&order=newest#R).
Please do not file issues against these legacy docker images.

### Development

New images will be build and automatically pushed to docker hub when merged on master.
Builds on branches and in PRs will not be pushed.
These images are no longer developed.

Each image directory contains a README with a separate Development section for how updates of that image generally work.
### Further reading

The following steps relate to all images:
- [Docker Wikibase Install Docs](https://www.mediawiki.org/wiki/Wikibase/Docker)
- [Announcement of new WMDE maintained docker images & releases](https://lists.wikimedia.org/hyperkitty/list/wikibaseug@lists.wikimedia.org/thread/WW4LZJINT3PIG3DOYKTXIWVP3WAKWXCT/)

- Update the various .travis/build-deploy.sh scripts to correctly tag the images that are being built, including updating the latest tag.
- Update .travis.yml to reflect new or removed images
### Old Repo Guide

### Further reading
Each legacy image contained within this repo has its own README with more detailed information:

For beginner-friendly light reading on the subject of using these images / this repo, check out the posts under [this tag](https://addshore.com/tag/wikibase-docker/), specifically:
- [Announcement of the creation of these images](https://addshore.com/2017/12/wikibase-docker-images/)
- [First basic example usage (the wikibase registry)](https://addshore.com/2018/04/wikibase-of-wikibases/)
- [Customizing the example docker-compose file](https://addshore.com/2018/06/customizing-wikibase-config-in-the-docker-compose-example/)
- [Creating your own Dockerfile for wikibase](https://addshore.com/2019/02/creating-a-dockerfile-for-the-wikibase-registry/)
- [Example update process for wikibase](https://addshore.com/2019/01/wikibase-docker-mediawiki-wikibase-update/)
- [Changing the concept URI of an existing Wikibase with data](https://addshore.com/2019/11/changing-the-concept-uri-of-an-existing-wikibase-with-data/)
LEGACY Image name | Description | README
------------------------ | ------------- | ----------
[`wikibase/wikibase`](https://hub.docker.com/r/wikibase/wikibase) | MediaWiki with the Wikibase extension| [README](https://github.com/wmde/wikibase-docker/blob/master/wikibase/README.md)
[`wikibase/wdqs`](https://hub.docker.com/r/wikibase/wdqs) | Blazegraph SPARQL query service backend | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs/README.md)
[`wikibase/wdqs-proxy`](https://hub.docker.com/r/wikibase/wdqs-proxy) | Proxy to make the query service READONLY and enforce query timeouts | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs-proxy/README.md)
[`wikibase/wdqs-frontend`](https://hub.docker.com/r/wikibase/wdqs-frontend) | UI for the SPARQL query service | [README](https://github.com/wmde/wikibase-docker/blob/master/wdqs-frontend/README.md)
[`wikibase/quickstatements`](https://hub.docker.com/r/wikibase/quickstatements) | UI to add data to Wikibase | [README](https://github.com/wmde/wikibase-docker/blob/master/quickstatements/README.md)