NOTE: We are currently in the process of deprecating this service due to:
- Performance limitations
- Maintainability issues
- Inability to scale effectively
- Limitations of the underlying HTTP framework
Zebedee is the CMS used by the ONS website and the internal website publishing system. It is a JSON API and does not have a user interface. It comes in 2 flavours:
Zebedee-reader is read-only. It's used by Babbage (the public facing web frontend of the ONS website) it returns the published site content as JSON.
Zebedee-cms is an extension of zebedee-reader. It's used by Florence and provides API endpoints for managing content, users, teams and publishing collecions. Zebedee-CMS is not public facing and requires authentication for the majority functionality. Pre-release content is encrypted and requires the appropriate permissions to be able to access it.
- git
- Java 8
- Maven
- Docker
As mentioned Zebedee is a JSON API, and does not have a user interface. The quickest and easiest way to use it is to set up a local copy of the "publishing" stack. Clone and set up the following projects following the README instructions in each repo:
If you encounter any issues or notice anything missing from this guide please update/add any missing/helpful information and open a PR.
Much appreciated. The Dev team
NOTE: The following set guide will set up Zebedee in "CMS" mode as this is typically how the devlopers will run the stack locally.
Getting the code:
git clone git@github.com:ONSdigital/zebedee.gitZebedee isn't backed by a database instead it uses a file system to store json files on disk ***. As a result it requires a specific directory structure in order to function correctly.
*** We know this is a terrible idea - but in our defence this is a legacy hangover and we are actively working towards deprecating it.
To save yourself some pain you can use the dp-zebedee-content tool to create the required directory structure and populate the CMS with some "default content" - follow the steps in the README before going any further.
Once the script has run successfully copy the generated script from dp-zebedee-utils/content/generated/run-cms.sh into
the root dir of your Zebedee project. This bash script compiles and runs Zebedee setting typical dev default
configuration and uses the content / directory structure generated by dp-zebedee-utils.
You may be required to make the bash script an executable before you can run it. If so run:
sudo chmod +x run.sh
<Enter your password when prompted>In order to use Zebedee you will need to have the following other project running:
- Florence
- Babbage
- Sixteens
- dp-compose
- dp-identity-api
Follow the steps in the README of each project.
Zebedee needs contents before it can be run. Basic demo content and file structures can be generated via the dp-zebedee-content by following the instructions in the dp-zebedee-content readme
./run.shAssuming Zebedee has started without error head to Florence login and login with the default account:
email: florence@magicroundabout.ons.gov.uk
password: Doug4l- If it's the first time logging in you will be prompted to change the password for that user.
- On the home screen create a new collection.
- Click
Create/editon the Collection Details screen.
If everything is working correctly you should now see the the ONS website displayed in the right hand pane. Congratulations 🎉! Advanced to GO collect £200 💵
Otherwise 🎻 kindly ask someone from the dev team to help troubleshoot.
| Environment variable | Default | Description |
|---|---|---|
| DEFAULT_WEBSITE_URL | "http://localhost:8080" | Service URL |
| DEFAULT_LEGACY_CACHE_API_URL | "http://localhost:29100" | Cache API Service URL |
| DEFAULT_SLACK_WARNING_CHANNEL | "slack-client-test" | Slack channel |
| DEFAULT_SLACK_ALARM_CHANNEL | "slack-client-test" | Slack alarm channel |
| DEFAULT_SLACK_USERNAME | "Zebedee" | Slack user |
| DEFAULT_PUBLIC_WEBSITE_URL | "http://localhost:8080" | Service public URL |
| DEFAULT_FLORENCE_URL | "http://localhost:8081" | Florence URL |
| brian_url | "http://localhost:8083" | Brian URL |
| DEFAULT_TRAIN_URL | "http://localhost:8084" | Train URL |
| DEFAULT_DYLAN_URL | "http://localhost:8085" | Dylan URL |
| CONTENT_DIRECTORY | "content" | Content directory |
| KAFKA_SEC_PROTO | unset | if set to "TLS", kafka connections will use TLS |
| KAFKA_SEC_CLIENT_CERT | unset | PEM for the client certificate 1 |
| KAFKA_SEC_CLIENT_KEY | unset | PEM for the client key 1 |
| MATHJAX_SERVICE_URL | "http://localhost:8888" | Mathjax service URL |
| DATASET_API_URL | "http://localhost:22000" | Dataset API URL |
| IMAGE_API_URL | "http://localhost:24700" | Image API URL |
| ENABLE_KAFKA | false | Feature flag to send kafka messages when a collection is published |
| KAFKA_ADDR | "localhost:9092" | Comma seperated list of kafka brokers |
| KAFKA_CONTENT_UPDATED_TOPIC | content-updated | Kafka topic to send content updated event to |
| DATASET_API_AUTH_TOKEN | "FD0108EA-825D-411C-9B1D-41EF7727F465" | Dataset API authentication token |
| SERVICE_AUTH_TOKEN | "15C0E4EE-777F-4C61-8CDB-2898CEB34657" | Service API authentication token |
| LEGACY_CACHE_API_AUTH_TOKEN | "748896205c3b42b43adb4b22fff11784c5d971187f280ab1b6f142c3d69e64e4" | Legacy Cache API authentication token |
| SESSIONS_API_URL | "http://localhost:24400" | Session API URL |
| KEYRING_SECRET_KEY | "KEYRING_SECRET_KEY"; | Keyring encryption key |
| KEYRING_INIT_VECTOR | "KEYRING_INIT_VECTOR" | Keyring init vector |
| VERIFY_RETRY_DELAY | 5000; //milliseconds | Retry delay duration |
| VERIFY_RETRY_COUNT | 10 | Retry count for how long, in seconds, to wait for retry |
| DEFAULT_PREPROCESS_SECONDS_BEFORE_PUBLISH | 30 | how many seconds before the actual publish time should we run the preprocess |
| DEFAULT_SECONDS_TO_CACHE_AFTER_SCHEDULED_PUBLISH | 30 | how many additional seconds after the publi |
| IDENTITY_API_URL | "http://localhost:25600" | Identity API URL |
| RESUMABLE_TYPE | "text/plain" | Resumable file type |
| IS_PUBLISHABLE | true | The file is publishable or not |
| LICENCE | "Open Government Licence v3.0" | The licence we are using |
| LICENCE_URL | "https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/" | The licence URL |
| UPLOAD_SERVICE_API_URL | "http://localhost:25100" | The upload service api URL |
The new central keyring feature is currently behind a feature flag:
export ENABLE_CENTRALISED_KEYRING=true/false- If enabled Zebedee will attempt to read/write from the new central keyring and default to the legacy keyring if unsuccessful.
- If disabled Zebedee will add/remove keys from both legacy and central keyring implementations but will only read from the legacy keyring.
The central keyring requires encryption config to be provided in app configutation. These secrets can be generated using the collection-keyring-secrets-generator.
Please see document in dp-operations: