Shakespeare Quotes App
Create an online collection of Shakespeare quotes for #ShakespeareSunday.
- submit quotes into the collection
- search for quotes that have been tagged with certain keywords
- tweet quotes
- edit quotes
- delete quotes
- pagination when displaying more than 10 quotes
This is up and running here: https://shakespeare.cazyw.dev/
- hot module loading in docker container
- refactor the react side and add testing
System Dependencies & Configuration
Using docker containers in development.
For this app:
- node - v12.13.0
- npm - v6.12.x
- Heroku account for hosting (online)
- MongoDB (local - docker), mLab (online)
- Buildkite agent (for deployment via CI)
package.json for other packages installed. As I also used
create-react-app for the front end; it has its own
package.json in the client/ folder.
Application Installation Instructions
Clone the repository.
# build and start the node and mongodb containers make dockerUp # get into the node docker container to execute commands make dockerX # stop and remove the docker containers make dockerDown # prune any stopped containers and unused images make dockerPrune
The mongodb data is stored in a local data folder.
The following commands are also available:
make npmInstall make npmTest make npmStart
To install the other required packages (for both the Express server and React server), run:
$ npm run install-all # to build the react-front-end $ npm run build-react
Start the Express server and React server (this also rebuilds when there are changes to fix now that we are using docker):
$ npm run dev
Open the browser to http://localhost:5000/
The following was added to
package.json to build and bundle the React/front-end files for production so the web app only runs on one server (Express server). Heroku automatically runs this.
"heroku-postbuild": "cd client && npm install --only=dev && npm install && npm run build"
CI Deployment with Buildkite
The Github repository now includes a number of restrictions including:
- Require pull request reviews before merging
- Require status checks to pass before merging
- Deployment to Heroku on merge into master (and tests passing)
The status checks includes a webhook using Buildkite. Buildkite will run the automated tests and ensure they all pass. Buildkite: https://buildkite.com
Buildkite requires an active agent in order to run and currently I am using an agent run on my local PC. The agent is started whenever a push is made up to the repository. Notes on Buildkite setup
Project file structure
The project uses
react-scripts to build the final front-end bundle for production. The following are the main final project files:
Made with linux tree command
. ├── client ├── docker ├── docker-compose.yml ├── models ├── server └── test # express server server ├── controllers │ └── quotes.js ├── routes.js └── server.js models └── quote.js # react front-end client/src ├── index.js ├── item-components │ ├── ButtonForm.css │ ├── ButtonForm.js │ ├── QuoteItem.css │ ├── QuoteItem.js │ ├── QuoteList.js │ └── Twitter_Logo_WhiteOnImage.png ├── page-components │ ├── DisplayQuotes.css │ ├── DisplayQuotes.js │ ├── Header.css │ ├── Header.js │ ├── PostQuote.css │ ├── PostQuote.js │ ├── SearchQuote.css │ ├── SearchQuote.js │ ├── UpdateQuote.css │ └── UpdateQuote.js ├── styles.css └── utils ├── apiCalls.js ├── constants.js ├── errorHandling.js ├── helperFunctions.js └── updateDisplay.js
Testing using the mocha framework, and using the supertest and chai libraries.
to fix: chrome in docker
Then run the following:
//run tests $ npm test // tests with coverage via Istanbul $ npm run test-coverage // tests with watch mode $npm run test-watch
To do: testing for React front-end.
The idea for this project came about because I participate in a worldwide twitter event each Sunday -
#ShakespeareSunday. Started by @HollowCrownFans, every Sunday users around the world tweet Shakespeare quotes that follow a particular theme (e.g. light and dark).
I thought it'd be fun to build an online collection of Shakespeare quotes I use. And it'd be a cool way for me to learn and put into practice Node, Express, APIs, databases (MongoDB), React, forms and testing!
Setup and Environment
New frameworks used in this project include mongoDB and React. I also added a
config.js file that included different settings for production (Heroku), development (locally) and testing (for testing). This was particularly important as I wanted my test cases to run against a test database and not the local development database.
Initially I had used Webpack to build my final
.js file however once I looked more into React, I decided to use the
create-react-app package which black-boxes the transpiling and compiling of React (webapck and babel configuration).
I also used ESLint in this project and am getting to know some of the settings and configuration options. To run eslint in the console:
$ npm run lint
Node.js and the
express framework to create my API. This time adding
express.Router to create modular route handlers and separating the controller logic into its own file. The main
server.js file sets up the server, and does the console logging and error handling.
A separate server is run (in development) for the React front-end (based on the
Database and Modelling
I decided to use
- The title of the piece of work (play, sonnet etc) [required]
- The act [conditional requirement - depends if Scene entered, must be a number]
- The scene [conditional requirement - depends if Act entered, must be a number]
- The quote itself [required]
- Tags / keywords [required]
At the moment there's not a lot of deep validation, although the Work must be listed in the predifined list of works (sonnets not currently included).
For online storage, I went with
mLab as it can be added as an add-on to Heroku, has a free (albeit slower) tier and is a cloud Database-as-a-Service for MongoDB. The
config file selects the correct database (online, local, test) depending on the
Exporting and importing a file into the local database (drop to remove existing data):
$ mongoexport --db shakespeare --collection quotes --out <filename>.json $ mongoimport --db shakespeare --collection quotes --drop --file ./<filename>.json
This is my first real dive into React so it's been a learning experience looking at how to build the front-end using React components, how to dynamically render content based on its state and pass data between parent and child elements. Initially I setup webpack and its configuration myself however I decided to switch to the
create-react-app package as that includes some additional features/minification of files. Also using react-bootstrap.
Added some functions to perform basic form validation checks however doesn't check that the details entered are necessarily correct.
Pagination has now also been added using react-paginate. It was a good experience diving into their
demo.js code and modifying it for this app's requirements.
APIs / External Functionality Used
Twitter's Tweet Web Intent is used to allow users to tweet quotes directly to Twitter.
I am using the
mocha framework, and using the
chai libraries. It's been an interesting learning experience as I'm getting a better understanding of how to do testing. Setup and teardown functions have been added. At the moment the tests run against a test database and running server. I'll look into mocking the setup and unit-level testing.
routes.test.js uses the test database (so the development database is not used) and resets the database before each test. It tests that valid data posted to the database is saved, that invalid data is not, and that getting data from the database returns all data.
I have now also added Istanbul for code coverage. This has been very useful to see whether I've added tests for various parts of the source code and armed with this knowledge I aim to increase code coverage. Having used Sinon at work for mocking, I've now also added it to this project.
Downloading of Shakespeare's works for analysis
As well as manually searching for quotes and then adding them to this application, I decided to build a feature that would automatically scan and retrieve quotes that matched the given keyword. This part of the project is still a work in progress. At the moment I'm using
Puppeteer to retrieve the list of shakespeare's works from the MIT Shakespeare site. Currently I've been able to download the full text of each piece of work (plays and sonnets/poems).
sanitize-html, I've been able to remove all HTML tags. The next steps will be to:
- reformat the HTML into a more useful structure
At the moment the text is in it's original short-length structure, with sentences split across multiple lines. I'll then need to consider how to best reformat and store the text to be able to search for and retrieve quotes/sentences.