pgHumor-clasificahumor
Web page for pgHumor and HUMOR corpora crowd-annotation. This is the input data for the HAHA competition as well.
Setup
There are two ways to run this code after cloning the repo: with Docker or via pipenv. The first one is the recommended way to get started and the second one is for advanced use (such as debugging with an IDE).
Docker way
You need Docker and Docker Compose for this. To run the Flask development server in debug mode, auto-detecting changes:
docker-compose up --build
Pipenv
-
Install the dependencies using pipenv:
pipenv install
-
Create a
.env
file with the following content (setting some env vars values):FLASK_APP=clasificahumor/main.py FLASK_DEBUG=1 FLASK_SECRET_KEY=SET_VALUE DB_HOST=SET_VALUE DB_USER=SET_VALUE DB_PASS=SET_VALUE DB_NAME=SET_VALUE
-
Run:
pipenv shell # It will load the environment, along with the .env file. flask run
-
Setup a MySQL 5.7 instance. It could be the instance generated with the Docker setup.
Tweets data
You need a data to mess with. There's a dump with the downloaded tweets in the HUMOR repo.
First, create a database with the options
DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_unicode_ci
. It could be
created with schema.sql:
mysql -u $USER -p < schema.sql
To load a database dump, run in another shell:
mysql -u $USER -p database < dump.sql
You can prefix docker-compose exec database
to the command to run it
in the database Docker container.
Useful SQL commands
List the databases:
show databases;
List pghumor
database tables:
use pghumor;
show tables;
Describe a particular table:
describe tweets;
Show some data from a table:
select * from tweets limit 10;
Testing
To run it using a WSGI server, just like in production, do:
docker-compose -f docker-compose.yml -f docker-compose.testing.yml up -d --build
Then you can do some testing, such as running a load test:
./load_test.sh
Manipulating production data
To backup data in production:
docker exec clasificahumor_database_1 mysqldump -u root -p pghumor > dump.sql
To run a SQL script in production (e.g., to restore some data):
docker exec -i clasificahumor_database_1 mysql -u root -p pghumor < dump.sql
To open a mysql interactive session in productrion:
docker exec -i clasificahumor_database_1 mysql -u root -p pghumor
For these commands, using directly Docker Compose (docker-compose exec database
) is also supported instead of the Docker CLI directly (docker exec clasificahumor_database_1
). However, the extra flags needed for each of them change as Docker Compose exec
subcommand uses a pseudo TTY and it's interactive by default while the Docker CLI exec
subcommand doesn't.