Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
This is the wiki for PageOneX. We'll use this to document everything related to the project.
Help us document the project!
We have divided this wiki in two sections:
PageOneX for users
How to use it
To start coding you must:
- Sign up in PageOneX.com
- Create a thread
- period: start and end date (we suggest to start short, 1 week, and then edit and make the thread larger)
- select newspapers
- topics (themes you want to code) and related color and description [you can edit and add more later]
- Code, that means drawing areas on top of the front page images.
- Display. PageOneX will display the data visualization with all the front pages and drawn areas and the bar chart on top of it.
Export data to a spreadsheet (.ods format)
From the display view it is possible to export to a multiple spreadsheet .ods file. Each of the sheets in the exported document contains the coverage per day per topic per newspaper. The first column of every sheet is a number. If you format the cell to "date format" it will render the date in human readable format.
Export thread to an image
From the display view it is possible to export to a jpg file. First you have to select the numbers of pixels you want for the width of your final image. To start with, 1200px is a good size, so just type "1200" without the ". You have two options:
- .jpg, downloads an image (it is crashing for big images, so better use the .zip option)
- .zip, downloads a zip folder with many files: two images per topic (just the areas and the areas an the front pages) + the composite image + just the newspaper images without the highlighted areas.
Use keyboard shortcuts to make some actions while coding:
- ← = previous image
- → = next image
- n = nothing to code
Tutorial: How to use PageOneX
Tutorial in slides in Spanish about How to use PageOneX by Anibal Rossi (in Spanish).
Video tutorial: How to use PageOneX (Spanish) | Cómo funciona PageOneX.
Video tutorial: How to create a thread in PageOneX | Cómo crear un hilo en PageOneX.
PageOneX for Developers
See more updated information about the data model: https://docs.google.com/presentation/d/1SnppQJLucy4OuTZ19oK94Q6S5tZY79TLEUvh3ljQlYs/edit#slide=id.ge24130b_0_0
How to deploy PageOneX locally or in your own server.
Note: You will need to have postgresSQL installed for the development environment, and mySQL for production environment.
You can avoid production environment by
bundle --without production
Manage ruby versions via RVM (ruby version manager): https://rvm.io/rvm/install/
Use RVM to get ruby 1.9 and make it the default with RVM (https://rvm.io/rubies/installing/):
rvm install 1.9.3
Then force to use version 1.9.3:
rvm use 1.9.3
- Install bundler for good package management (http://gembundler.com):
gem install bundler
- Grab the PageOneX code from github: https://github.com/numeroteca/PageOneX
git clone email@example.com:numeroteca/pageonex.git
It will generate a directory called 'pageonex'.
- Go to the directory you cloned the code into
cd pageonex, and install all the gem dependencies listed in the Gemfile:
gem install bundler
- Set up the development sqlite database:
Load the media (newspaper data) to the database:
Run the server:
You should be able to hit http://localhost:3000 and see something.
To install make a Heroku install
You will need to add to these gems to the Gemfile:
gem 'rails_12factor' # Heroku logging y servir assets (without this it doesn't serve images, css…) gem 'thin' gem 'nokogiri'
Then you need to
bundle install and push again to Heroku.
Install in Debian Squeeze
apt-get install ruby apt-get install rubygems apt-get install rails apt-get install imagemagick apt get install libmagick-dev apt-get install libmagickwand-dev apt-get install libxslt-dev libxml2-dev apt-get install libpq-de mysql2 curl -L https://get.rvm.io | bash -s stable gem install bundler
Controlling pagination: number of threads in /threads index:
Change the value of
self.per_page = 20 in app/models/threadx.rb
The pagination is made using
will_paginate gem and the styling with 'bootstrap-will_paginate'.
The default order of the threads is set at
default_scope order('created_at DESC') in app/models/threadx.rb
How to update the list of newspapers available from Kiosko.net?
First you need to run the scraper, it will check all the newspapers available in kiosko.net and will write the new found ones in public/kiosko_media_list.csv
Then you need to update the list of newspapers into the data base:
Switch between embedding image files from kiosko.net or storing locally
Change value to false at config/initializers/application.rb:
Pageonex::Application.config.use_local_images = true
For Heroku: This is useful when the sever where the PageOneX is deployed has not much space. It will embed the images from kiosko.net
Granting admin privileges
From the console (replace firstname.lastname@example.org with the user's email). The console should be run in production mode if you are using a production environment:
RAILS_ENV="production" rails console
or in development:
RAILS_ENV="development" rails console
Once inside the console, run the following:
User.find_by_email('email@example.com').update_attribute :admin, true
Prevent certain user names from being used
To add invalid usernames, just modify the list in the ok_username method of app/models/user.rb.
If you want to try and get all the images for a thread again:
Padding for grid of images in display
You can edit it in line 141 of the threadx model
Forcing to rescrape all the images for a thread
Open rails console:
rails console production
And once inside: