an offline digital library
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
public add nginx settings for no buffer Aug 13, 2016
views
.gitignore
Readme.md
config.json.default
index.js
library-database.js
library-router.js
library-sync.js
package.json

Readme.md

a-library

About

a-library is simply that a library. Or more specifically, it is an offline digital library; a localised wifi network that hosts a free and open collection of documents. a-library was developed as a way for us to easily share books and articles with each other, and in the process to collectively develop a local archive of all the publications that are meaningful to us. By us, I mean you and I. even if we have not personally met, we have shared this location, and these texts are now between us.

a-library is currently installed in a number of physical locations. To access a-library visit frontyard.

Run locally

To install a-library on your own server simply download the code and run the following commands:

npm install
node index.js

Setup Rpi as local library

a-library is intended to be installed on a raspberry pi, configured as an open wifi access point.

Note: these instructions assume you are using a raspberry pi 3 with a clean install of the default operating system Jesse and have setup a new user named librarian.

  1. Install all required software:

    sudo apt-get update
    sudo apt-get install dnsmasq hostapd nginx git
  2. Install node js v4:

    cd ~
    wget https://nodejs.org/dist/v4.0.0/node-v4.0.0-linux-armv7l.tar.gz 
    tar -xvf node-v4.0.0-linux-armv7l.tar.gz 
    cd node-v4.0.0-linux-armv7l
    sudo cp -R * /usr/local/
    cd ..
    sudo rm -r node-v4.0.0-linux-armv7l
  3. Installing a-library:

    git clone https://github.com/e-e-e/a-library
    cd a-library
    npm install
  4. Configure a-libary:

    set config.json with your own preferences

    {
    	"server" :{
    		"port":8080,
    		"admin_path": "/management"
    	},
    	"database": {
    		"location":"./tmp"
    	}
    }

    server.admin_path is the url path used for managing the library. It presents a simple GUI for renaming and deleting documents from within the database. It is recommended that you change this to an obsure key too limit access to administrative features.

    location.database is used to set the location of the sqlite3 database. It is recommend that you use an external USB or harddrive for the library as this will limit reading and writing to the system disk.

  5. Automatically start a-library:

    We use Forever to keep a-library running on a read only system. Read below for instructions on setting up a-library readonly.

    First install forever: sudo npm install forever -g. Then make a new startup process sudo nano /etc/init.d/nodeup with the following settings:

    #!/bin/sh
    #/etc/init.d/nodeup
    
    ### BEGIN INIT INFO
    # Provides:          nodeup
    # Required-Start:    $remote_fs $syslog
    # Required-Stop:     $remote_fs $syslog
    # Default-Start:     2 3 4 5
    # Default-Stop:      0 1 6
    # Short-Description: Starts all node apps
    # Description:       Starts all node apps like AAM, AMT,...
    ### END INIT INFO
    
    export PATH=$PATH:/usr/local/bin
    export NODE_PATH=$NODE_PATH:/usr/local/lib/node_modules
    
    start() {
      forever --sourceDir=/home/librarian/a-library --workingDir=/home/librarian/a-library -a -l /tmp/alibrary.log -e /tmp/alibrary-err.log -o /tmp/alibrary-output.log --pidFile=/tmp/node.pid index.js 2>&1 > /dev/null &
      return 0
    }
    
    case "$1" in
    start)
      start
      ;;
    stop)
      exec forever stopall
      ;;
    *)
      echo "Usage: /etc/init.d/nodeup {start|stop}"
      exit 1
      ;;
    esac

    Set it's permissions to make it executable: sudo chmod 755 /etc/init.d/nodeup, And tell Debian to start it on launch via Debian's update-rc.d: update-rc.d nodeup defaults.

    If you want to stop it automatically starting use update-rc.d -f nodeup remove.

    Use pm2 for non-readonly version.

    We use PM2 process manager to keep a-library running.

    sudo npm install pm2 -g
    pm2 start index.js
    sudo pm2 startup systemd -u yourusername
    # follow instructions provide by pm2 to install startup script.
    pm2 save

    Now a-library will be running automatically on start up, and will be relaunched if there are any problems.

  6. Setup nginx as a reverse proxy:

    create new site settings

    sudo nano /etc/nginx/sites-available/alibrary

    Include the following settings, importantly setting the media folder and port number to match your a-library settings:

    server {
    	listen 443 ssl;
    	ssl_certificate /etc/nginx/ssl/nginx.crt; 
    	ssl_certificate_key /etc/nginx/ssl/nginx.key; 
    	listen 80 default_server;
    	client_body_buffer_size     10M;
    	client_max_body_size        10M;
    
    	# this is to fake captive portal
    	if ($http_user_agent ~* (CaptiveNetworkSupport)) {
    		return 200;
    	}
    
    	return 302 http://a.library$request_uri;
    	#$scheme://a-library;
    }
    
    upstream a_library {
    	# Important: listening on the same port as set in config.json
    	server 127.0.0.1:8080;
    }
    
    server {
    	listen 80;
    	server_name a.library;
    	root /var/www/library/public;
    	index index.html index.htm;
    	client_body_buffer_size     10M;
    	client_max_body_size        10M;
    
    	location / {
    		try_files $uri @node;
    	}
    
    	# serve documents statically through nginx
    	location ~* ^/!/(.+\.(?:pdf|epub))$ {
    		# Important: alias should be the directory where database is stored
    		alias /media/usb/tmp/;
    		#add_header X-Whom www-a-library;
    		try_files $1 @node;
    	}
    	
    	location @node {
    		# disable caching and buffering for read only system
    		expires off;
    		proxy_cache off;
    		proxy_buffering off;
    		proxy_set_header Host $http_host;
    		proxy_set_header X-Forwarded-For $remote_addr;
    		proxy_pass http://a_library;
    	}
    
    }

    Generate a self signed certificate for Nginx.

    apt-get install openssl
    sudo mkdir /etc/nginx/ssl 
    sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt

    Activate proxy by creating a symbolic link to /etc/nginx/sites-enabled/ and restarting Nginx.

    sudo ln -s /etc/nginx/sites-available/alibrary /etc/nginx/sites-enabled/
    #remove default site settings
    rm /etc/nginx/sites-enabled/default
    sudo service nginx restart
  7. Set up static ip addresses

    add the following lines to /etc/dhcpcd.conf to set static ip addresses for both wan0 and eth0.

    interface wlan0
    static ip_address=172.24.1.1/24
    
    interface eth0
    static ip_address=192.168.3.10/24
    static routers=192.168.3.1
    static domain_name_servers=192.168.3.1
    

    Stop wpa_supplicant from messing around with setting up wlan0 as an access point. Open /etc/network/interfaces and comment out the line containing wpa-conf in the wlan0 section, so that it looks like this:

    allow-hotplug wlan0  
    iface wlan0 inet manual  
    #    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

    Restart dhcpcd sudo service dhcpcd restart to assign static addresses.

  8. Configure Hostapd

    Make a new config file sudo nano /etc/hostapd/hostapd.conf with the following settings.

    # This is the name of the WiFi interface we configured above
    interface=wlan0
    
    # Use the nl80211 driver with the brcmfmac driver
    driver=nl80211
    
    # This is the name of the network
    ssid=a-library
    
    # Use the 2.4GHz band
    hw_mode=g
    
    # Use channel 6
    channel=6
    
    # Enable 802.11n
    ieee80211n=1
    
    # Enable WMM
    wmm_enabled=1
    
    # Enable 40MHz channels with 20ns guard interval
    ht_capab=[HT40][SHORT-GI-20][DSSS_CCK-40]
    
    # Accept all MAC addresses
    macaddr_acl=0
    
    # Require clients to know the network name
    ignore_broadcast_ssid=0

    You can check if this works by running sudo /usr/sbin/hostapd /etc/hostapd/hostapd.conf. If you look for wifi networks now you should see 'a-library' available. CTRL-C to kill the process.

    Tell hostapd to use this config file to use this config file by opening up the default configuration file - sudo nano /etc/default/hostapd. Replace the line #DAEMON_CONF="" with DAEMON_CONF="/etc/hostapd/hostapd.conf".

  9. Configure DNSMasq

    Replace the default settings with a clean copy:

    sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig  
    sudo nano /etc/dnsmasq.conf  

    Use the following settings:

    interface=wlan0      # Use interface wlan0  
    bind-interfaces      # Bind to the interface to make sure we aren't sending things elsewhere  
    #server=8.8.8.8       # Forward DNS requests to Google DNS  
    domain-needed        # Don't forward short names  
    bogus-priv           # Never forward addresses in the non-routed address spaces.  
    dhcp-range=172.24.1.50,172.24.1.150,12h # Assign IP addresses between 172.24.1.50 and 172.24.1.150 with a 12 hour lease time 

    To redirect all traffic to the address a.library you need to set up a custom dnsmasq rule.

    sudo echo address=/#/172.24.1.1 > /etc/dnsmasq.d/a-library

    This will automatically assign the ip 172.24.1.1 to all domain names. Nginx listens to all traffic and directs the requests to a.library. This only partially works for https traffic, as browsers with strong security will reject the self-signed certificate that nginx serves.

  10. Start Hostapd and DNSMasq

    Start hostapd and dnsmasq to make a-library discoverable via wifi.

    sudo service hostapd start
    sudo service dnsmasq start

  11. Finally

    There is an issue with dnsmasq starting before hostapd on startup. A dirty fix is to restart dnsmasq by adding service dnsmasq restart to rc.local.

    The rc.local should look something like this:

    #!/bin/sh -e
    #
    # rc.local
    #
    # This script is executed at the end of each multiuser runlevel.
    # Make sure that the script will "exit 0" on success or any other
    # value on error.
    
    # Print the IP address
    _IP=$(hostname -I) || true
    if [ "$_IP" ]; then
      printf "My IP address is %s\n" "$_IP"
    fi
    
    printf "Configuring DNSMasq\n"
    sudo service dnsmasq restart
    
    exit 0
    

Make filesystem read only

Back up fstab sudo cp /etc/fstab /etc/fstab.original. Then change /etc/fstab file to:

proc            /proc            proc    defaults  0       0
/dev/mmcblk0p6  /boot            vfat    ro        0       2
/dev/mmcblk0p7  /                ext4    ro        0       1
tmpfs           /tmp             tmpfs   defaults,noatime,mode=1777      0       0
tmpfs           /var/log         tmpfs   defaults,noatime,mode=0755      0       0
tmpfs           /var/log/nginx   tmpfs   defaults,noatime,mode=0755      0       0
tmpfs           /var/lib/systemd tmpfs   defaults,noatime,mode=0755      0       0
tmpfs           /run             tmpfs   defaults,noatime,mode=0755      0       0

/dev/sda1        /media/usb/      vfat    auto,users,rw,uid=1001,gid=1001,umask=0002 0 0
/dev/sdb1        /media/backup/      vfat    auto,users,rw,uid=1001,gid=1001,umask=0002 0 0

these lines automatically mount usb drives

/dev/sda1        /media/usb/      vfat    auto,users,rw,uid=1001,gid=1001,umask=0002 0 0
/dev/sdb1        /media/backup/      vfat    auto,users,rw,uid=1001,gid=1001,umask=0002 0 0

for this to work you need to make the directorys /media/usb and /media/backup.

To test fstab run mount -a.

Disable nginx logs by setting logs in \etc\nginx\nginx.config:

access_log  /dev/null;
error_log /dev/null;

Note: you need to make user that nginx log folder exists.

DHCP and DNSMasq will fail to start in a readonly system because they need to write information to their lease files. This is easily fixed by creating symbolic links to the tmp directory.

# for dhcp.leases
rm -rf /var/lib/dhcp/
ln -s /tmp /var/lib/dhcp

# for dnsmasq.leases
sudo rm -rf /var/lib/misc/
sudo ln -s /tmp /var/lib/misc

These instructions are an amalgamation from these sources:

To do:

Raspberry pi:

  • nothing urgent

Interface:

  • Simple search
  • Paginate
  • Feature random text at the top