Skip to content

pulibrary/figgy

Repository files navigation

Figgy

A digital repository application in use at Princeton University Library for storing and managing digital representations of manuscripts, ephemera, vector, and raster data for export into a variety of front-end displays.

CircleCI Coverage Status Browserstack

Dependencies

  • Ruby
  • Node v.8.2.1
  • Java (to run Solr server)
  • Postgres (for OSX dev systems, install via homebrew)
  • Redis
    • Start Redis with redis-server or if you're on certain Linuxes, you can do this via sudo service redis-server start.
  • ImageMagick
    • On a mac, do brew install imagemagick --with-little-cms2 --with-openjpeg
  • Kakadu
    • On a mac, extract the file and run the pkg installer therein (don't get distracted by the files called kdu_show)
  • RabbitMQ (Optional)
    • Start with rabbitmq-server
    • Used for publishing create/update/delete events for systems such as Pomegranate
  • GDAL
    • You can install it on Mac OSX with brew install gdal.
    • On Ubuntu, use sudo apt-get install gdal-bin.
  • Tesseract
    • Version 3.04 is on the servers; homebrew installs 3.05: brew install tesseract --with-all-languages
    • For Ubuntu you'll have to compile it.
  • MediaInfo
    • You can install it on Mac OSX with brew install mediainfo.
    • On Ubuntu, use sudo apt-get install mediainfo.
  • [FreeTDS]
    • Needed for migration of music reserve data; we'll likely remove this dependency once that's all done
    • brew install FreeTDS

Simple Tiles

Figgy requires the image generation library Simple Tiles.

Mac OS X:

  • Install via Homebrew: brew install simple-tiles

Linux:

  • Install dependencies:

    apt-get install gdal-bin libgdal-dev libcairo2-dev libpango1.0-dev
    
  • Compile:

    git clone git@github.com:propublica/simple-tiles.git
    cd simple-tiles
    ./configure
    make && make install
    
    • Python:

      Should you receive the following error during the installation...

      TypeError: unsupported operand type(s) for +: 'dict_items' and 'list' make: *** [install] Error 2
      

      ...please know that you must downgrade to the latest stable release of Python 2.x.

Initial Setup

git clone https://github.com/pulibrary/figgy.git
cd figgy
bundle install
yarn install

Remember you'll need to run bundle install and yarn install on an ongoing basis as dependencies are updated.

Setup server

  1. For test:
    • RAILS_ENV=test rake db:setup
    • rake figgy:test
    • In a separate terminal: bundle exec rspec
    • Run jest tests: yarn test
  2. For development:

Load sample development data

  1. Log in to your development instance using your princeton credentials; this creates your user in figgy's db.
  2. Start sidekiq (see below)
  3. rails db:seed # pipe through grep -v WARN to ignore log warnings about the rabbitmq port

Background workers

Some tasks are performed by background workers. To run a Sidekiq background worker process to execute background jobs that are queued:

bundle exec sidekiq

Loading controlled vocabularies

To load the controlled vocabularies in config/vocab/:

  • rails vocab:load CSV=config/vocab/iso639-1.csv NAME="LAE Languages"
  • rails vocab:load CSV=config/vocab/iso639-2.csv NAME="ISO-639-2 Languages"
  • rails vocab:load CSV=config/vocab/lae_areas.csv NAME="LAE Areas"
  • rails vocab:load CSV=config/vocab/lae_genres.csv NAME="LAE Genres" LABEL=pul_label
  • rails vocab:load CSV=config/vocab/lae_subjects.csv NAME="LAE Subjects" CATEGORY=category LABEL=subject

Uploading files

By default, Figgy provides users with the ability to upload binaries from the local file system environment using the directory https://github.com/pulibrary/figgy/tree/master/staged_files. One may copy files into this directory for aiding in development, and may upload these files in this directory using the "File Manager" interface (exposed after saving a Work).

Google Drive Storage Support

Figgy may also be configured to upload files from hosted storage providers. Support for users with Google Drive accounts has been tested and verified. Please reference the Browse Everything documentation for more details.

Preservation Configuration in Development

Figgy uses Google Cloud Storage buckets for providing support for preserving certain resources. Please find further documentation outlining the configuration for Google Cloud service authentication and permissions management here.

By default, in development, preserved objects will be stored in the directory "tmp/cloud_backup." If you'd like to configure and test Google Cloud storage instead, do the following:

  1. Download and save gcs_pulibrary-staging-credentials.json from LastPass into the tmp directory.
  2. Create a .env file in the root with the following settings:
    STORAGE_PROJECT=pulibrary-figgy-storage-1
    STORAGE_CREDENTIALS=tmp/gcs_pulibrary-staging-credentials.json
    
  3. Restart the server. Now items marked with the cloud preservation policy will save to a bucket you can view at https://console.cloud.google.com/storage/browser
  4. Items only last in this bucket for 2 days, and aren't versioned.

Administering Figgy

To put figgy in readonly mode, use the ansible playbook. Be mindful of the value of the figgy_read_only_mode variable when provisioning during readonly downtime. It defaults to false and could therefore turn off readonly mode prematurely if you don't override it.

Cloud Fixity Checking

Documentation on setup for staging/production Fixity configuration can be found in preservation_documentation.md.

More

For links to helpful valkyrie documentation and troubleshooting tips, visit the wiki pages.