ERMrest (rhymes with "earn rest") is a general relational data storage service for web-based, data-oriented collaboration.
Switch branches/tags
v0.0-alpha.1.4 v0.0-alpha.1.3 v0.0-alpha.1.2 v0.0-alpha.1 v0.0-alpha.0 synapse-20181005.1 synapse-20180129.1 synapse-20180117.1 synapse-20171211.1 synapse-20171109.1 synapse-20170913.2 synapse-20170913.1 synapse-20170718.2 synapse-20170718.1 synapse-20170612.1 synapse-20170419.1 synapse-20170308.1 synapse-20161205.1 synapse-20161004.1 rbk-pre-faceting rbk-20180917 rbk-20180731 rbk-20180516 rbk-20180222 rbk-20171205 rbk-20171031 rbk-20171031-2 rbk-20171030 rbk-20171002 rbk-20170803 rbk-20170727 rbk-20170724 rbk-20170608 rbk-20170530 rbk-20170427 rbk-20170417 rbk-20170221 rbk-20170208 rbk-20170206 rbk-20170123 rbk-20161117 rbk-20161117-prod rbk-20161027 rbk-20161019 rbk-20161014 rbk-20161003 rbk-20160812 rbk-20160805 rbk-20160803 rbk-20160601.1 rbk-20160503.1 rbk-2016-03-16 latest-stateless-model gpcr-20180831.1 gpcr-20170612.1 gpcr-20170609.2 gpcr-20170609.1 gpcr-20170601.1 gpcr-20170306.1 gpcr-20161024.1 gpcr-20161007.1 gpcr-20161003.1 gpcr-20160822.1 gpcr-20160726.2 gpcr-20160726.1 gpcr-20160617.1 gpcr-20160610.1 fb-2016-04-25 facebase-20181108.1 facebase-20180926.1 facebase-20180727.1 facebase-20180605.1 facebase-20180511.1 facebase-20180311.1 facebase-20180111.1 facebase-20171115.1 facebase-20171023.1 facebase-20171008.2 facebase-20171008.1 facebase-20170913.1 cirm-deploy-3 cirm-deploy-2 cirm-20180523.1 cirm-20171110.1 cirm-20170901.1 cirm-20170802.1 cirm-20170609.1 cirm-20170203.1 cirm-20170124.1 cirm-20170117.1 cirm-20161214.1 2017-06-08 2015-04-02 45-software-checkout.sh-20170609.1
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
config cull out dead and misleading config vars, try to get travis build goi… Aug 9, 2017
docs
ermrest
sbin use ON_ERROR_STOP in ermrest-deploy to stop on SQL errors Aug 28, 2017
test patch foreign key visibility bug when constituent columns are non-enu… Aug 22, 2018
.gitignore
.travis.yml remove non-working addons option for postgres 10 Jul 12, 2018
LICENSE skeleton import for new experiment Sep 12, 2013
Makefile
README.md docs reorganization... move api-doc and user-doc under docs/ Jun 27, 2018
install-script fix semanage command in install-script Dec 21, 2017
setup.py

README.md

ERMrest

ERMrest (rhymes with "earn rest") is a general relational data storage service for web-based, data-oriented collaboration. It allows general entity-relationship modeling of data resources manipulated by RESTful access methods.

Status

Build Status

ERMrest is research software, but its core features have proven stable enough to use in several production science projects.

Known Issues: See our list of known issues at GitHub.

Using ERMrest

As a protocol, the ERMrest REST API can be easily accessed by browser-based applications or any basic HTTP client library. Its main features:

  • Exposes a PostgreSQL RDBMS containing science data.
  • Model neutrality
    • Allows use of natural, domain-specific relational data schema rather than forcing a fixed and generic schema.
    • Supports schema and data introspection by clients to allow generic presentation of tabular data rather than only hard-coded and domain-specific clients.
  • An expressive set of data access methods
    • Set-based single and bulk whole-entity (table row) create/read/update/delete (CRUD);
    • Set-based single and bulk partial-entity (table cell) read/update (RU);
    • Aggregate and grouped aggregate queries;
    • Convenient ERM navigation to map common relational inner join idioms into URL path structures.
  • Fine-grained access control lists to control client access privileges.
    • Group/role-based permissions
    • Control of granted access methods
      • Model visibility
      • Schema management
      • Data retrieval
      • Data modification
    • Control of granted access scope
      • Whole catalog
      • Single schema
      • Single table
      • Single column
  • Multi-tenancy to easily allow multiple catalogs, each with its own schema, data content, and policy.

Our companion client software for the ERMrest service includes:

Prerequisites

ERMrest is developed and tested primarily on the Fedora Linux distribution with Python 2.7. It has a conventional web service stack:

  • Apache HTTPD
  • mod_wsgi
  • web.py lightweight web framework
  • psycopg2 database driver
  • PostgreSQL
  • webauthn security adaptation layer (another product of our group)

Installation

See ERMrest Installation (Red Hat derivatives).

Operational Model

  1. The HTTPS connection is terminated by Apache HTTPD.
  2. The mod_webauthn Apache HTTPD module determined authenticated client context for requests
  3. The ERMrest service code executes as the ermrest daemon user
  4. The service configuration is loaded from ~ermrest/ermrest_config.json:
    • Core access control policy for catalog creation.
    • Data type presentation.
  5. All dynamic data is stored in the RDBMS.
    • The catalog's data model
    • The catalog's fine-grained access control lists
    • The catalog's data content
  6. Client authentication context is retrieved from Apache request environment
    • Client identity
    • Client roles/group membership.
  7. Catalog-level authorization of service requests is determined by the service code:
    • ACLs retrieved from RDBMS
    • ACLs are intersected with authenticated client context.
  8. The RDBMS is accessed using daemon service credentials
    • Fine-grained static authorization is handled in service prior to executing SQL
    • Fine-grained dynamic authorization is handled in service by compiling policy checks into SQL

Help and Contact

Please direct questions and comments to the project issue tracker at GitHub.

License

ERMrest is made available as open source under the Apache License, Version 2.0. Please see the LICENSE file for more information.

About Us

ERMrest is developed in the Informatics group at the USC Information Sciences Institute.