ESA Elasticsearch Endpoint

Proxies requests to Elasticsearch. It replicates the http API, but adds an authorization layer.

Prerequisites

We need an Elasticsearch server running that we have access to. Then we must configure the application to connect to it. This is done in the src/ratpack/config/EsaSampleConfig.groovy file. Example configuraiton:

elasticsearch {
    cluster {
        name = "elasticsearch"
        host = 'esa.local'
        port = 9300
    }
}

Starting the server

./gradlew run

This will start the server on port 5051. We can then send Elasticsearch queries to this address.

Running tests

Tests can be run in Intellij, or via the command line:

./gradlew test

We need to have https://github.com/Spantree/elasticsearch-appliance in our local machine, with the vm up and running:

git clone https://github.com/Spantree/elasticsearch-appliance
cd elasticsearch-appliance
vagrant up

Note: Don't clone the elasticsearch-applicance repo inside this one.

Configuring Elsaticsearch Permissions

Restrictions can be added to our proxy via src/ratpack/config/EsaPermissions.js. This is a normal javascript file and configuration is done by building an object literal with key fields. This object literal must be assigned to a variable named base. Example configuration:

var base = {
  indices: {
      _default: {
          access: "allow",
          fields: ["name"],
          source_filters: ["directed_by"]
      }
  }
};

This creates default permissions that are applied to all indices. The following restrictions are supported:

• access => If this field is not present or empty, it prevents any query from being executed. To allow queries its value must be "allow".
• fields => A list of strings that specify the fields that are allowed to be queried. If a query is submitted that contains a field that is not present the proxy ignores those fields and the results will not contain them.
• source_filters => Allows us to add restrictions to source filter queries. All source filters queries are denied by default. The following setting: source_filters: ["directed_by"] is compiled to:

_source: {
  include: ["directed_by"]
}

Rules can also be applied per index:

var base = {
  indices: {
      _default: [{
          access: "allow",
          fields: ["name", "produced_by"],
          source_filters: ["directed_by"]
      },
      freebase: {
        access: "allow",
        fields: ["name"],
        source_filters: ["directed_by"]
      }]
  }
};

A query submitted to the "freebase" index to a proxy with this setup will only return the name fields, even if other fields are specified in the query.

User/Roles Permissions

This feature tries to emulate other database management systems that support the ability to create users to specific roles that restricts or enables features. We can create these users within src/ratpack/config/EsaPermissions.js.

var base = {
  indices: {
      _default: [{
          access: "allow",
          fields: ["name", "produced_by"],
          source_filters: ["directed_by"]
      }],
      freebase: [{
        access: "allow",
        fields: ["name"],
        source_filters: ["directed_by"],
        roles: ["DRUMMER", "GUITARS"]
      }]
  }
};

var users = [
    {username: "ringo", roles: ["DRUMMER"]},
    {username: "george", roles: ["GUITAR", "VOCALS"]},
    {username: "john", roles: ["GUITAR", "VOCALS"]},
    {username: "paul", roles: ["BASS", "VOCALS"]}
];

This setup will only allow users that have "GUITARS" or "DRUMMER" roles to query "freebase" index. If a user that does not belong to any of those roles tries to query "freebase", then a 401 will be returned.

Indices without any roles restrictions will allow any user to query them and only apply the rules for fields and source_fields.

We can also create role-specific rules for an index. Each index takes a list of configurations, and each of these lists can be restricted by roles:

anIndex: [{
    access: "allow",
    fields: ["name"],
    roles: ["DRUMMER"]
    },
    {
    access: "allow",
    fields: ["description"],
    roles: ["BASS"]
    },
    {
    access: "allow",
    fields: ["name", "description"]
    roles: ["GUITARS"]
    }
];

The above provides separate settings for "DRUMMERS", "BASS", and "GUITARS" when they try to query "anIndex".

Note that when query is sent for a particular user, and that user is in multiple groups, the configuration that is used will be the first one found to match for that user group.

Contributing

1. Every new feature must include a test.
2. Every new file must also include the Apache License at the beginning of the file.
3. Add at least a sentence describing what the class that is created does.
4. As much as possible, when opening an issue, please try to include a branch with a failing test.