Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugins architecture and development environment #93

Closed
subnetmarco opened this issue Mar 25, 2015 · 6 comments
Closed

Plugins architecture and development environment #93

subnetmarco opened this issue Mar 25, 2015 · 6 comments

Comments

@subnetmarco
Copy link
Member

Kong needs to expose a documented API to allow the creation of plugins of any kind. The Plugins API should expose what functionalities and functions Kong core is programmatically exposing to Plugins, in order for them to be able to interact with the system.

It makes sense to start this conversation sooner than later to avoid rewriting the plugins if the API changes.

Below is a list of operations plugins should be able to perform:

  • Being installable from the CLI CLI roadmap #92
  • Being "verified" if they are written by Mashape
  • Provisioning DB migrations to create ad-hoc tables (also, Cassandra schema possible improvements #18).
  • Being able to create custom DAOs to access those tables. It would be nice if those DAOs could be DB-agnostic for most cases.
  • Being able to extend nginx locations that are handled by the plugin. Some plugins, like an OAuth server, need to be able to create new nginx locations, like /oauth-callback.
  • Being able to extend internal API endpoints: /apis/{API_ID}/keyauth/

Below is a list of operations plugins are not allowed to do:

  • Interfering with other plugins and Kong core.
  • Altering the DB data that's out of their scope.
  • Changing the nginx configuration.

Location entries

A quick note on the location entries editing. This feature doesn't necessarily need to change the nginx configuration and it shouldn't, since it can be handle programmatically by reading the requested url and pass the execution to the appropriate handler.

@thibaultcha
Copy link
Member

Any installed module running in Kong can just require the DAO or create a DB access and do whatever they want. Permissions need to be defined on the DB, example for Cassandra: https://www.datastax.com/documentation/cql/3.0/cql/cql_reference/list_permissions_r.html

Note: the Cassandra lua driver does not support authentication.

@thibaultcha thibaultcha added the idea/new plugin [legacy] those issues belong to Kong Nation, since GitHub issues are reserved for bug reports. label Mar 25, 2015
This was referenced Mar 27, 2015
@thibaultcha thibaultcha changed the title Plugins API Plugins architecture and development environment Apr 24, 2015
@ahmadnassri ahmadnassri added this to the 0.5.0 milestone Jul 24, 2015
thibaultcha added a commit that referenced this issue Aug 1, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to #93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed
thibaultcha added a commit that referenced this issue Aug 3, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to #93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed
thibaultcha added a commit that referenced this issue Aug 11, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to #93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed
thibaultcha added a commit that referenced this issue Aug 13, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to #93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed
thibaultcha added a commit that referenced this issue Aug 20, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to #93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed
ctranxuan pushed a commit to streamdataio/kong that referenced this issue Aug 25, 2015
Proper architecture and CLI update for plugin-specific migrations. This
is the first step in separating the plugins out of the core repo as
planed for 0.5.0. It is related to Kong#93.

This is implemented by using the same table as before
(`schema_migrations`) to keep track of the executed migrations, except
that each plugin and the core itself all have their own row.

For simplifications, migrations (plugins or core) now live in a single
file. The `database` folder disappeared for core and the migration lives
in the DAO. Plugins have migrations in a `migration` folder, named after
the type of the database (`cassandra.lua` currently).

For now, only the keyauth plugin has its own migrations.

The `kong migrations up|down` commands slightly changed. It takes a `-t`
parameter to specify which migrations to run (core or a plugin name) and
is running all migrations by default. `kong migrations list` lists all
executed migration for the core and all plugins.

- Needs tests (integration)
- Old unit tests were removed
- `database` was removed
- `kong migrations` slightly changed


Former-commit-id: 1be811953b825e97f7139e05a2b1b294ccd407b3
@Tieske
Copy link
Member

Tieske commented Aug 26, 2015

Below is a list of operations plugins are not allowed to do:

  • Interfering with other plugins and Kong core.
  • Altering the DB data that's out of their scope.
  • Changing the nginx configuration.

The question here is whether the plugins are trusted or not. If not, it would require sandboxing all plugin code. If they are trusted, then the above requirements can be dropped.

@thibaultcha thibaultcha removed this from the 0.5.0 milestone Aug 27, 2015
@subnetmarco
Copy link
Member Author

To be trusted, we would need to review the code of each plugin - it logically makes sense to have them sandboxed.

@Tieske
Copy link
Member

Tieske commented Aug 28, 2015

I'm not sure about that. Sandboxing is hard (even impossible to catch all evils). What are the use cases for using Kong? I cannot imagine some dynamic environment where plugins come and go all the time. Most of the time it's an administrator of some service that sets it up once. So fairly static.
If I had to administer a farm of servers, I would setup my own luarocks server, containing only the rocks I trust. And use that to roll out new installations/images/whatever.
I think the benefits are too limited to warrant the complexity and performance penalties of sandboxing.

Or am I missing something?

An alternative could be for Mashape/Kong to host a rocksserver containing all the reviewed and approved plugins, though reviewing and updating might be a huge effort.

@subnetmarco
Copy link
Member Author

Sandboxing is hard (even impossible to catch all evils

This is true. It would make logically sense to have them sandboxed, but the implementation may be hard.

I cannot imagine some dynamic environment where plugins come and go all the time.

In the future we could build something like https://atmospherejs.com/ where users can submit their own plugins and install them. It's still a little bit too far from now, but if that's the case then security could be a concern.

Because all of this is still in the air, we could talk about it when we finally decide what to do a few months from now.

@thibaultcha thibaultcha added task/feature Requests for new features in Kong [about] plugins and removed idea/new plugin [legacy] those issues belong to Kong Nation, since GitHub issues are reserved for bug reports. labels Oct 15, 2015
@ahmadnassri ahmadnassri added the BC label May 13, 2016
@thibaultcha thibaultcha added area/plugins-dev-api and removed area/plugins task/feature Requests for new features in Kong labels Sep 1, 2016
@hishamhm
Copy link
Contributor

I think it's fair to say that Kong has a plugin architecture by now. :)

javierguerragiraldez pushed a commit that referenced this issue Sep 3, 2021
hutchic added a commit that referenced this issue Jun 10, 2022
* ci(docker) login to docker hub

* ci(cache) rebuild the base images to clear stale cache
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants