Public API: Create new project (fixes #1095) #1106

andrewn · 2019-06-17T12:38:17Z

This implements the following new endpoints that match the Public API spec (#1076). I've modified npm run fetch-examples-ml5 so that it uses the new endpoints. It expects a user to exists and a Public Access Token to be provided to the script.

GET /api/:user/sketches to list all sketches
DELETE /api/:user/sketches/:id to delete a sketch
POST /api/:user/sketches to create a new sketch
Tests for the mapping of the API files data structure to the mongo models
Descriptive error messages returned by the API
Tests for the three controller functions

I need to add more tests around the controllers, but I wanted to share now to get feedback.

I have verified that this pull request:

has no linting errors (npm run lint)
is from a uniquely-named feature branch and has been rebased on top of the latest master. (If I was asked to make more changes, I have made sure to rebase onto master then too)
is descriptively named and links to an issue number, i.e. Fixes #123

The endpoints don't exist yet, but this is a good way to see how the implementation of the data structures differ.

This transforms the nested tree of file data into a mongoose Project model

server/controllers/project.controller.js

server/domain-objects/Project.js

andrewn · 2019-06-17T12:41:54Z

server/server.js

@@ -95,6 +96,7 @@ app.use(session({

 app.use(passport.initialize());
 app.use(passport.session());
+app.use('/api', requestsOfTypeJSON(), api);


I think these public endpoints should be mounted at /api and all the rest that only the editor uses should be mounted at /editor?

to me that sounds logical. do you think we should add the API versioning as well? i was trying to figure out what best practices are for this, but it seems like the options all have their own drawbacks (here's a blog post that breaks this down).

I've added URL-based versioning in 57ced9c. Although it's not ideal, I think it's more obvious for casual users that may write scripts to import data into the Editor.

I'll change the namespaces of the private and public APIs in another PR.

catarak · 2019-06-17T16:33:18Z

server/controllers/project.controller/createProject.js

@@ -30,3 +31,27 @@ export default function createProject(req, res) {
    .then(populateUserData)
    .catch(sendFailure);
 }
+
+// TODO: What happens if you don't supply any files?


i think there are two options here: (1) create default files (the ones that are currently the default when you click sketch-> new) or (2) the project isn't valid. my instinct here is (1)...

This is done. See ee6891d. The default files are created unless a root-level .html file is created. Also, if sketch.js and style.css exist, then they are not overwritten by defaults.

catarak · 2019-06-17T16:39:24Z

server/utils/createApplicationErrorClass.js

@@ -0,0 +1,35 @@
+/**


thanks for writing this class and and moving towards better error handling/capturing ✨

Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided.

andrewn · 2019-06-19T16:56:53Z

@catarak Just pushed some tests and a check for slug uniqueness for the user. If a slug is specified and it is not unique in the sketches that the user has created, then the ids of the project(s) with identical slugs are returned as an error.

Eventually, I think this should live as a pre-save hook on the Project mongoose model. But as we don't have a UI for editing slugs, then we'd need to discuss what that behaviour should be first.

- implements tests - update apiKey tests to use new User mocks

`message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format

Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided.

- implements tests - update apiKey tests to use new User mocks

`message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format

…n/p5.js-web-editor into feature/public-api-create-project

andrewn · 2019-08-03T13:34:59Z

I only can't reproduce the slug validation issue, which is weird.

which version of mongo are you using? i'm using 3.6.6, since that's what mlab uses.

@catarak I still can't reproduce this. I dropped the p5js-web-editor mongo database and switched to using 3.6:

$ ps aux | grep mongo
andrew           11711   0.3  0.1  5106368  13004   ??  S    Mon09am  12:23.74 /usr/local/opt/mongodb-community@3.6/bin/mongod --config /usr/local/etc/mongod.conf

I created a new user (andrewn), with a new personal access token and POSTed to create a new sketch:

* Preparing request to http://localhost:8000/api/v1/andrewn/sketches
* Using libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1
* Current time is 2019-08-03T13:29:15.247Z
* Disable timeout
* Enable automatic URL encoding
* Disable SSL validation
* Enable cookie sending with jar of 0 cookies
* Found bundle for host localhost: 0x7fb7e1f28530 [can pipeline]
* Re-using existing connection! (#4) with host localhost
* Connected to localhost (::1) port 8000 (#4)
* Server auth using Basic with user 'andrewn'

> POST /api/v1/andrewn/sketches HTTP/1.1
> Host: localhost:8000
> Authorization: Basic YW5kcmV3bjpORFZrWVRCbU0ySXhOV1ExWTJVM05tTmxaamhrT1RWbVkyTXhPVEV6WkRJNU1EZzNPRGhqT0E9PQ==
> User-Agent: insomnia/6.5.4
> Content-Type: application/json
> Accept: */*
> Content-Length: 44

| {
|   "name": "My cute sketch",
| 	"files": {}
| }

* upload completely sent off: 44 out of 44 bytes

< HTTP/1.1 201 Created
< X-Powered-By: Express
< Vary: Origin
< Access-Control-Allow-Credentials: true
< Content-Type: application/json; charset=utf-8
< Content-Length: 18
< ETag: W/"12-7Z0Dd/UqbdLyYIPp5BjeLo/dpLU"
< Date: Sat, 03 Aug 2019 13:29:15 GMT
< Connection: keep-alive


* Received 18 B chunk
* Connection #4 to host localhost left intact

mongoose is at v 4.13.18:

$ npm ls mongoose
p5.js-web-editor@0.0.1 /Users/andrew/Projects/oss/processing/p5.js-web-editor
└── mongoose@4.13.18

Perhaps my mongod.conf is different?

catarak · 2019-08-06T16:27:06Z

let me try testing this again to sanity check, it's super weird!

catarak · 2019-08-06T19:35:17Z

i'm using postman to test this, maybe that has to do with it?

catarak · 2019-08-13T14:54:59Z

In testing this again, I found a bug. I made a POST to /api/v1/garbage_username/sketches with the correct authorization header (my real username, cassie + the API key), and a sketch body ({ "name": "My cute sketch", "files": {} }) and it successfully created a sketch under my username.

…mespace Previously, the project was always created under the authenticated user's namespace, but this not obvious behaviour.

andrewn · 2019-08-30T13:32:09Z

In testing this again, I found a bug. I made a POST to /api/v1/garbage_username/sketches with the correct authorization header (my real username, cassie + the API key), and a sketch body ({ "name": "My cute sketch", "files": {} }) and it successfully created a sketch under my username.

This is now fixed. An UNAUTHORIZED response code is returned with the username of the authenticated user, and the username specified in the URL.

catarak · 2019-08-30T18:26:39Z

sweet, thanks for fixing that! i don't think there's any other issues so i'll merge this in.

* Converts import script to use public API endpoints The endpoints don't exist yet, but this is a good way to see how the implementation of the data structures differ. * Exposes public API endpoint to fetch user's sketches * Implements public API delete endpoint * Adds helper to create custom ApplicationError classes * Adds create project endpoint that understand API's data structure This transforms the nested tree of file data into a mongoose Project model * Returns '201 Created' to match API spec * Removes 'CustomError' variable assignment as it shows up in test output * transformFiles will return file validation errors * Tests API project controller * Tests toModel() * Creates default files if no root-level .html file is provided * Do not auto-generate a slug if it is provided Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided. * Validates uniqueness of slugs for projects created by the public API * Adds tests for slug uniqueness * Configures node's Promise implementation for mongoose (fixes warnings) * Moves createProject tests to match controller location * Adds support for code to ApplicationErrors * deleteProject controller tests * getProjectsForUser controller tests - implements tests - update apiKey tests to use new User mocks * Ensure error objects have consistent property names `message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format * Assert environment variables are provided at script start * Version public API * Expect "files" property to always be provided * Fixes linting error * Converts import script to use public API endpoints The endpoints don't exist yet, but this is a good way to see how the implementation of the data structures differ. * Exposes public API endpoint to fetch user's sketches * Implements public API delete endpoint * Adds helper to create custom ApplicationError classes * Adds create project endpoint that understand API's data structure This transforms the nested tree of file data into a mongoose Project model * Returns '201 Created' to match API spec * Removes 'CustomError' variable assignment as it shows up in test output * transformFiles will return file validation errors * Tests API project controller * Tests toModel() * Creates default files if no root-level .html file is provided * Do not auto-generate a slug if it is provided Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided. * Validates uniqueness of slugs for projects created by the public API * Adds tests for slug uniqueness * Configures node's Promise implementation for mongoose (fixes warnings) * Moves createProject tests to match controller location * deleteProject controller tests * Adds support for code to ApplicationErrors * getProjectsForUser controller tests - implements tests - update apiKey tests to use new User mocks * Ensure error objects have consistent property names `message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format * Assert environment variables are provided at script start * Version public API * Expect "files" property to always be provided * Fixes linting error * Checks that authenticated user has permission to create under this namespace Previously, the project was always created under the authenticated user's namespace, but this not obvious behaviour.

andrewn added 5 commits June 17, 2019 14:20

Converts import script to use public API endpoints

999125f

The endpoints don't exist yet, but this is a good way to see how the implementation of the data structures differ.

Exposes public API endpoint to fetch user's sketches

e9a1bae

Implements public API delete endpoint

c45f24f

Adds helper to create custom ApplicationError classes

aa6cbe5

Adds create project endpoint that understand API's data structure

075f9b6

This transforms the nested tree of file data into a mongoose Project model

andrewn added this to the API and Collection Features milestone Jun 17, 2019

andrewn requested a review from catarak June 17, 2019 12:38

andrewn commented Jun 17, 2019

View reviewed changes

server/controllers/project.controller.js Outdated Show resolved Hide resolved

andrewn commented Jun 17, 2019

View reviewed changes

server/domain-objects/Project.js Outdated Show resolved Hide resolved

andrewn commented Jun 17, 2019

View reviewed changes

catarak reviewed Jun 17, 2019

View reviewed changes

andrewn added 8 commits June 19, 2019 12:11

Returns '201 Created' to match API spec

cef4dca

Removes 'CustomError' variable assignment as it shows up in test output

8699c1b

transformFiles will return file validation errors

dbeada5

Tests API project controller

a38527d

Tests toModel()

728c427

Creates default files if no root-level .html file is provided

ee6891d

Do not auto-generate a slug if it is provided

c2064e9

Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided.

Validates uniqueness of slugs for projects created by the public API

0f393ec

andrewn added 7 commits June 27, 2019 17:48

Adds tests for slug uniqueness

5ada625

Configures node's Promise implementation for mongoose (fixes warnings)

1549fc8

Moves createProject tests to match controller location

acdac37

Adds support for code to ApplicationErrors

db717b7

deleteProject controller tests

c75cdad

getProjectsForUser controller tests

64e39aa

- implements tests - update apiKey tests to use new User mocks

Ensure error objects have consistent property names

3041b35

`message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format

andrewn force-pushed the feature/public-api-create-project branch from ba4ead7 to 3041b35 Compare July 4, 2019 13:04

Assert environment variables are provided at script start

613f33a

andrewn and others added 18 commits July 22, 2019 16:16

Tests toModel()

cb767af

Creates default files if no root-level .html file is provided

d89be67

Do not auto-generate a slug if it is provided

558ad58

Fixes a bug where the slug was auto-generated using the sketch name, even if a slug property had been provided.

Validates uniqueness of slugs for projects created by the public API

475179c

Adds tests for slug uniqueness

f80ba5c

Configures node's Promise implementation for mongoose (fixes warnings)

ca66043

Moves createProject tests to match controller location

5bd757d

Adds support for code to ApplicationErrors

a08603a

deleteProject controller tests

8798c6d

getProjectsForUser controller tests

afb30c2

- implements tests - update apiKey tests to use new User mocks

Ensure error objects have consistent property names

d3a5621

`message` is used as a high-level description of the errors `detail` is optional and has an plain language explanation of the individual errors `errors` is an array of each individual problem from `detail` in a machine-readable format

Assert environment variables are provided at script start

e3c93e5

Version public API

2b0a218

Expect "files" property to always be provided

218e54d

Fixes linting error

9f06995

Merge branch 'feature/public-api-create-project' of github.com:andrew…

a925cb2

…n/p5.js-web-editor into feature/public-api-create-project

Merge branch 'feature/public-api' into feature/public-api-create-project

24e9c55

Merge branch 'feature/public-api' into feature/public-api-create-project

47e0ea8

Checks that authenticated user has permission to create under this na…

3c23fee

…mespace Previously, the project was always created under the authenticated user's namespace, but this not obvious behaviour.

andrewn mentioned this pull request Aug 30, 2019

Public API: Namespace private and public APIs #1148

Merged

3 tasks

catarak merged commit d44a058 into processing:feature/public-api Aug 30, 2019

catarak added this to Done in Sketch Collections and Web Editor API Dec 11, 2019

duskvirkus mentioned this pull request Apr 1, 2020

Fixing Web Editor API CodingTrain/website-archive#2200

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Public API: Create new project (fixes #1095) #1106

Public API: Create new project (fixes #1095) #1106

andrewn commented Jun 17, 2019 •

edited

andrewn Jun 17, 2019

catarak Jun 17, 2019

andrewn Jul 4, 2019

catarak Jun 17, 2019

andrewn Jun 19, 2019 •

edited

catarak Jun 17, 2019

andrewn commented Jun 19, 2019

andrewn commented Aug 3, 2019

catarak commented Aug 6, 2019

catarak commented Aug 6, 2019

catarak commented Aug 13, 2019

andrewn commented Aug 30, 2019 •

edited

catarak commented Aug 30, 2019

Public API: Create new project (fixes #1095) #1106

Public API: Create new project (fixes #1095) #1106

Conversation

andrewn commented Jun 17, 2019 • edited

andrewn Jun 17, 2019

Choose a reason for hiding this comment

catarak Jun 17, 2019

Choose a reason for hiding this comment

andrewn Jul 4, 2019

Choose a reason for hiding this comment

catarak Jun 17, 2019

Choose a reason for hiding this comment

andrewn Jun 19, 2019 • edited

Choose a reason for hiding this comment

catarak Jun 17, 2019

Choose a reason for hiding this comment

andrewn commented Jun 19, 2019

andrewn commented Aug 3, 2019

catarak commented Aug 6, 2019

catarak commented Aug 6, 2019

catarak commented Aug 13, 2019

andrewn commented Aug 30, 2019 • edited

catarak commented Aug 30, 2019

andrewn commented Jun 17, 2019 •

edited

andrewn Jun 19, 2019 •

edited

andrewn commented Aug 30, 2019 •

edited