Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement login/logout, registry, bandwidth used. #33

Merged
merged 27 commits into from Feb 23, 2021
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e764a1a
Implement login/logout.
ro-tex Feb 17, 2021
a88f5b2
Sort all uploads and downloads, most recent first.
ro-tex Feb 12, 2021
7106ee0
Make sure new downloads also have their meta fetched.
ro-tex Feb 12, 2021
f947969
Allow reporting on download status code and range.
ro-tex Feb 15, 2021
dc00859
User's data used is calculated on demand.
ro-tex Feb 15, 2021
0478934
Restore "storage_used".
ro-tex Feb 16, 2021
ce8626a
Report registry bandwidth use.
ro-tex Feb 16, 2021
fd1ebfc
Update upload bandwidth and storage prices.
ro-tex Feb 16, 2021
c8bb4f7
Reduce the amount of excess data being communicated back from the DB.
ro-tex Feb 16, 2021
8470814
Better log messages.
ro-tex Feb 16, 2021
4a4387c
Implement partial and full downloads, accommodated by the fetcher.
ro-tex Feb 16, 2021
2aedde0
Add a custom aggregation pipeline for downloads, so we can choose the…
ro-tex Feb 16, 2021
a1d3b10
Only count the bandwidth used during the current month. Months reset …
ro-tex Feb 17, 2021
d6f970f
Use the new base+increment bandwidth pricing for downloads.
ro-tex Feb 17, 2021
92b4f80
Uploads report their size based on storage used, not on their downloa…
ro-tex Feb 17, 2021
c8126b7
Run the integration test DB on a non-standard port, so it doesn't col…
ro-tex Feb 17, 2021
6aa0fb8
Always validate offset and page size.
ro-tex Feb 17, 2021
7d1c01c
Add some basic tests that validate used storage calculations.
ro-tex Feb 17, 2021
f098151
Minor logging changes.
ro-tex Feb 18, 2021
6b3d758
Fix JSON field name.
ro-tex Feb 18, 2021
595f50f
Call `UserUpdateUsedStorage` from within `UploadCreate`.
ro-tex Feb 18, 2021
0a7b7d4
Return error on zero-size downloads.
ro-tex Feb 18, 2021
1bd5bc8
Respond with 200 to zero-sized downloads but do not process them.
ro-tex Feb 18, 2021
b3838a0
Add some tests.
ro-tex Feb 18, 2021
54f47ad
Address PR remarks.
ro-tex Feb 19, 2021
ecc001d
Address PR remarks.
ro-tex Feb 23, 2021
cd01adb
Fix a broken test.
ro-tex Feb 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
131 changes: 89 additions & 42 deletions API.md
Expand Up @@ -4,19 +4,18 @@

### ORY, Kratos, Oathkeeper, and JWx

While `skynet-accounts` handles account information in the context of a Skynet
portal, the baseline account management (account CRUD, email verification,
password resets, etc.) is handled by [ORY](https://www.ory.sh/)
([Kratos](https://www.ory.sh/kratos/) and [Oathkeeper](https://www.ory.sh/oathkeeper/))
to which we often refer to as "Kratos". This also covers the login/logout
process and the issuance of JTW tokens. When we talk about JWT (or JWK, or JWKS)
While `skynet-accounts` handles account information in the context of a Skynet portal, the baseline account management (
account CRUD, email verification, password resets, etc.) is handled by [ORY](https://www.ory.sh/)
([Kratos](https://www.ory.sh/kratos/) and [Oathkeeper](https://www.ory.sh/oathkeeper/))
to which we often refer to as "Kratos". This also covers the login/logout process and the issuance of JTW tokens. When
we talk about JWT (or JWK, or JWKS)
we mean the tokens issued by ORY.

The workflow of verification follows a simple pattern:
* Oathkeeper exposes a public link on which it shares the public keys with
which anyone can verify the validity of the JWT tokens it issues.
* `skynet-accounts` fetches those keys and uses them to validate the JWTs it
receives in requests.

* Oathkeeper exposes a public link on which it shares the public keys with which anyone can verify the validity of the
JWT tokens it issues.
* `skynet-accounts` fetches those keys and uses them to validate the JWTs it receives in requests.

### User tiers

Expand All @@ -28,89 +27,137 @@ The tiers communicated by the API are numeric. This is the mapping:
3. Premium 20.
4. Premium 80.

## Auth endpoints

### POST `/login`

Sets the `skynet-jwt` cookie.

* Requires valid JWT: `true`
* GET params: none
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500

### POST `/logout`

Removes the `skynet-jwt` cookie.

* Requires valid JWT: `true`
* GET params: none
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500

## User endpoints

### GET `/user`

This request combines the "get user data" and "create user" requests - if the
users exists in the DB, their data will be returned. If they don't exist in the
DB, an account will be created on the Free tier.
This request combines the "get user data" and "create user" requests - if the users exists in the DB, their data will be
returned. If they don't exist in the DB, an account will be created on the Free tier.

* Requires valid JWT: `true`
* Returns:
- 200 JSON object
- 200 JSON object
```json
{
"tier": 1
}
```
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)

### PUT `/user` (TODO)

This endpoint allows us to update the user's tier, membership expiration dates,
etc.
This endpoint allows us to update the user's tier, membership expiration dates, etc.

* Requires valid JWT: `true`
* POST params:
- TBD
* Returns:
- 200 JSON object
- 200 JSON object
```json
{
"tier": 1
}
```
- 400
- 401 (missing JWT)
- 500
- 400
- 401 (missing JWT)
- 500

### GET `/user/uploads`

Returns a list of all skylinks uploaded by the user.

* Requires valid JWT: `true`
* Returns:
- 200 JSON Array (TBD)
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)
- 200 JSON Array (TBD)
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)

### GET `/user/downloads`

Returns a list of all skylinks downloads by the user.

* Requires valid JWT: `true`
* Returns:
- 200 JSON Array (TBD)
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)
- 200 JSON Array (TBD)
- 401 (missing JWT)
- 424 (when there is no such user, and we fail to create it)
- 500 (on any other error)

## Reports endpoints

### POST `/track/upload/:skylink`

* Requires valid JWT: `true`
* GET params:
- skylink: just the skylink hash, no path, no protocol
- skylink: just the skylink hash, no path, no protocol
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500
- 204
- 400
- 401 (missing JWT)
- 500

### POST `/track/download/:skylink`

* Requires valid JWT: `true`
* GET params:
- skylink: just the skylink hash, no path, no protocol
- skylink: just the skylink hash, no path, no protocol
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500

### POST `/track/registry/read`

* Requires valid JWT: `true`
* GET params: none
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500

### POST `/track/registry/write`

* Requires valid JWT: `true`
* GET params: none
* POST params: none
* Returns:
- 204
- 400
- 401 (missing JWT)
- 500
2 changes: 1 addition & 1 deletion Makefile
Expand Up @@ -64,7 +64,7 @@ start-mongo:
--rm \
--detach \
--name skynet-accounts-mongo-test-db \
-p 127.0.0.1:27017:27017 \
-p 127.0.0.1:17017:27017 \
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=aO4tV5tC1oU3oQ7u \
mongo
Expand Down
5 changes: 4 additions & 1 deletion README.md
@@ -1,6 +1,7 @@
# skynet-accounts

`skynet-accounts` is a service that stores [Skynet](https://siasky.net) user account data. It uses MongoDB for data storage. It uses ORY Kratos for the actual account management.
`skynet-accounts` is a service that stores [Skynet](https://siasky.net) user account data. It uses MongoDB for data
storage. It uses ORY Kratos for the actual account management.

## Setup steps

Expand All @@ -9,6 +10,7 @@
All local secrets are loaded from a `.env` file in the root directory of the project.

Those are (example values):

```.env
SKYNET_DB_HOST="localhost"
SKYNET_DB_PORT="27017"
Expand All @@ -21,5 +23,6 @@ COOKIE_ENC_KEY=""
```

## Recommended reading

- [JSON and BSON](https://www.mongodb.com/json-and-bson)
- [Using the official MongoDB Go driver](https://vkt.sh/go-mongodb-driver-cookbook/)
6 changes: 2 additions & 4 deletions api/api.go
Expand Up @@ -4,14 +4,12 @@ import (
"encoding/json"
"net/http"

"github.com/NebulousLabs/skynet-accounts/metafetcher"

"github.com/sirupsen/logrus"

"github.com/NebulousLabs/skynet-accounts/build"
"github.com/NebulousLabs/skynet-accounts/database"
"github.com/NebulousLabs/skynet-accounts/metafetcher"

"github.com/julienschmidt/httprouter"
"github.com/sirupsen/logrus"
"gitlab.com/NebulousLabs/errors"
)

Expand Down
13 changes: 7 additions & 6 deletions api/cookie.go
Expand Up @@ -6,13 +6,12 @@ import (
"time"

"github.com/gorilla/securecookie"
"github.com/joho/godotenv"
)

const (
// CookieName is the name of the cookie where we store the user's JWT token.
CookieName = "skynet-jwt"
// cookieValidity specifies for how long the cookie is valid. In seconds.
cookieValidity = 604800 // one week

// envCookieDomain holds the name of the environment variable for the
// domain name of the portal
Expand All @@ -27,14 +26,17 @@ const (

var (
secureCookie = func() *securecookie.SecureCookie {
var hashKey = []byte(os.Getenv(envCookieHashKey))
var blockKey = []byte(os.Getenv(envCookieEncKey))
_ = godotenv.Load()
// These keys need to be *exactly* 16 or 32 bytes long.
peterjan marked this conversation as resolved.
Show resolved Hide resolved
var hashKey = []byte(os.Getenv(envCookieHashKey))[:32]
var blockKey = []byte(os.Getenv(envCookieEncKey))[:32]
return securecookie.New(hashKey, blockKey)
}()
)

// writeCookie is a helper function that writes the given JWT token as a
// secure cookie.
// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
func writeCookie(w http.ResponseWriter, token string, exp int64) error {
encodedValue, err := secureCookie.Encode(CookieName, token)
if err != nil {
Expand All @@ -51,8 +53,7 @@ func writeCookie(w http.ResponseWriter, token string, exp int64) error {
HttpOnly: true,
Path: "/",
Domain: domain,
Expires: time.Unix(exp, 0),
MaxAge: cookieValidity,
MaxAge: int(exp - time.Now().UTC().Unix()),
Secure: true, // do not send over insecure channels, e.g. HTTP
SameSite: 1, // https://tools.ietf.org/html/draft-ietf-httpbis-cookie-same-site-00
}
Expand Down