Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement API pipeline #140

Merged
merged 83 commits into from
Apr 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
f7d6551
Implement first draft
f-f Jan 24, 2021
a38b611
Fill in some more of the publishing pipeline
f-f Jan 24, 2021
971f498
Fix #16
f-f Jan 24, 2021
e7171c9
Upgrade to purs 0.14
f-f Mar 8, 2021
f3d2e57
Merge branch 'master' into api-pipeline
f-f Mar 8, 2021
3d81f79
Some fixmes
f-f Mar 17, 2021
4675246
Add simple SPDX license parsing
thomashoneyman Mar 18, 2021
94a3964
Switch to use parse function for SPDX licenses
thomashoneyman Mar 18, 2021
6894195
Read GitHub events and parse into an Operation
f-f Mar 18, 2021
3cf2245
Merge branch 'master' into api-pipeline
f-f Mar 18, 2021
9848b65
Minor tweaks while reading through
thomashoneyman Mar 18, 2021
b73f941
Add ability to comment on issues in the case of error.
thomashoneyman Mar 18, 2021
b3a7540
Update tests
thomashoneyman Mar 18, 2021
bed889a
Add ApiM monad and route errors to comments
thomashoneyman Mar 19, 2021
14a272d
Add a comment test
thomashoneyman Mar 19, 2021
7bf639c
Modify workflow to include NPM installation
thomashoneyman Mar 19, 2021
aefaacf
Move npm installation to ci directory
thomashoneyman Mar 19, 2021
d853635
Add setup-purescript to CI
thomashoneyman Mar 19, 2021
29bf66d
actions -> purescript-contrib
thomashoneyman Mar 19, 2021
414f143
Add GITHUB_TOKEN to environment
thomashoneyman Mar 19, 2021
447d12b
Add NPM caching / npm ci
thomashoneyman Mar 19, 2021
01a35a1
Add SPACES_KEY and SPACES_SECRET env variables
thomashoneyman Mar 19, 2021
361d021
Remove comment test
thomashoneyman Mar 19, 2021
5d53405
ApiM -> RegistryM and smart constructor for PackageName
thomashoneyman Mar 21, 2021
9ed46b8
Make a smart constructor for SPDX licenses
thomashoneyman Mar 22, 2021
e60d0c3
Fix main.yaml
thomashoneyman Mar 22, 2021
be051f4
Fix main.yaml
thomashoneyman Mar 22, 2021
1ad3394
Update the manifest to use newtypes
thomashoneyman Mar 22, 2021
24809a2
Merge branch 'api-pipeline' of github.com:purescript/registry into ap…
thomashoneyman Mar 22, 2021
9a59b80
Update cache location in main.yml
thomashoneyman Mar 22, 2021
7e1fa1e
Add npm install to ci.yml
thomashoneyman Mar 22, 2021
a7ae4b4
Add version newtype
thomashoneyman Mar 22, 2021
b23c67d
Merge branch 'api-pipeline' of github.com:purescript/registry into ap…
thomashoneyman Mar 22, 2021
86adfc9
Fix typo in SPDX parse function
thomashoneyman Mar 22, 2021
84a5427
Merge branch 'master' into api-pipeline
thomashoneyman Mar 22, 2021
21790ce
Read packages metadata
f-f Mar 22, 2021
fd189c5
Fix build
thomashoneyman Mar 22, 2021
dbe0322
Add custom prelude
f-f Mar 23, 2021
9e2f6c3
Add SemVer module, some more refactoring
f-f Mar 23, 2021
2f27f57
Remove draft support for revisions
f-f Mar 23, 2021
6f60ba5
Rename API CI job
f-f Mar 23, 2021
6552901
Fix tests
f-f Mar 23, 2021
79ab400
Update URL to our storage backend
f-f Mar 23, 2021
66d0925
Update ci/src/BowerImport.purs
f-f Mar 23, 2021
190f350
Update ci/src/BowerImport.purs
f-f Mar 23, 2021
dfc9427
Use commit function from reader
f-f Mar 23, 2021
ee847bf
Group modules in folders
f-f Mar 23, 2021
41ca227
Convert SemVer ranges to simple ones
f-f Mar 23, 2021
e6a3f1c
Update CI to use pacchettibotti token
f-f Mar 24, 2021
7feaa34
It's actually always traverse
f-f Mar 24, 2021
c44c18b
Implement push to master
f-f Mar 24, 2021
d224014
Run test pipeline!
f-f Mar 24, 2021
d2ebdc6
Fix encoding bug
f-f Mar 24, 2021
f59d43c
Point to master
f-f Mar 24, 2021
8e3f180
Get more git logs
f-f Mar 24, 2021
cab4a24
Use master ref
f-f Mar 24, 2021
99f3e5c
Back to default token
f-f Mar 24, 2021
977dd9e
Apply suggestions from code review
f-f Mar 25, 2021
83d09bd
Merge branch 'master' into api-pipeline
f-f Mar 25, 2021
208eeb1
Add Scripts folder
f-f Mar 26, 2021
fb89666
Fix module names
f-f Mar 26, 2021
d1fcfe4
Move metadata functions to the schema module
f-f Mar 26, 2021
d1ec723
Detail TODO
f-f Mar 26, 2021
c242906
Apply suggestions from code review
f-f Mar 26, 2021
c566efa
Catch FS errors
f-f Mar 26, 2021
12a85ba
Inline fromJsonFile
f-f Mar 26, 2021
2799a02
Test commenting
f-f Mar 26, 2021
b96d7e2
Fix CI
f-f Mar 26, 2021
fc07c21
Fix tests
f-f Mar 26, 2021
63c5bde
Fix dependency list
f-f Mar 28, 2021
1cbf3c6
Merge branch 'master' into api-pipeline
thomashoneyman Mar 30, 2021
c27907d
Pix nixpkgs to 20.09
f-f Mar 30, 2021
08dd764
Add type synonym for version number
f-f Mar 30, 2021
4fbfa89
Move mkEnv to API module
f-f Mar 30, 2021
6bb8b20
Include original representation when parsing semver range
f-f Mar 30, 2021
2b183ef
Rename FFI modules to Foreign
f-f Mar 30, 2021
d7a9021
Fix TODOs
f-f Mar 30, 2021
4130169
Remove maintainers for now
f-f Mar 30, 2021
97292cf
Fix build
f-f Mar 30, 2021
d9960a4
Merge branch 'master' into api-pipeline
f-f Apr 2, 2021
2e8fbb1
Merge branch 'master' into api-pipeline
f-f Apr 4, 2021
aa70706
Add check for build-metadata
f-f Apr 4, 2021
51ae3d8
Update package-lock
f-f Apr 4, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 37 additions & 6 deletions .github/workflows/API.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,45 @@ on:
issue_comment:
types: [created]
jobs:
run:
api:
# This condition is here because the issue_comment event is triggered
# for PRs too, but we don't want that here
if: ${{ !github.event.issue.pull_request }}
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.PACCHETTIBOTTI_TOKEN }}
SPACES_KEY: ${{ secrets.SPACES_KEY }}
SPACES_SECRET: ${{ secrets.SPACES_SECRET }
steps:
- uses: actions/checkout@v2
- run: echo ${GITHUB_EVENT_NAME}
- run: cat ${GITHUB_EVENT_PATH}
# TODO: write some PureScript that reads this file, gets the body of the issue/comment,
# tries to conform it to an Operation, and fails the build if that doesn't work
- uses: actions/checkout@v2
with:
token: ${{ secrets.PACCHETTIBOTTI_TOKEN }}
ref: master

- uses: actions/setup-node@v2
with:
node-version: "14"

- uses: actions/cache@v2
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('./ci/**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- uses: purescript-contrib/setup-purescript@main

- name: Cache PureScript dependencies
uses: actions/cache@v2
with:
key: ${{ runner.os }}-spago-${{ hashFiles('./ci/**/*.dhall') }}
path: |
.spago
output

- uses: cachix/install-nix-action@v12
with:
nix_path: nixpkgs=channel:nixos-20.09

- name: "Run API"
run: cd ci && npm ci && nix-shell ../default.nix --run "spago run -m Registry.API"
60 changes: 45 additions & 15 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,55 @@
name: tests

on:
push:
push:
branches: [master]
pull_request:

jobs:
tests:
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SPACES_KEY: ${{ secrets.SPACES_KEY }}
SPACES_SECRET: ${{ secrets.SPACES_SECRET }}
steps:
# Setup
- uses: actions/checkout@v2
- uses: cachix/install-nix-action@v12
with:
nix_path: nixpkgs=channel:nixos-20.09

# Pro-tempore checks, should go away once new registry is in place
- run: ./verify-registry-unique.sh

# Checks for the Registry Draft
- name: "Check that all Dhall compiles, and examples correctly conform to a Manifest"
run: ./ci/scripts/verify-dhall.sh
- name: "Check all the example Manifests roundtrip (read+write) through PureScript"
run: cd ci && nix-shell ../default.nix --command "spago run -m Registry.Check.ManifestRoundtrip"
# Setup
- uses: actions/checkout@v2

- uses: actions/setup-node@v2
with:
node-version: "14"

- uses: actions/cache@v2
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('./ci/**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- uses: purescript-contrib/setup-purescript@main

- name: Cache PureScript dependencies
uses: actions/cache@v2
with:
key: ${{ runner.os }}-spago-${{ hashFiles('./ci/**/*.dhall') }}
path: |
.spago
output

- uses: cachix/install-nix-action@v12
with:
nix_path: nixpkgs=channel:nixos-20.09

# Pro-tempore checks, should go away once new registry is in place
f-f marked this conversation as resolved.
Show resolved Hide resolved
- run: ./verify-registry-unique.sh

# Checks for the Registry Draft
- name: "Check that all Dhall compiles, and examples correctly conform to a Manifest"
run: ./ci/scripts/verify-dhall.sh

- name: "Run tests"
run: cd ci && npm ci && nix-shell ../default.nix --run "spago test" && cd ..

- name: "Check all the example Manifests roundtrip (read+write) through PureScript"
run: cd ci && npm ci && nix-shell ../default.nix --run "spago run -m Registry.Scripts.ManifestRoundtrip"
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,8 +233,7 @@ Every package will have its own file in the `packages` folder of this repo.
You can see the schema of this file [here](./v1/Metadata.dhall), and the main reasons for this file to exist are to track:
- the upstream location for the sources of the package
- published versions and the SHA256 for their tarball as computed by [our CI](#Adding-a-new-package).
Note: these are going to be sorted in ascending order according to [SemVer](https://semver.org) - when in
doubt the sorting provided by [the `semver` package on NPM](https://www.npmjs.com/package/semver#comparison) is correct.
Note: these are going to be sorted in ascending order according to [SemVer](https://semver.org)
- unpublished versions together with the reason for unpublishing
- GitHub usernames of package maintainers, so that we'll be able to contact them if any action is needed for any of their packages

Expand Down Expand Up @@ -364,11 +363,11 @@ What has happened already:
New packages are referenced [in this file](./new-packages.json), while all the packages from the Bower
registry are referenced [here](./bower-packages.json)
- we have drafted how the registry should behave, what's the API, how things will look like, etc (this document)
- we set up the first [storage backend](#Storage-backends), maintained by the Packaging Team

What is happening right now:
- we're figuring out the last details of [the package `Manifest`](#The-Package-Manifest), which is the big blocker
for proceeding further, since it will be baked into all the tarballs uploaded to the storage.
- setting up a [storage backend](#Storage-backends) maintained by the Packaging Team
- writing up the [CI code](#The-Registry-CI) to import the Bower packages as described above

What will happen after this:
Expand Down
Loading