Dev (#223)

* reduce conns testing

* update

* bring back count one for lock test

* update

* debug conns in ci

* remove block test pool closing

* use tags for skipping count one - broken negative regex

* update tags

* trying to refactor post migrations to use regular migrate

* update and notes for handling single migration in parallel test suites

* remove migration tables from gen

* refactor

* fix ci call

* update

* only run post migrations up in x.migrate

* notes

* rename countone tag tests for clarity

* default ci env

* default ci env

* FIXME race condition in lock status check - require tx

* dynamic go test flags - eg shuffle for ci only

* test xo caching and attempt to use excludes

* fixme

* cache_all now handles exclusion regexes

* fix arg parsing in cache_all when main process receives args

* shuffle on ci

* refactor

* fix dumb driver swap

* refactor migrate

* notes

* debug ci

* update

* using postgres test db for each test suite

* ignore tracing test specific spans

* leave full path for testutil postgres dbs

* update

* remove debug ci

* update

* fix cache_all

* notes for fixed cleanpu

* no exit cleanup with flag

* increase ci testcount

* use postgres_test for lock tests

* refactor testutil newdb

* migrations lock

* fix

* ntoes

* ci not verbose

* refactor

* TODO - ignore constraints refactor

* refactor

* update

* ignore line endings in vendor

* ignoring constraints properly - pending restore of pagination when type is changed

* notes

* original type check for valid cursors

* cleanup

* notes

* auto install apt

* update

* update

* parallel typecheck exhaustruct

* fix typo

* vite host

* udpate

* bit of parallelism

* cleanup

* refactor

* TODO - automated shorthand schema name gen e.g. RestUser - User

* working on better rest gen

* update

* update

* always generate all rest models

* always generate rest models file

* FIXME

* always gen rest models replacing spec

* clashes update

* show line number for clashes

* refactor

* notes

* crud gen update

* fix data race

* implement catch rest refs that dont exist for automatic generation

* properly create arrays in go-utils

* dont print ast-parser results if empty

* easy caching of go bin tools

* notes

* refactor

* FIXME xo tests

* fix schema for xo tests gen

* utils

* use dedicated db for xo tests

* notes

* debug

* debug

* debug

* debug

* fix setup

* trying to update ref gen

* fixed automatic package gen

* remove component schemas gen from crud generator - will autogenerate

* fix ref schema gen when both missing

* clean

* shfmt

* shell lint

* sync

* notes

* use bare regex in place of ast-parser --create-generics-map

* ignore force regen for install-tools and refactor frontend gen

* speed up orval gen

* remove msw references

* notes

* broken - try to output rest structs without Rest prefix directly and overwrite

* update

* gen without rest models

* fix new rest gen

Author: danicc097

.gitattributes

@@ -0,0 +1 @@
+vendor/* -text

.github/actions/setup/action.yaml

@@ -45,16 +45,20 @@ runs:
         restore-keys: |
           ${{ runner.os }}-pnpm-store-
 
+    - name: Update
+      shell: bash
+      run: |
+        sudo apt-get update
+
     - name: Install dependencies
       shell: bash
       run: |
         source .envrc
 
-        sudo apt-get update
-
-        { yes || :; } | project bootstrap --x-env=ci
+        echo Bootstrapping...
+        { yes y 2>/dev/null || :; } | project bootstrap --x-debug
 
-        # project e2e.sync-deps --x-env=ci # won't use python playwright
+        # project e2e.sync-deps # won't use python playwright
 
         echo "$HOME/.local/bin" >> $GITHUB_PATH
         echo "$HOME/bin" >> $GITHUB_PATH

.github/workflows/openapi-check.yaml

@@ -1,5 +1,6 @@
 name: OpenAPI check
-
+env:
+  CI: true
 on:
   push:
     branches:

.github/workflows/tests.yaml

@@ -1,5 +1,6 @@
 name: Tests
-
+env:
+  CI: true
 on:
   push:
     branches:
@@ -29,9 +30,9 @@ jobs:
           # go mod download # it will access network/cache, which is not necessary with -mod=vendor
           go mod verify
 
-          project test.backend.setup --x-env=ci
+          project test.backend.setup
 
-          project lint --x-env=ci
+          project lint
 
       - name: Diff check
         run: |
@@ -42,13 +43,21 @@ jobs:
         run: |
           source .envrc
 
+          {
+            while sleep 1; do project db.conns-db postgres_test --x-no-exit-cleanup; done
+          } &
+          bg_pid=$!
+
           echo -------1-------
-          POSTGRES_TRACE=false project test.create-crud-gen -v --x-env=ci
+          POSTGRES_TRACE=false project test.create-crud-gen --x-debug
           echo -------2-------
-          POSTGRES_TRACE=false project test.backend -v -race -count=30 -timeout=40m --x-env=ci
+          POSTGRES_TRACE=false project test.backend -race -count=50 -timeout=40m
           echo -------3-------
           # some tests are skipped from race
-          POSTGRES_TRACE=false project test.backend -v -count=3 -timeout=40m --x-env=ci
+          POSTGRES_TRACE=false project test.backend -count=3 -timeout=40m
+
+          { test -n "$bg_pid" && kill $bg_pid; } || true
+
 
       - name: Postgres logs
         if: failure() # always()
@@ -58,13 +67,13 @@ jobs:
         run: |
           source .envrc
 
-          project test.frontend --x-env=ci
+          project test.frontend
 
       - name: Test E2E
         run: |
           source .envrc
 
-          project e2e.run --x-env=ci
+          project e2e.run
 
       - uses: actions/upload-artifact@v3
         if: always()

.vscode/settings.json

@@ -12,6 +12,7 @@
     "-count=1",
     "-v"
   ],
+  // "go.testEnvFile": null,
   // --
   "shellcheck.executablePath": "shellcheck",
   "shellcheck.customArgs": [
@@ -74,7 +75,7 @@
       "db/migrations/*.down.sql"
     ],
     "postMigrationFiles": [
-      "db/post-migration/*.sql"
+      "db/post-migrations/*.sql"
     ],
     "target": "all",
   },

README.md

@@ -84,20 +84,33 @@ Fill in `.env.template`'s:
 find . -name ".env*.template"
 ```
 
+For `dev` environment, you may use `env.ci` as a starting point,
+replacing `ci` with `dev` and using `API_PREFIX=""` since it will run
+the backend without docker containers.
+
 Assuming a recent Ubuntu release:
 
 ```bash
 
-sudo apt install direnv
+sudo apt install direnv -y
 direnv allow # you can also customize direnv with .envrc.local as you would a regular .envrc, see example
 
 cp openapi-go.code-workspace.example openapi-go.code-workspace # edit as desired
 
 project bootstrap # dependency and tools interactive installation
 project gen
+project recreate-shared-services
 project run-dev
 ```
 
+For first test run:
+
+```bash
+project test.backend.setup
+project test.backend
+project test.frontend
+```
+
 ### Tracing, monitoring...
 
 ```bash