E2E Integration test environment setup

.github/workflows/run-e2e-integ-tests.yml

@@ -0,0 +1,47 @@
+name: Run E2E integration tests
+on: [push, pull_request]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    # We only want to run on external PRs, since internal PRs are covered by "push"
+    # This prevents this from running twice on internal PRs
+    if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
+    steps:
+      - uses: actions/checkout@v2
+
+      # In this step, this action saves a list of existing images,
+      # the cache is created without them in the post run.
+      # It also restores the cache if it exists.
+      - uses: satackey/action-docker-layer-caching@v0.0.11
+        # Ignore the failure of a step and avoid terminating the job.
+        continue-on-error: true
+        with:
+          key: mathesar-docker-cache-integ-tests-{hash}
+          restore-keys: |
+            mathesar-docker-cache-integ-tests-
+
+      - name: Copy env file
+        run: cp .env.example .env
+
+      # The code is checked out under uid 1001 - reset this to 1000 for the
+      # container to run tests successfully
+      - name: Fix permissions
+        run: sudo chown -R 1000:1000 .
+
+      - name: Build the stack
+        run: docker-compose up --build -d
+
+      - name: Sleep for 60 seconds
+        run: sleep 60s
+        shell: bash
+
+      # TODO: This needs to be handled inside the tests
+      - name: Run migrations
+        run: docker exec mathesar_service python manage.py migrate
+
+      - name: Run type installation
+        run: docker exec mathesar_service python install.py
+
+      - name: Run integ tests with pytest
+        run: docker exec mathesar_service pytest --browser chromium --browser webkit --browser firefox tests/

.github/workflows/run-pytest.yml

@@ -33,4 +33,4 @@ jobs:
         run: docker-compose up --build -d
 
       - name: Run tests with pytest
-        run: docker exec mathesar_service_1 pytest
+        run: docker exec mathesar_service pytest mathesar/

Dockerfile

@@ -1,21 +1,4 @@
-FROM python:3.9-buster
-
-# These should be run as a single command to avoid caching issues.
-# See: http://lenguyenthedat.com/docker-cache/
-RUN apt update && apt install -y sudo
-
-# Add mathesar user
-ENV PYTHONUNBUFFERED=1
-ENV DOCKERIZE_VERSION v0.6.1
-
-# Install dockerize
-RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
-    && tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
-    && rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz
-
-# Install node
-RUN curl -fsSL https://deb.nodesource.com/setup_14.x | bash -
-RUN apt install -y nodejs
+FROM pavish73/mathesar-base:latest-all
 
 # Change work directory
 WORKDIR /code/

README.md

@@ -41,8 +41,8 @@ docker-compose up
 
 If it's your first time running the application, you'll also need to run database migrations and install Mathesar types and functions:
 ```
-docker exec mathesar_service_1 python manage.py migrate
-docker exec -it mathesar_service_1 python install.py
+docker exec mathesar_service python manage.py migrate
+docker exec -it mathesar_service python install.py
 ```
 
 You should now have a web server and database server running. Opening `http://localhost:8000` in your browser will open the application. For sample table data, you can create a new table in the UI using the `patents.csv` file found in `/mathesar/tests/data`. 
@@ -85,19 +85,19 @@ If you'd like to run tests before pushing, here's how you do it:
 
 Backend tests:
 ```
-docker exec mathesar_service_1 pytest
+docker exec mathesar_service pytest
 ```
 
 Frontend tests:
 ```
-docker exec mathesar_service_1 bash -c "cd mathesar_ui && npm test"
+docker exec mathesar_service bash -c "cd mathesar_ui && npm test"
 ```
 
 ### Opening a shell in the container
 
 If you need to do some work on the container that's running the code, here's how you access it:
 ```
-docker exec -it mathesar_service_1 bash
+docker exec -it mathesar_service bash
 ```
 
 To open a PostgreSQL psql terminal for the data in Mathesar:

docker-compose.yml

@@ -3,6 +3,7 @@ version: "3.9"
 services:
   db:
     image: postgres:13
+    container_name: mathesar_db
     environment:
       - POSTGRES_DB=${POSTGRES_DB-mathesar_django}
       - POSTGRES_USER=${POSTGRES_USER-mathesar}
@@ -13,6 +14,7 @@ services:
       - ./.volumes:/var/lib/postgresql/data
   service:
     build: .
+    container_name: mathesar_service
     environment:
       - MODE=${MODE-PRODUCTION}
     command: dockerize -wait tcp://db:5432 -timeout 30s ./run.sh

mathesar_ui/README.md

@@ -49,7 +49,7 @@ If you want to add or remove packages, or bascially run any npm action, **always
 You can connect to the container and open the ui folder by running:
 
 ```bash
-docker exec -it mathesar_service_1 /bin/bash
+docker exec -it mathesar_service /bin/bash
 cd mathesar_ui
 ```
 
@@ -71,7 +71,7 @@ For guidelines on component development, refer [README of components](https://gi
 You can start storybook in dev mode by connecting to the container and then running `npm run storybook`:
 
 ```bash
-docker exec -it mathesar_service_1 /bin/bash
+docker exec -it mathesar_service /bin/bash
 
 root@c273da65c52d:/code/mathesar_ui$ cd mathesar_ui && npm run storybook
 ```

setup.cfg

@@ -12,3 +12,4 @@ env =
 	DJANGO_DATABASE_URL=postgres://mathesar:mathesar@db:5432/mathesar_django
 	MATHESAR_DATABASES=(mathesar_db_test|postgres://mathesar:mathesar@db:5432/mathesar_db_test)
 	TEST=True
+	MODE=DEVELOPMENT

tests/test_base_page.py

@@ -0,0 +1,3 @@
+def test_base_page(page):
+    page.goto('http://localhost:8000')
+    assert page.inner_text('h1') == 'Welcome to Mathesar!'