Merge staging into production - 2024-Jan-12 edition #1257

jacobdgm · 2024-01-12T20:53:48Z

Merge staging into production - 2024-Jan-12 edition:

APIs
- provenance json export API has been created
- notation json export API has been created
- concordances json export API has been created (concordances for all cantus IDs are saved in a single JSON object, which is updated daily by cron)
Chant Create page
- autocomplete widgets are now the same width as nearby fields
Chant Edit page
- autocomplete widgets are now the same width as nearby fields (this doesn't seem to have been applied to the Differentiae Database field - see Chant Edit: Differentiae Database field is very wide #1311 - but I don't think this should be enough to prevent merging of this PR)
Chant by Cantus ID page
- This view has been refactored/optimized to minimize database calls
Chant Detail page
- links to Chant List page have been reworked
Chant Inventory page
- URIs for this page are now constructed in the format /source/<source ID>/inventory/ instead of /index/?source=<source ID>
Chant List page
- URIs for this page are now constructed in the format /source/<source ID>/chants/ instead of /chants/?source=<source ID>
- links to the Chant List page have been reworked
- links to the Full Inventory page have been reworked
Feast Detail page
- links to the Chant List page have been reworked
(Full Index page - we now call this the Full Inventory page)
Full Inventory page
- This view has been refactored/optimized to minimize database calls
Source Create page
- autocomplete widgets are now the same width as nearby fields (this appears to not be strictly true, but they look a lot better than they were -JdGM)
Source Detail page
- In sidebar, replace "image gallery" with "images"
- Under "Full Source/Fragment", "Fragment" has been updated to read "Fragment or Fragmented"
- links to the Chant List page have been reworked
- links to the Full Inventory page have been reworked
Source Edit page
- links to the Chant List page have been reworked
- autocomplete widgets are now the same width as nearby fields
- links to the Full Inventory page have been reworked
Source List page:
- Title column has been moved to the left of the Siglum column
- An unnecessary </a> in the template was removed
Network configuration
- traffic to mass.cantusdatabase.org is now redirected to cantusdatabase.org
Under the hood:
- https certificate renewal is now done using Lego rather than Certbot, and is now run monthly rather than weekly
- our backup cron/shell jobs have been fixed and refactored
- Our Postgres configuration, which was being overwritten during the building of our docker containers, is now properly set up
- ujson added to requirements.txt, and ijson removed
- several unnecessary shell scripts related to fixtures have been removed
- AjaxSearchBarTest has been refactored so as to not fail at random anymore
- duplicate test in SourceEditChantsViewTest has been removed
- our deployment process has been streamlined, with separate development vs. staging vs. production versions of various configuration files
- settings.py now reads the site's ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS from environment variables, and our env file has been updated to specify these hosts/origins explicitly

…ith 'images'

- Uses docker rather than docker-compose (needed for cp command) - Fixes copy -> cp command

Fix postgres pack-up cron scripts

Source Detail/Edit, Chant List: in sidebar, replace 'image gallery' with 'images'

…urce-detail Change "Fragment" to "Fragment or Fragmented" on source detail page

Add css styling for autocomplete widgets

Move source title to the left of source siglum

Merge jacobdgm's develop into CantusDB's develop

Merge develop into staging, 2024 Feb 06 edition

Make separate production and staging versions of cron.txt

Bumps [django](https://github.com/django/django) from 4.2.3 to 4.2.10. - [Commits](django/django@4.2.3...4.2.10) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>

…oject/django-4.2.10 Bump django from 4.2.3 to 4.2.10 in /django/cantusdb_project

… config file path

Add postgres command with path to config file to docker-compose staging

Merge develop into staging, 2024 Feb 08 edition

…date

jacobdgm · 2024-02-08T18:46:35Z

I just opened #1308, which should fix the issue discovered at #1288 (comment). We should fix this before deploying these changes to Production.

…m environment variables

Ensure CSRF_TRUSTED_ORIGINS and ALLOWED_HOSTS are specified

Change URI construction for Chant List and Chant Inventory

Merge develop into staging, 2024 Feb 09 edition

jacobdgm · 2024-02-09T20:28:20Z

update_cached_concordances failed to run again this morning. Perhaps I missed a step when pulling changes yesterday? But the running crontab should not change based on docker containers coming down and going up.... This is a thing to check in on again early next week.

dchiller · 2024-02-09T20:59:08Z

update_cached_concordances failed to run again this morning.

The file that just downloaded when I went to staging.cantusdatabase.org/concordances has this at the end:

"metadata":{"last_updated":"2024-02-09T04:10:12.812415"}

Isn't that what we would expect if it had run successfully this morning?

jacobdgm · 2024-02-13T13:52:05Z

update_cached_concordances failed to run again this morning.

The file that just downloaded when I went to staging.cantusdatabase.org/concordances has this at the end:
"metadata":{"last_updated":"2024-02-09T04:10:12.812415"}
Isn't that what we would expect if it had run successfully this morning?

Checking again this morning, it seems to be doing fine. I had files named concordances, concordances-1, concordances-2, etc. in my Downloads folder - current hypothesis is that I inspected concordances, i.e. an older version of the download, when I was writing this comment on Friday.

jacobdgm · 2024-02-13T15:15:57Z

Everything looks good to go! Going to deploy these changes to the Production VM.

jacobdgm · 2024-02-13T16:19:52Z

These changes have been deployed, and are running on the Production server! All tests are passing! Good job everyone!

jacobdgm and others added 30 commits December 18, 2023 20:19

Source Detail/Edit, Chant List: in sidebar, replace 'image gallery' w…

529e613

…ith 'images'

Add automatic cert renewal with lego

69fc5ad

Update frequency of certificate renewal cron job

aeff7ed

Run pg_dump in in-container script for backup

567c48e

Make postgres_backup.sh executable on postgres container build

67451ef

Fixes backup cron commands

437b952

- Uses docker rather than docker-compose (needed for cp command) - Fixes copy -> cp command

Create single backup script

8103924

Merge pull request #1235 from dchiller/i-982-fix-postgres-backups

aca9824

Fix postgres pack-up cron scripts

source_list: Move source title to the left of source siglum

9978f7d

Merge pull request #1228 from jacobdgm/1224-image-gallery

d723945

Source Detail/Edit, Chant List: in sidebar, replace 'image gallery' with 'images'

add css styling for autocomplete widgets

1aff405

Source list: Change table column header order

5d82553

Change "Fragment" to "Fragment or Fragmented" on source detail page

46ec42d

Merge pull request #1243 from DDMAL/i-1242-change-fragment-lang-on-so…

a831ee7

…urce-detail Change "Fragment" to "Fragment or Fragmented" on source detail page

Merge pull request #1237 from lucasmarchd01/issue-1199

cf0c2e6

Add css styling for autocomplete widgets

urls: change chant-list uri construction

1ef3ba5

chant detail: reconfigure templates for new chant-list uri

b60ed6e

chant list: reconfigure templates for new chant-list uri

824cb31

feast detail: reconfigure templates for new chant-list uri

d9408d4

source detail: reconfigure templates for new chant-list uri

4015593

source edit: reconfigure templates for new chant-list uri

09da4ab

test views: reconfigure tests for new chant-list uri

90f1b07

chant list: reconfigure ChantListView for new chant-list uri

7c06782

urls: change chant-index uri construction

d4bb2e2

chant list: reconfigure templates for new chant-inventory uri

169dd83

rename full_index template to full_inventory

5b9cdaa

source detail: reconfigure templates for new chant-inventory uri

4bcc919

source edit: reconfigure templates for new chant-inventory uri

5b2d1d2

test views: reconfigure tests for new chant-inventory uri

a2410b7

Merge pull request #1236 from lucasmarchd01/issue-1223

a80b235

Move source title to the left of source siglum

jacobdgm and others added 14 commits February 6, 2024 11:58

docker-compose.yml.development: Re-add 443:443 port mapping

dee9b1b

Merge 1269-prod-stag-dev-config into develop

74dad5e

Merge pull request #1297 from jacobdgm/develop

383f69d

Merge jacobdgm's develop into CantusDB's develop

Merge pull request #1298 from DDMAL/develop

86ad579

Merge develop into staging, 2024 Feb 06 edition

rename cron-staging.txt -> cron.txt.staging

d5321bb

rename cron.txt -> cron.txt.production

9197b83

add cron.txt to .gitignore

36f9b75

Add postgres command with path to config file to docker-compose staging

aa920f3

Merge pull request #1299 from jacobdgm/1269-standardize-cron

0383346

Make separate production and staging versions of cron.txt

Merge pull request #1302 from DDMAL/dependabot/pip/django/cantusdb_pr…

9ef6b7b

…oject/django-4.2.10 Bump django from 4.2.3 to 4.2.10 in /django/cantusdb_project

Update docker-compose.yml.development with postgres command including…

c2891b3

… config file path

Merge pull request #1300 from DDMAL/fix-docker-compose-staging

69ae2f5

Add postgres command with path to config file to docker-compose staging

Merge pull request #1304 from DDMAL/develop

08f807e

Merge develop into staging, 2024 Feb 08 edition

jacobdgm marked this pull request as ready for review February 8, 2024 18:13

jacobdgm requested review from dchiller and lucasmarchd01 February 8, 2024 18:13

Ensure CSRF_TRUSTED_ORIGINS and ALLOWED_HOSTS are complete and up-to-…

86616b0

…date

jacobdgm and others added 5 commits February 8, 2024 20:16

Reconfigure settings.py to read allowed hosts and trusted origins fro…

deb8910

…m environment variables

Merge pull request #1308 from jacobdgm/1288-allowed-hosts

9a9507e

Ensure CSRF_TRUSTED_ORIGINS and ALLOWED_HOSTS are specified

Merge branch 'develop' into issue-1202

cd8b601

Merge pull request #1245 from lucasmarchd01/issue-1202

57092ca

Change URI construction for Chant List and Chant Inventory

Merge pull request #1309 from DDMAL/develop

6b7a782

Merge develop into staging, 2024 Feb 09 edition

jacobdgm merged commit 07be678 into production Feb 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge staging into production - 2024-Jan-12 edition #1257

Merge staging into production - 2024-Jan-12 edition #1257

jacobdgm commented Jan 12, 2024 •

edited

jacobdgm commented Feb 8, 2024

jacobdgm commented Feb 9, 2024

dchiller commented Feb 9, 2024

jacobdgm commented Feb 13, 2024

jacobdgm commented Feb 13, 2024

jacobdgm commented Feb 13, 2024

Merge staging into production - 2024-Jan-12 edition #1257

Merge staging into production - 2024-Jan-12 edition #1257

Conversation

jacobdgm commented Jan 12, 2024 • edited

jacobdgm commented Feb 8, 2024

jacobdgm commented Feb 9, 2024

dchiller commented Feb 9, 2024

jacobdgm commented Feb 13, 2024

jacobdgm commented Feb 13, 2024

jacobdgm commented Feb 13, 2024

jacobdgm commented Jan 12, 2024 •

edited