Merge Staging into Production, 2023-06-16 edition

README.md

@@ -11,16 +11,29 @@ Discrepancies between text stored in the CantusDB database and the manuscript te
 
 ### Differences in functionality/behavior:
 #### Visible to All Users (Logged-In and Anonymous)
+- General:
+  - The formatting/style of the entire site is slightly different.
+  - In some places, links (to Genre, Office, Feast detail pages, etc.) have been added where there were no links in OldCantus.
+  - In some places, title attributes have been added, especially where abbreviations are displayed (visible on mouse-over).
+  - Certain pages have an upward-pointing arrow at the bottom that brings the user back to the top of the page. We don't currently plan to implement this.
+  - If you click on the items in the header on OldCantus, it usually links you to one of the pages within the dropdown. On NewCantus, you need to open the dropdown and click on an item before you go to any of the pages. We have no plans to implement this on NewCantus, as OldCantus's system is hard to use on mobile devices (for a sad chuckle, check out https://cantus.uwaterloo.ca/about, which you get when you click/tap on "About")
 - Homepage:
   - When logged out, OldCantus displays a sidebar displaying recently posted articles. NewCantus displays this sidebar, but only displays the articles' titles, rather than the full content of the article (Currently, we don't plan to display the full content of news articles here)
 - Article List page:
   - Images from within the news stories are not displayed on the Article List page, whereas they are in OldCantus. (Currently, we don't plan to display images on this page)
   - Articles were manually copied from OldCantus to New, and when I initially did this, I preserved the date but not the time. It's fine that most of the timestamps say "00:00".
+- Source List:
+  - NewCantus is paginating the sources list page to display 100 sources per page. This is not implemented in OldCantus and is an intentional chant
+- Chant Search:
+  - New Cantus will do a full text search by keyword by default whereas OldCantus will display options to do either a full text search or a Cantus Index search (if the user clicks on the main Chant Search menu option)
+- Provenance detail page - on OldCantus, the Provenance Detail page displays only the provenance's name/title. On NewCantus, this page displays a list of sources that have this provenance.
 - Several of the APIs from OldCantus are not implemented in NewCantus, or are implemented differently:
   - json-activity (accessed via https://cantus.uwaterloo.ca/json-activity and https://cantus.uwaterloo.ca/json-activity?all) is not implemented in NewCantus ([Discussion](https://github.com/DDMAL/CantusDB/issues/126))
   - json-analysis-export (accessed, we think, via [https://cantus.uwaterloo.ca/json-analysis-export/?src=\<source id\>](https://cantus.uwaterloo.ca/json-analysis-export/?src=123591)) is not implemented in NewCantus ([Discussion](https://github.com/DDMAL/CantusDB/issues/124))
   - some of the keys in the json generated by json-node (accessed via [https://cantus.uwaterloo.ca/json-node/\<id\>](https://cantus.uwaterloo.ca/json-node/123591)) are different in NewCantus ([Discussion](https://github.com/DDMAL/CantusDB/issues/106))
-  - in the csv API, if the source in question has only sequences and no chants, OldCantus outputs a csv file with only a header and no rows. In this situation, NewCantus outputs a list of all the sequences in the source using the same headers as for chant sources.
+  - in the csv API,
+    - if the source in question has only sequences and no chants, OldCantus outputs a csv file with only a header and no rows. In this situation, NewCantus outputs a list of all the sequences in the source using the same headers as for chant sources.
+    - OldCantus has a `cao_concordances` column. We were asked not to display chants' CAO concordances anywhere, so this column has been left out. (see [issue 125](https://github.com/DDMAL/CantusDB/issues/125))
 - all people are represented as Users in NewCantus. This is in contrast to OldCantus, where there were two separate lists of people: one of Indexers and one of Users. ([Discussion](https://github.com/DDMAL/CantusDB/issues/218))
 
 #### Logged-In Users

django/cantusdb_project/align_text_mel.py

@@ -84,6 +84,12 @@ def syllabize_text(text, pre_syllabized=False):
 
 
 def syllabize_melody(volpiano):
+    # there exist several chants in the database whose volpiano is just `\n` -
+    # if a chant essentially has no volpiano, bail out early.
+    volpiano = volpiano.strip()
+    if not volpiano:
+        return []
+
     # the clef in volpiano should be 1--- with three dashes, if missing any dash, insert it
     if volpiano[1] != "-":
         volpiano = volpiano[:1] + "-" + volpiano[1:]

django/cantusdb_project/articles/templates/article_list.html

@@ -14,11 +14,14 @@ <h4>
                 <a href="{% url 'article-detail' article.id %}">{{ article.title }}</a>
             </h4>
             <div class="container">
-                {{ article.body|safe|truncatechars_html:3000 }}
+                <small>
+                {{ article.body.html|safe|truncatechars_html:3000 }}
+                </small>
             </div>
         </div>
     </div>
     {% endfor %}
+    {% include "pagination.html" %}
     <br>
 </div>
 {% endblock %}

django/cantusdb_project/articles/tests/test_articles.py

@@ -1,20 +1,24 @@
 from django.test import TestCase
 from django.urls import reverse
 from articles.models import Article
+from faker import Faker
 from main_app.tests.make_fakes import (
-    make_fake_text,
     make_fake_user,
 )
 
 # run with `python -Wa manage.py test articles.tests.test_articles`
 # the -Wa flag tells Python to display deprecation warnings
 
+# Create a Faker instance with locale set to Latin
+faker = Faker("la")
+
 
 def make_fake_article(user=None):
     if user is None:
         user = make_fake_user()
     article = Article.objects.create(
-        title=make_fake_text(max_size=12),
+        # updated to use Faker (previously called method from make_fakes that no longer exists)
+        title=faker.sentence(),
         author=make_fake_user(),
     )
     return article

django/cantusdb_project/articles/urls.py

@@ -1,8 +1,12 @@
 from django.urls import path
-from articles.views import ArticleDetailView
-from articles.views import ArticleListView
+from articles.views import (
+    ArticleDetailView,
+    ArticleListView,
+    article_list_redirect_from_old_path,
+)
 
 urlpatterns = [
     path("articles/", ArticleListView.as_view(), name="article-list"),
     path("article/<int:pk>", ArticleDetailView.as_view(), name="article-detail"),
+    path("news/", article_list_redirect_from_old_path),
 ]

django/cantusdb_project/articles/views.py

@@ -1,4 +1,5 @@
-from django.shortcuts import render
+from django.shortcuts import render, redirect
+from django.urls.base import reverse
 from django.views.generic import DetailView, ListView
 from articles.models import Article
 
@@ -12,6 +13,10 @@ class ArticleDetailView(DetailView):
 class ArticleListView(ListView):
     model = Article
     queryset = Article.objects.order_by("-date_created")
-    paginate_by = 100
+    paginate_by = 10
     context_object_name = "articles"
     template_name = "article_list.html"
+
+
+def article_list_redirect_from_old_path(request):
+    return redirect(reverse("article-list"))

django/cantusdb_project/cantusdb/settings.py

@@ -163,8 +163,34 @@
 # https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-auto-created-primary-keys
 DEFAULT_AUTO_FIELD = "django.db.models.AutoField"
 
-EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"
-
+EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
+EMAIL_HOST = "email-smtp.us-west-2.amazonaws.com"
+EMAIL_PORT = 587
+EMAIL_HOST_USER = os.getenv("AWS_EMAIL_HOST_USER")
+EMAIL_HOST_PASSWORD = os.getenv("AWS_EMAIL_HOST_PASSWORD")
+EMAIL_USE_TLS = True
+
+DEFAULT_FROM_EMAIL = "noreply@cantusdatabase.simssa.ca"
+
+# automatically disable all panels which user can then manually enable
+DEBUG_TOOLBAR_CONFIG = {
+    "DISABLE_PANELS": {
+        "debug_toolbar.panels.history.HistoryPanel",
+        "debug_toolbar.panels.versions.VersionsPanel",
+        "debug_toolbar.panels.timer.TimerPanel",
+        "debug_toolbar.panels.settings.SettingsPanel",
+        "debug_toolbar.panels.headers.HeadersPanel",
+        "debug_toolbar.panels.request.RequestPanel",
+        "debug_toolbar.panels.sql.SQLPanel",
+        "debug_toolbar.panels.staticfiles.StaticFilesPanel",
+        "debug_toolbar.panels.templates.TemplatesPanel",
+        "debug_toolbar.panels.cache.CachePanel",
+        "debug_toolbar.panels.signals.SignalsPanel",
+        "debug_toolbar.panels.logging.LoggingPanel",
+        "debug_toolbar.panels.redirects.RedirectsPanel",
+        "debug_toolbar.panels.profiling.ProfilingPanel",
+    },
+}
 
 INTERNAL_IPS = [
     "127.0.0.1",

django/cantusdb_project/main_app/forms.py

@@ -45,6 +45,20 @@ def __init__(self, *args, **kwargs):
 """
 
 
+class NameModelChoiceField(forms.ModelChoiceField):
+    """
+    A custom ModelChoiceField that overrides the label_from_instance method
+    to display the object's name attribute instead of str(object).
+    This field is specifically designed for handling genre and office objects.
+    Rather than displaying the name along with its description, sometimes we
+    only want the shorthand notation for the genre and office objects.
+    (Eg. [AV] Antiphon verse --> AV)
+    """
+
+    def label_from_instance(self, obj):
+        return obj.name
+
+
 # 3 best
 class ChantCreateForm(forms.ModelForm):
     class Meta:
@@ -115,13 +129,19 @@ class Meta:
         help_text="Each folio starts with '1'.",
     )
 
-    office = forms.ModelChoiceField(
-        queryset=Office.objects.all().order_by("name"), required=False
+    # We use NameModelChoiceField here so the dropdown list of office/mass displays the name
+    # instead of [name] + description
+    office = NameModelChoiceField(
+        queryset=Office.objects.all().order_by("name"),
+        required=False,
     )
     office.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    genre = forms.ModelChoiceField(
-        queryset=Genre.objects.all().order_by("name"), required=False
+    # We use NameModelChoiceField here so the dropdown list of genres displays the name
+    # instead of [name] + description
+    genre = NameModelChoiceField(
+        queryset=Genre.objects.all().order_by("name"),
+        required=False,
     )
     genre.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
@@ -329,13 +349,19 @@ class Meta:
     )
     feast.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    office = forms.ModelChoiceField(
-        queryset=Office.objects.all().order_by("name"), required=False
+    # We use NameModelChoiceField here so the dropdown list of office/mass displays the name
+    # instead of [name] + description
+    office = NameModelChoiceField(
+        queryset=Office.objects.all().order_by("name"),
+        required=False,
     )
     office.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    genre = forms.ModelChoiceField(
-        queryset=Genre.objects.all().order_by("name"), required=False
+    # We use NameModelChoiceField here so the dropdown list of genres displays the name
+    # instead of [name] + description
+    genre = NameModelChoiceField(
+        queryset=Genre.objects.all().order_by("name"),
+        required=False,
     )
     genre.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
@@ -430,13 +456,17 @@ class Meta:
     )
     feast.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    office = forms.ModelChoiceField(
+    # We use NameModelChoiceField here so the dropdown list of office/mass displays the name
+    # instead of [name] + description
+    office = NameModelChoiceField(
         queryset=Office.objects.all().order_by("name"),
         required=False,
     )
     office.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
 
-    genre = forms.ModelChoiceField(
+    # We use NameModelChoiceField here so the dropdown list of genres displays the name
+    # instead of [name] + description
+    genre = NameModelChoiceField(
         queryset=Genre.objects.all().order_by("name"), required=False
     )
     genre.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})
@@ -594,7 +624,9 @@ class Meta:
             "image_link": TextInputWidget(),
         }
 
-    genre = forms.ModelChoiceField(
+    # We use NameModelChoiceField here so the dropdown list of genres displays the name
+    # instead of [name] + description
+    genre = NameModelChoiceField(
         queryset=Genre.objects.all().order_by("name"), required=False
     )
     genre.widget.attrs.update({"class": "form-control custom-select custom-select-sm"})

django/cantusdb_project/main_app/management/commands/add_all_users.py

@@ -3,6 +3,10 @@
 import csv
 from django.contrib.auth.models import Group
 from main_app.models import Source, Chant
+import string
+import secrets
+
+ALPHABET: str = string.ascii_letters + string.digits
 
 
 class Command(BaseCommand):
@@ -28,10 +32,16 @@ def handle(self, *args, **kwargs):
                 if created:
                     user.first_name = name
                     user.last_name = surname
+                    full_name: str = " ".join((name, surname))
+                    user.full_name = full_name
+
                     user.institution = institution
                     user.city = city
                     user.country = country
-                    user.set_password("cantusdb")
+
+                    password: str = "".join(secrets.choice(ALPHABET) for _ in range(32))
+                    user.set_password(password)
+
                     user.save()
 
                 if user.groups.first() is None:

django/cantusdb_project/main_app/management/commands/sync_indexers.py

@@ -69,12 +69,9 @@ def get_new_indexer(indexer_id):
             full_name=indexer_full_name,
             # assign random email to dummy users
             email=f"{faker.lexify('????????')}@fakeemail.com",
-            # leave the password empty for dummy users
-            # the password can't be empty in login form, so they can't log in
-            password="",
             old_indexer_id=indexer_id,
             is_indexer=True,
-        )
+        ).set_unusable_password()  # Set unusable password so the user can't log in or access reset password page
 
 
 class Command(BaseCommand):

django/cantusdb_project/main_app/models/source.py

@@ -75,6 +75,10 @@ class Source(BaseModel):
         get_user_model(), related_name="sources_user_can_edit", blank=True
     )
 
+    ######
+    # The following five fields have nothing to do with user permissions,
+    # instead they give credit to users who are indexers and are displayed
+    # on the user detail page as sources the user has contributed to.
     inventoried_by = models.ManyToManyField(
         get_user_model(), related_name="inventoried_sources", blank=True
     )
@@ -90,6 +94,7 @@ class Source(BaseModel):
     other_editors = models.ManyToManyField(
         get_user_model(), related_name="edited_sources", blank=True
     )
+    ######
 
     segment = models.ForeignKey(
         "Segment", on_delete=models.PROTECT, blank=True, null=True

django/cantusdb_project/main_app/templates/century_detail.html

@@ -9,7 +9,7 @@ <h3>{{ century.name }}</h3>
     <ul>
         {% for source in sources|dictsort:"title" %}
             <li>
-                <a href="{% url 'source-detail' source.id %}">
+                <a href="{% url 'source-detail' source.id %}" title="{{ source.siglum }}">
                     {{ source.title }}
                 </a>
             </li>

django/cantusdb_project/main_app/templates/chant_create.html

@@ -152,13 +152,11 @@ <h3>Create Chant</h3>
                         {{ form.melody_id }}
                     </div>
 
-                    <div class="form-row align-items-top">
-                        <div class="form-group m-1 col-lg">
-                            <small>{{ form.addendum.label_tag }}</small>
-                            {{ form.addendum }}
-                        </div>
+                    <div class="form-group m-1 col-lg">
+                        <small>{{ form.addendum.label_tag }}</small>
+                        {{ form.addendum }}
                     </div>
-
+                    
                 </div>
 
                 <div class="form-row align-items-end">