Merge pull request #371 from raft-tech/raft-tdp-main

Multi-Issue PR

.circleci/config.yml

@@ -498,7 +498,7 @@ jobs:
           cf-space: tanf-staging
           cf-username: CF_USERNAME_STAGING
 
-  deploy-live-comms:
+  deploy-project-updates-site:
     parameters:
       cf-org:
         default: CF_ORG
@@ -524,8 +524,8 @@ jobs:
           cf-space: <<parameters.cf-space>>
           cf-username: <<parameters.cf-username>>
       - run:
-          name: Deploy Live Comms Site
-          command: ./scripts/deploy-live-comms.sh rolling tdp-live-comms
+          name: Deploy TDP Project Updates Site
+          command: ./scripts/deploy-tdp-product-update-site.sh rolling tdp-project-updates
 
   deploy-prod:
     executor: docker-executor
@@ -595,7 +595,7 @@ workflows:
   staging-deployment:
     unless: << pipeline.parameters.run_dev_deployment >>
     jobs:
-      - deploy-live-comms:
+      - deploy-project-updates-site:
           filters:
             branches:
               only:

README.md

@@ -18,19 +18,22 @@ Our vision is to build a new, secure, web-based data reporting system to improve
 # Table of Contents
 
 + **[Background](./docs/Background)**: Project, agency, legacy system, TDP prototype, acquisition, and program background
-+ **[Product-Strategy](./Product-Strategy)**: Product Vision, roadmap, and planning
++ **[Product-Strategy](./docs/Product-Strategy)**: Product Vision, roadmap, and planning
 + **[How-We-Work](./docs/How-We-Work)**: Team composition, sprint schedule, regular meetings, and workflows
-+ **[Security-Compliance](./Security-Compliance)**: Supplementary information in support of the ATO process
++ **[Security-Compliance](./docs/Security-Compliance)**: Supplementary information in support of the ATO process
 + **[Sprint-Review](./docs/Sprint-Review)**: Summaries of delivered stories per sprint
 + **[Technical-Documentation](./docs/Technical-Documentation)**: Architectural Decision Records, System documentation; technical workflows
 + **[User-Experience](./docs/User-Experience)**: Research-related project background, strategy and planning documents, and research syntheses
-+ **[Frontend](./tdrs-frontend)**: Frontend ReactJS codebase
-+ **[Backend](./tdrs-backend)**: Django codebase for backend
++ Codebase
+  + **[Frontend](./tdrs-frontend)**: Frontend ReactJS codebase
+  + **[Backend](./tdrs-backend)**: Backend Django codebase
+  + **[Terraform](./terraform)**: Documentation and syntax on CI process for automated provisioning of Cloud.gov brokered services
 + Web-based tools
-    + **[Figma]():** Design
+    + **[Figma](https://www.figma.com/file/irgQPLTrajxCXNiYBTEnMV/TDP-Mockups-For-Feedback):** Design
     + **[MURAL](https://app.mural.co/t/raft2792):** User research and product planning collaboration
-    + **[HHS Teams]()**: File storage, historic chats
-    + **[Zenhub]()**: Tracking issues
+    + **[ACF's TDP Sharepoint Site](https://hhsgov.sharepoint.com/sites/TANFDataPortalOFA/Shared%20Documents/Forms/AllItems.aspx)**: File storage, historic chats
+    + **[Zenhub](https://app.zenhub.com/workspaces/tdrs-sprint-board-5f18ab06dfd91c000f7e682e/board?repos=281707402)**: Tracking issues
+    + **[Product Updates](./product-updates)**: communication on project updates and research findings to the broader TDP stakeholders and target users
 
 ## Infrastructure
 
@@ -47,7 +50,7 @@ See [Architecture Decision Record 005 - Application Authentication](https://gith
 
 [Cloud.gov](https://cloud.gov/) is being used as the cloud environment. This platform-as-a-service (PaaS) removes almost all of the infrastructure monitoring and maintenance from the system, is already procured for OFA, and has a FedRAMP Joint Authorization Board Provisional Authority to Operate (JAB P-ATO) on file. 
 
-See [Architecture Decision Record 003 - Application Hosting](https://github.com/raft-tech/TANF-app/blob/raft-tdp-main/docs/Technical-Documentation/Architecture-Decision-Record/003-Application-hosting) - for more details.
+See [Architecture Decision Record 003 - Application Hosting](https://github.com/raft-tech/TANF-app/blob/raft-tdp-main/docs/Technical-Documentation/Architecture-Decision-Record/003-Application-hosting.md) - for more details.
 ### CI/CD Pipelines with CircleCI
 
 #### Continuous Integration (CI)

docs/How-We-Work/our-priorities-values-expectations.md

@@ -76,6 +76,7 @@ Also, review to ensure methods, variables, etc. are appropriately named. For met
 
 Review to ensure each screen follows the guidelines below to meet the accessibility WCAG2.1 AA performance standard (**_see dropdown below for more details on how gov will test a11y_**). Aditionally, `raft review` should include documenting evidence that the guidelines below were followed. 
 
+- Follow [Raft’s Accessibility Do’s and Don’ts](https://github.com/raft-tech/TANF-app/blob/raft-tdp-main/docs/Technical-Documentation/rafts-accessibility-dos-and-donts.md#raftsaccessibility-dos-and-donts) guidelines
 - Use [DHS Trusted Tester v5 Conformance Test](https://section508coordinators.github.io/TrustedTester/), [Tota11y](https://github.com/Khan/tota11y), and [WAVE](https://wave.webaim.org/)
 - Run the Accessibility Insight for [Web “Fast Pass” tool](https://accessibilityinsights.io/docs/en/web/getstarted/fastpass)
 - Run the Accessibility Insight for Web "Manual test for tab stops"

docs/Technical-Documentation/Architecture-Decision-Record/009-git-workflow.md

@@ -50,7 +50,7 @@ The proposed workflow below provides a remedy to these issues, as well as many o
 
 5. Assigned Raft reviewer(s) perform the review and/or requests changes.
     * For project management, research, and design work, review is conducted async.
-    * For development work, author is expected to schedule a [tabletop meeting]() at least 48 hours after `raft review` label added. 
+    * For development work, author is expected to schedule a [tabletop meeting](https://github.com/raft-tech/TANF-app/docs/How-We-Work/Developer-Tabletops.md) at least 48 hours after `raft review` label added. 
     * When changes are requested, the changes are made by the author.
     * When satisfied, the reviewer(s) confirm that ACs are met,  `approve` the PR, remove `raft review` and add `QASP review` labels.
         * See exceptions in [Notes](https://github.com/raft-tech/TANF-app/blob/dffd79adf7a5ae87cf1a93c8adf655c76cf45089/docs/Architecture%20Decision%20Record/009-git-workflow.md#notes) section

docs/Technical-Documentation/Architecture-Decision-Record/013-download-strategy.md

@@ -1,6 +1,6 @@
 # 13. Download Strategy
 
-Date: 2021-04-06 (_Updated 2021-12-29_)
+Date: 2021-04-06 (_Updated 2022-02-07_)
 
 ## Status
 
@@ -33,6 +33,56 @@ In light of these facts we have decided to shift our efforts to download files f
 - Eases path to ATO.
 
 **Risks**
+- None that the team is aware of at this time. 
 
 ## Notes
-- [Technical Documentation](../Technical-Documentation/data-file-downloads.md)
+**<details><summary>Data Files Download Architecture</summary>**
+
+This application provides a secure means to both store and download files from
+AWS S3 through the use of an open source Django plugin `django-storages`. By
+utilizing built in Django classes in conjunction with this plugin we can enable
+downloading of these files through an API endpoint without having to write
+the files to the local storage of the server, thus removing a performance
+penalty that would be incurred by essentially downloading the file twice.
+
+### Process Flow
+![](diagrams/tdp-data-file-download-api.png)
+
+### S3 File Storage
+`django-storages` provides a custom Storage Backend for Django that enables
+storing files in S3 instead of on the local Django server. This application
+has historically used this library for collection and storage of static files
+served for the Django admin. However, with this change we will move towards
+using this to interface with the Data Files as well.
+
+#### S3Boto3Storage
+This storage backend provides the support for opening files in read or write
+mode and supports streaming (buffering) data in chunks to S3 when writing.
+
+[Source code](https://github.com/jschneier/django-storages/blob/master/storages/backends/s3boto3.py#L233)
+
+#### S3Boto3StorageFile
+This class extends Django's File class to support file streaming using the
+[boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
+library's multipart uploading functionality. It provides a wrapper to access
+the buffered file contents.
+
+[Source code](https://github.com/jschneier/django-storages/blob/master/storages/backends/s3boto3.py#L79)
+
+#### InMemoryUploadedFile
+This file class is provided by Django and represents a file that has been
+uploaded into memory via streaming. The file returned by `django-storages` for
+a given FileField associated with an S3 object will leverage the functionality
+of this class to prevent needing to write the file to disk, resulting in a more
+performant download experience.
+
+[Source code](https://github.com/django/django/blob/main/django/core/files/uploadedfile.py#L78)
+
+### ReportFile model
+This is a custom model for the application that stores information about a
+Data File that has been uploaded to the system. To leverage the features
+mentioned above this model will have a [FileField](https://docs.djangoproject.com/en/3.2/ref/models/fields/#filefield)
+on the model which is linked to S3 via `django-storages`. From the perspective
+of the API, this will make downloading the file as simple as calling the `open`
+method on the file property of a `ReportFile` instance.
+</details>