2: remove discussions backend code, reddit, channels, moira, notifications #64

collinpreston · 2023-08-30T16:19:29Z

Pre-Flight checklist

Testing
- Changes have been manually tested

What are the relevant tickets?

#2

What's this PR do?

Removes all Reddit related functionality (praw, posts, channels, comments, notifications, etc).
Removes Moira related functionality.
Removes some remaining auth functionality that will be replaced by Keycloak.
Removes unused environment variables.
Removes Akismet which was used for mitigating spam comments in Reddit.

How should this be manually tested?

All tests should pass.
All MIT Open related functionality perform the same as the current.
Review modified to code and management commands.

Questions for reviewer

Should ckeditor remain or be removed?
Should uploading to S3 functionality remain or be removed?

gitguardian · 2023-08-30T16:24:29Z

⚠️ GitGuardian has uncovered 156 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	Secret	Commit	Filename
7518733	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_forbidden.json	View secret
7518854	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_delete_comment.json	View secret
7518741	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel_not_found.json	View secret
7518333	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[put].json	View secret
7518333	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[put].json	View secret
7518333	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[put].json	View secret
7946742	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment_404.json	View secret
7946742	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment_404.json	View secret
7946743	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels.json	View secret
7946743	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels.json	View secret
7946744	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel.json	View secret
7946744	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel.json	View secret
7946745	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_create_channel_already_exists.json	View secret
7946746	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment_404.json	View secret
7946746	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment_404.json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946747	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946748	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946748	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params1-extra_expected1-0].json	View secret
7946749	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_validate_image[banner].json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7518396	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_ordered.json	View secret
7946750	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_comment.json	View secret
7946750	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_comment.json	View secret
7946751	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params2-extra_expected2--1].json	View secret
7946751	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params2-extra_expected2--1].json	View secret
7518416	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_title_non_superuser_forbidden.json	View secret
7518416	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_title_non_superuser_forbidden.json	View secret
7518416	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_title_non_superuser_forbidden.json	View secret
7518416	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_title_non_superuser_forbidden.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946752	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_get_comment.json	View secret
7946753	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel_anonymous.json	View secret
7518425	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel.json	View secret
7518425	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel.json	View secret
7518183	Base64 Basic Authentication	`1012d83`	cassettes/channels.views.comments_test.test_delete_comment.json	View secret
7518692	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_nonstaff.json	View secret
7518443	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_create_channel_no_descriptions.json	View secret
7518443	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_create_channel_no_descriptions.json	View secret
7518443	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_create_channel_no_descriptions.json	View secret
7518786	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_not_found.json	View secret
7518193	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params0-extra_expected0-1].json	View secret
7518193	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params0-extra_expected0-1].json	View secret
7518193	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params0-extra_expected0-1].json	View secret
7518193	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params0-extra_expected0-1].json	View secret
7518193	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment[extra_params0-extra_expected0-1].json	View secret
7518454	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[patch].json	View secret
7518454	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[patch].json	View secret
7518454	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[patch].json	View secret
7518454	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_update_title_non_superuser_forbidden[patch].json	View secret
7518196	Base64 Basic Authentication	`1012d83`	cassettes/channels.views.channels_test.test_list_channels_anonymous.json	View secret
7518196	Base64 Basic Authentication	`1012d83`	cassettes/channels.views.channels_test.test_get_channel_anonymous.json	View secret
7946754	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_moderator.json	View secret
7946754	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_moderator.json	View secret
7946754	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_moderator.json	View secret
7946754	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_moderator.json	View secret
7946754	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_patch_channel_moderator.json	View secret
7946755	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel_with_avatar_banner.json	View secret
7946755	Bearer Token	`1012d83`	cassettes/channels.views.channels_test.test_get_channel_with_avatar_banner.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946756	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_reply_to_deleted_comment.json	View secret
7946757	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_moderator.json	View secret
7946757	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_moderator.json	View secret
7946757	Bearer Token	`1012d83`	cassettes/channels.views.comments_test.test_create_comment_moderator.json	View secret

and 76 others.

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

codecov · 2023-08-30T17:36:55Z

Codecov Report

Merging #64 (7be7b7f) into main (161295b) will decrease coverage by 1.70%.
Report is 7 commits behind head on main.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #64      +/-   ##
==========================================
- Coverage   86.25%   84.56%   -1.70%     
==========================================
  Files         399      316      -83     
  Lines       16251    11929    -4322     
  Branches     2552     1875     -677     
==========================================
- Hits        14018    10088    -3930     
+ Misses       1920     1583     -337     
+ Partials      313      258      -55

see 83 files with indirect coverage changes

mbertrand

Should ckeditor remain or be removed?

I'd defer to @ChristopherChudzicki on that, but I think there's a chance we might want it for field channels and/or their widgets?

Should uploading to S3 functionality remain or be removed?

Are you referring to MITOPEN_USE_S3 and DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"? I think we should keep that, for when FieldChannel images are uploaded, and maybe user profile images too (unless keycloak will be replacing that part).

mbertrand · 2023-09-06T17:41:21Z

fixtures/betamax.py

@@ -1,15 +1,33 @@
 """Betamax fixtures"""


I think you should be able to delete this entire file and remove references to these fixtures elsewhere (like users.py)

mbertrand · 2023-09-06T17:43:12Z

fixtures/betamax.py

 from betamax.fixtures.pytest import _casette_name

-from channels.test_utils import no_ssl_verification
 from open_discussions.betamax_config import setup_betamax


You can probably remove the the file open_discussions.betamax_config.py completely and any usages of its methods

mbertrand · 2023-09-06T18:05:12Z

embedly/views.py

@@ -6,8 +6,7 @@

 from django.conf import settings

-from channels.models import LinkMeta
-from embedly.api import get_embedly_summary, THUMBNAIL_URL


You can probably delete the constant THUMBNAIL_URL from embedly.api

mbertrand · 2023-09-06T18:12:40Z

search/management/commands/update_index.py

@@ -47,7 +47,7 @@ def add_arguments(self, parser):
        super().add_arguments(parser)

    def handle(self, *args, **options):
-        """Index the comments and posts for the channels the user is subscribed to"""
+        """Index Courses"""


"""Index all learning resources"""?

ChristopherChudzicki · 2023-09-06T19:01:14Z

Should ckeditor remain or be removed?

I'd defer to @ChristopherChudzicki on that, but I think there's a chance we might want it for field channels and/or their widgets?

I would keep it. The API endpoint gives us a key for CKEditor's image upload service, which we may well continue to use.

mbertrand

LGTM! 👍

collinpreston added 7 commits August 30, 2023 10:57

All tests passing

1012d83

Remove some more stuff, pylint everything

ec402db

Format

8812da0

pylint needs

212c237

More pylint stuff

397a493

Please be the last pylint thing

07de010

Fix search test

0deacbd

collinpreston linked an issue Aug 30, 2023 that may be closed by this pull request

Remove discussions backend code #2

Closed

collinpreston added 2 commits August 30, 2023 13:26

Remove akismet, TOS, any remaining comments

944e5d8

Remove more reddit posts stuff

437aae6

collinpreston added 4 commits August 30, 2023 13:39

Remove some more, lint fix

ad5bf23

Delete timeago.py

9652bd1

Remove subscription references

c82525c

Delete unused images

5bc41dc

ChristopherChudzicki mentioned this pull request Aug 30, 2023

Add pre-commit rules for all linting and replace Pylint with Ruff #62

Merged

collinpreston marked this pull request as ready for review August 30, 2023 21:32

collinpreston added the Needs Review An open Pull Request that is ready for review label Aug 30, 2023

Remove infinite corridor feature flag

bce0f31

mbertrand self-assigned this Sep 6, 2023

mbertrand requested changes Sep 6, 2023

View reviewed changes

mbertrand added Waiting on author and removed Needs Review An open Pull Request that is ready for review labels Sep 6, 2023

collinpreston added 5 commits September 7, 2023 07:17

Address code review comments

674f849

Remove betamax fixture

a46418c

Remove betamax calls

96c6898

Remove cassettes test

0356490

Add back db fixture to user helpers

697f1d0

Remove betamax

7be7b7f

collinpreston requested a review from mbertrand September 7, 2023 13:47

collinpreston added Needs Review An open Pull Request that is ready for review and removed Waiting on author labels Sep 7, 2023

mbertrand approved these changes Sep 7, 2023

View reviewed changes

mbertrand added Waiting on author and removed Needs Review An open Pull Request that is ready for review labels Sep 7, 2023

collinpreston merged commit 885cdd7 into main Sep 7, 2023

collinpreston deleted the 2-remove-discussions-backend-code-2 branch September 7, 2023 14:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

2: remove discussions backend code, reddit, channels, moira, notifications #64

2: remove discussions backend code, reddit, channels, moira, notifications #64

Uh oh!

collinpreston commented Aug 30, 2023 •

edited

Loading

Uh oh!

gitguardian bot commented Aug 30, 2023 •

edited

Loading

Uh oh!

codecov bot commented Aug 30, 2023 •

edited

Loading

Uh oh!

mbertrand left a comment

Uh oh!

mbertrand Sep 6, 2023

Uh oh!

mbertrand Sep 6, 2023

Uh oh!

mbertrand Sep 6, 2023

Uh oh!

mbertrand Sep 6, 2023

Uh oh!

ChristopherChudzicki commented Sep 6, 2023

Uh oh!

mbertrand left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

2: remove discussions backend code, reddit, channels, moira, notifications #64

2: remove discussions backend code, reddit, channels, moira, notifications #64

Uh oh!

Conversation

collinpreston commented Aug 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Flight checklist

What are the relevant tickets?

What's this PR do?

How should this be manually tested?

Questions for reviewer

Uh oh!

gitguardian bot commented Aug 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ GitGuardian has uncovered 156 secrets following the scan of your pull request.

Uh oh!

codecov bot commented Aug 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mbertrand left a comment

Choose a reason for hiding this comment

Uh oh!

mbertrand Sep 6, 2023

Choose a reason for hiding this comment

Uh oh!

mbertrand Sep 6, 2023

Choose a reason for hiding this comment

Uh oh!

mbertrand Sep 6, 2023

Choose a reason for hiding this comment

Uh oh!

mbertrand Sep 6, 2023

Choose a reason for hiding this comment

Uh oh!

ChristopherChudzicki commented Sep 6, 2023

Uh oh!

mbertrand left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

collinpreston commented Aug 30, 2023 •

edited

Loading

gitguardian bot commented Aug 30, 2023 •

edited

Loading

codecov bot commented Aug 30, 2023 •

edited

Loading