Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add slurs to reserved words #1314

Conversation

jesslark
Copy link
Contributor

@jesslark jesslark commented Jul 13, 2023

Task:
Added slurs to the reserved words file.

Purpose:
Prevent users from using slurs as handles.

Challenges:
It is not a perfect solution but to prioritize the safety of marginalized users it is a good first step. This list will only prevent these exact words from being used.

Next Steps:

  • Needs additional work to add any other words that should be caught.
  • Needs regex added to catch common variations and attempts to circumvent.
  • Does not address the issue of using slurs in posts, only handles. A separate ticket and PR, potentially an entire project/epic, is required to censor slurs while allowing appropriate expression.

Recommendation:
Merge as-is and follow up with additional functionality.

jesslark added 2 commits July 12, 2023 19:42
Add slurs to reserved words
@jesslark jesslark mentioned this pull request Jul 13, 2023
@Bossett Bossett mentioned this pull request Jul 13, 2023
@Bossett
Copy link

Bossett commented Jul 13, 2023

@jesslark can you have a look at https://bsky.app/profile/cwonus.org/post/3k2er3cftha2c? I think worth removing a couple more and leaving as worst-of-the-worst?

@jesslark
Copy link
Contributor Author

@jesslark can you have a look at https://bsky.app/profile/cwonus.org/post/3k2er3cftha2c? I think worth removing a couple more and leaving as worst-of-the-worst?

Thank you for the link! I took a look at the list I have here and I think it's pretty much worst of the worst. Please point out if there are any I have left on that I shouldn't. There is another PR with a lot more words, this one is pretty short.

Copy link
Collaborator

@dholms dholms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together. Will merge & deploy once CI runs and follow up with more tightening

@dholms dholms changed the base branch from main to patch-add-slurs-to-reserved July 13, 2023 04:05
@dholms dholms merged commit 03a25c8 into bluesky-social:patch-add-slurs-to-reserved Jul 13, 2023
@jesslark
Copy link
Contributor Author

Thanks for putting this together. Will merge & deploy once CI runs and follow up with more tightening

Thanks so much!

dholms added a commit that referenced this pull request Jul 13, 2023
* add slurs to reserved words (#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>
@mariohoff
Copy link

How does the word "kraut" attack the safety of marginalized users? It doesn't even have a bad connotation imo

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

caught?

@mattfg
Copy link

mattfg commented Jul 13, 2023

I'm in favor of this as a put-out-the-fire measure but as we have already seen from the discussion around which words to include, the bigger problem is much less simple. Long-term, I'm not sure this logic belongs at the protocol level, and should instead be handled either by the client, or by the server instance (so, it could be a rule at english-speaking bsky.social that "retard" is banned but on a french-speaking server it's allowed). Once it's figured out how to implement it at the instance level, this code should probably be removed.

(ofc, I'm also not very familiar with this project structure and might be misunderstanding how this repo gets used and what it represents)

@Sominemo
Copy link

Sominemo commented Jul 14, 2023

Dictionary filtering is a very basic but also very flawed technique:

  • It has a lot of false positives: filtered words inside of legitimate words; words from other languages; junctions between two words, — and combinations of these. Even English has a word which contains the hard R inside itself. Trying to consider all the other languages and the junctions, and it becomes unmanageable.
  • It's easy to bypass: no matter how sophisticated your anti-bypass filters are, humans always find ways to put something offensive in all sorts of places like usernames

I expect it to hurt legitimate users more than preventing abuse.

This way of things is "good enough" for the current state of the project when moderation is not in priority and is unaffordable, but I think it shouldn't be cemented into the foundation of the software and expected to be improved.

In my opinion, it's better to consider this solution to be temporary instead of thinking how to improve it.

For the future in far perspective, I'd suggest relying on human moderation and heuristics instead. For example, a warning system for moderators when an account with a handle where a dictionary word is detected signs up or a few reports are sent about such handle.

dholms added a commit that referenced this pull request Jul 18, 2023
* add slurs to reserved words (#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>
dholms added a commit that referenced this pull request Jul 19, 2023
* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (#1318)

* add slurs to reserved words (#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (#1319)

* Refactor appview repo subscription for memleak (#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
dholms added a commit that referenced this pull request Jul 24, 2023
* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (#1318)

* add slurs to reserved words (#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (#1319)

* Refactor appview repo subscription for memleak (#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
dholms added a commit that referenced this pull request Jul 24, 2023
* refactor appview services

* tidy types

* Allow moderators to take and reverse actor takedowns (#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (#1355)

patch up a couple sqlite tests

* Increase appview keepalive (#1360)

increase keepalive to 90s

* Appview - serve feed skeletons (#1265)

* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (#1318)

* add slurs to reserved words (#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (#1319)

* Refactor appview repo subscription for memleak (#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>

* add optional ilike query to getPopularFeedGenerators

* fix lint

* fix lint

* handle sqlite too in pds only

* fix lint

* v0.4.3

* Include takendown posts for admins (feature branch) (#1361)

* 🚧 WIP including takendown posts on author feed

* ✨ Add takedown id on posts when including taken down posts

* 🧹 Cleanup the auth verifier and other bsky package code

* ✅ Add test for admin getAuthorFeed

* 🧹 Cleanup lexicon and exclude takedownId

* more explicit plumbing for post hydration w/o requester or with takedown info

* pass along flag for soft-deleted actors

* cleanup getAuthorFeed w/ auth

* reorg getAuthorFeed logic around role/access-based auth

---------

Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>

* ✨ Disable signing up with invite code from takendown account (#1350)

* fix viewer data

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Eric Bailey <git@esb.lol>
mloar pushed a commit to mloar/atproto that referenced this pull request Sep 26, 2023
* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>
mloar pushed a commit to mloar/atproto that referenced this pull request Sep 26, 2023
* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (bluesky-social#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (bluesky-social#1318)

* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (bluesky-social#1319)

* Refactor appview repo subscription for memleak (bluesky-social#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (bluesky-social#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
mloar pushed a commit to mloar/atproto that referenced this pull request Sep 26, 2023
* refactor appview services

* tidy types

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* Increase appview keepalive (bluesky-social#1360)

increase keepalive to 90s

* Appview - serve feed skeletons (bluesky-social#1265)

* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (bluesky-social#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (bluesky-social#1318)

* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (bluesky-social#1319)

* Refactor appview repo subscription for memleak (bluesky-social#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (bluesky-social#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>

* add optional ilike query to getPopularFeedGenerators

* fix lint

* fix lint

* handle sqlite too in pds only

* fix lint

* v0.4.3

* Include takendown posts for admins (feature branch) (bluesky-social#1361)

* 🚧 WIP including takendown posts on author feed

* ✨ Add takedown id on posts when including taken down posts

* 🧹 Cleanup the auth verifier and other bsky package code

* ✅ Add test for admin getAuthorFeed

* 🧹 Cleanup lexicon and exclude takedownId

* more explicit plumbing for post hydration w/o requester or with takedown info

* pass along flag for soft-deleted actors

* cleanup getAuthorFeed w/ auth

* reorg getAuthorFeed logic around role/access-based auth

---------

Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>

* ✨ Disable signing up with invite code from takendown account (bluesky-social#1350)

* fix viewer data

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Eric Bailey <git@esb.lol>
mloar pushed a commit to mloar/atproto that referenced this pull request Nov 15, 2023
* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>
mloar pushed a commit to mloar/atproto that referenced this pull request Nov 15, 2023
* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (bluesky-social#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (bluesky-social#1318)

* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (bluesky-social#1319)

* Refactor appview repo subscription for memleak (bluesky-social#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (bluesky-social#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
mloar pushed a commit to mloar/atproto that referenced this pull request Nov 15, 2023
* refactor appview services

* tidy types

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* Increase appview keepalive (bluesky-social#1360)

increase keepalive to 90s

* Appview - serve feed skeletons (bluesky-social#1265)

* proxy timeline skeleton construction to appview

* add getFeedSkeleton to appview

* mount route

* smart proxy feed skeletons

* tests

* proper error code

* only proxy specific feeds

* build branch

* update proxyable feed logic, should use feed publisher rather than generator

* fix feed proxy tests, configure feed publisher (in addition to generator)

* hotfix: prevent user-supplied rkey on posts with createRecord (bluesky-social#1313)

* prevent user-supplied rkey on posts with createRecord

* allow empty-string rkey parameter

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* add slurs to reserved words (bluesky-social#1318)

* add slurs to reserved words (bluesky-social#1314)

* Update reserved.ts

Add slurs to reserved words

* Update reserved.ts

fix typo

* Update reserved.ts to clean up the slur list

* linting

* pluralise

---------

Co-authored-by: jess <jesslarkdev+gh@gmail.com>

* identifier: tweaks and additions to slur list (bluesky-social#1319)

* Refactor appview repo subscription for memleak (bluesky-social#1308)

* refactor to remove closure in loop

* move consecutive item out of p-queue

* Handle validation improvements (bluesky-social#1336)

* Handle matches and false positives for unacceptable words in handles

* move handle validation logic to pds

* missed merge

* add cfg flag

* encode lists

* fix build issues

* move words to cfg

* tidy

---------

Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>

* Allow moderators to take and reverse actor takedowns (bluesky-social#1330)

allow moderators to take and reverse actor takedowns

* ✨ Allow searching reports by moderator did (bluesky-social#1283)

* ✨ Allow searching reports by moderator did

* ✅ Remove .only flags on tests

* ✅ Update snapshot

* ✅ Add checks for did match in actions

* v0.4.1

* Make sequencer leader behavior optional on pds (bluesky-social#1250)

* make sequencer leader behavior optional on pds

* tidy

* use 127.0.0.1 in with-test-db.sh for colima (bluesky-social#1297)

So, since Docker Desktop has licensing issues, some folks use colima for
running containers on their macOS machines (The licensing exempted
CLI-only version of Docker only exists on Linux).

Unfortunately, colima binds host ports only on the IPv4 localhost
address (`127.0.0.1`) while the atproto postgres clients will attempt to
connect to the IPv6 localhost address (`::1`) that macOS sets in
/etc/hosts.  See abiosoft/colima#583 and
lima-vm/lima#1330 for the tickets against
colima. (Docker Desktop binds to both IPv4 and IPv6 localhost addresses
and so doesn't have this issue.)

To workaround this silly issue, we can use `localhost` within the docker
containers and docker-compose, but need to set the `DB_POSTGRES_URL` env
var to use the IPv4 localhost explicitly.

(Asking folks to edit /etc/hosts causes other tools to break and will be
overridden on each OS upgrade.)

* Subscription util tests (bluesky-social#1295)

* consecutive list tests

* flesh out subscription util tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>

* Content reporting on record fields (bluesky-social#1351)

* content reporting on record fields

* fix test

* tests

* tidy

* Check rkey contents just for non-tids (bluesky-social#1353)

check rkey content for non-tids

* ✨ Added new procedure for sending admin email (bluesky-social#1312)

* 🚧 Added new lexicon for sending admin email

* ✨ Add moderation mailer

* ✨ Switch to text email content from html

* 🧹 Cleanup some early implementation code and reflect PR reivew

* ✨ Use smtp host instead of gmail service config

* ✨ Move to using single smtp url

* v0.4.2

* Patch up a couple sqlite tests (bluesky-social#1355)

patch up a couple sqlite tests

* enable feeds & build branch

* disable branch building & enable without proxy header

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>

* add optional ilike query to getPopularFeedGenerators

* fix lint

* fix lint

* handle sqlite too in pds only

* fix lint

* v0.4.3

* Include takendown posts for admins (feature branch) (bluesky-social#1361)

* 🚧 WIP including takendown posts on author feed

* ✨ Add takedown id on posts when including taken down posts

* 🧹 Cleanup the auth verifier and other bsky package code

* ✅ Add test for admin getAuthorFeed

* 🧹 Cleanup lexicon and exclude takedownId

* more explicit plumbing for post hydration w/o requester or with takedown info

* pass along flag for soft-deleted actors

* cleanup getAuthorFeed w/ auth

* reorg getAuthorFeed logic around role/access-based auth

---------

Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>

* ✨ Disable signing up with invite code from takendown account (bluesky-social#1350)

* fix viewer data

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Foysal Ahamed <cfaion341@gmail.com>
Co-authored-by: Jeff Hodges <jeff@somethingsimilar.com>
Co-authored-by: David Buchanan <d@vidbuchanan.co.uk>
Co-authored-by: jess <jesslarkdev+gh@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Jaz Volpert <ericvolp12@gmail.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants