Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Instance only statuses #8427

Open
wants to merge 220 commits into
base: main
from

Conversation

@renatolond
Copy link
Collaborator

@renatolond renatolond commented Aug 25, 2018

This is a WIP PR for instance-only statuses. This is largely based on the glitch-soc implementation of instance-only to make it easier to incorporate back into glitch-soc.

This assumes that clients are still unaware of local-only statuses and to avoid leaking replies outside, if the local_only flag is missing, replies to local_only posts are posted as local_only.

As in the glitch-soc implementation, local_only statuses can only be seen by users that are logged in, logged out users do not see the local_only statuses in hashtag timelines or in the "take a look inside" one. local_only cannot be fetched by the other instances, even with the link. The default option is to federate the toot

Fixes #861, #896, #1761, #2698, #3285

What is missing:

  • Refactor "local_only" into something else to avoid confusion with "local", maybe "instance_only"
  • Change icons into something more meaningful
  • Add customization in preferences to which federation status is preferred.
  • Add "local_only" indication in moderations screens
  • Fix react privacydropdown warning, probably something is duplicated since the federationdropdown component is copied over from the privacy one.
  • Run i18n, add pt-br translations

image
image
image
image
image
image

@renatolond renatolond force-pushed the masto-donte-com-br:instance_only_statuses branch 2 times, most recently Aug 25, 2018
@Gargron
Copy link
Member

@Gargron Gargron commented Aug 25, 2018

I think it introduces very real centralization dynamics into the software. Our selling point is "pick whatever server you like, you can talk to your friends from anywhere". With this it becomes "unless you want ALL of their content, in which case you better sign up on the same server as them". So imagine mastodon.social gets this feature, and suddenly signing up somewhere else is not an option because you'd be missing out!

Mastodon works on a simple principle: Your posts go to your followers. Why is that not sufficient? You could block all followers from other servers you might have, and achieve the same result.

I know that there are some benefits to this feature, such as letting admins announce things for their own users only (you could add UI for mass e-mail announcements instead?) but I think the long-term effects of this feature could have catastrophic consequences.

Please explain why you think I'm wrong about this.

@Cassolotl
Copy link

@Cassolotl Cassolotl commented Aug 25, 2018

With this it becomes "unless you want ALL of their content, in which case you better sign up on the same server as them".

I guess the difference then is I'm cynical in that I think most people will want to blast their content to as many followers as possible and I think people will be lazy and ignore it if it's default-off, and I'm also trusting in that I think most people will post local-only only when they're posting content that is truly not relevant to other servers - like, "wow our instance is slow today, huh?" or "I think [new feature] on glitch.social could be cool, what do you think?"

I've also seen people setting up instances for particular towns, cities, universities and colleges. So if someone was a member of one of those, they might want to post "hey, bake sale in the student union foyer at 2pm today!" or "does anyone know where I can get a wide-screen TV near Main Street?" or whatever, without bothering people who would not be anywhere near those places.

I don't think Mastodon is diminished by allowing people to be insular. I feel like if you're going to promote Mastodon as being many small, safe communities focused on particular interests or geographical locations then it absolutely makes sense to allow people to post to only their own instance. I think this is a good idea, and I like having it on dev.glitch.social even though I have only used it a couple of times.

But Mastodon is already pretty insular compared to similar centralised services so I can see why you're concerned and not wanting it to be more insular.

@trwnh
Copy link
Contributor

@trwnh trwnh commented Aug 25, 2018

Would it be possible to instead treat this as a targeting issue than a federation issue? If the aim is to have a certain post visible to users of an instance, then could there be a Collection of all the users on the instance that could be cc'ed, in the same way that the #public is cc'd on public posts? It sounds like it would be better long-term to support this as a specialized case of arbitrary audiences, rather than make an entirely new feature for basically the same concept.

@nightpool
Copy link
Member

@nightpool nightpool commented Aug 25, 2018

my main concern is that this adds yet another layer of complexity when explaining the privacy settings to users. people already make serious mistakes because they don't understand the existing privacy levels (like set their account to locked and then posting publically because they use a third party client that doesn't respect default privacy settings).

adding yet another checkbox doubles the level of complexity.

I think the usecase here would be better served by having an "announcements" feature for admins that gets stickied to the top of local user's feeds and doesn't federate. this would be analogous to "pinned posts" on Discourse or other forum stickies.

@Cassolotl
Copy link

@Cassolotl Cassolotl commented Aug 25, 2018

I think the usecase here would be better served by having an "announcements" feature for admins that gets stickied to the top of local user's feeds and doesn't federate. this would be analogous to "pinned posts" on Discourse or other forum stickies.

I do agree that this is a good idea, but it wouldn't help the non-admins who want an instance-only visibility setting for their posts, I think it serves a different purpose.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 26, 2018

Mastodon works on a simple principle: Your posts go to your followers. Why is that not sufficient? You could block all followers from other servers you might have, and achieve the same result.

Not the same result, since it could happen that I share a public post on my instance and it gets boosted out of it by other users.

The thing for me is small instances work differently than big ones. Not only for instance-only banter, as the ones that @Cassolotl suggested, but sometimes people want to be on smaller instances and trust the admin and other users of that instance, but maybe not the wider fediverse.

I don't think it breaks federation, even if a user only posted locally and allowed for other users to follow them from the outside, it would look like they were never posting anything as from the outside it would look like a blank profile.

Also, the feature exists in glitch-soc, there's some considerably big instances running glitch-soc and we still interact with users on those instances, if they use local-only posts I don't notice any holes in their posting.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 26, 2018

adding yet another checkbox doubles the level of complexity.

That's why the help needed tag on this one. Frontend is not my thing and I added in a way that I think it's consistent with the rest of the UI while still explaining the option (a button like the CW option I think it's not ideal because it does not explain the option) but maybe there's a better way to integrate this, maybe it can be more hidden while still being somewhere in the compose box.

@Cassolotl
Copy link

@Cassolotl Cassolotl commented Aug 26, 2018

I liked the buttons you used, Lond, because they feel in-keeping with Gargron's intentions! A linked chain to federate, a broken chain to break the link from the rest of the fediverse - so the broken chain of not federating has negative connotations and is a little discouraging. It's like, only do this if you understand what it means and you're sure that's what you want.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 26, 2018

Would it be possible to instead treat this as a targeting issue than a federation issue?

I think it could be the case, but I don't think adding the cc'ed features for groups with meta-groups like local is such an easy change, and if we integrate this into mastodon and later change this into groups, I believe this could be changed in a migration to work with such a feature.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 26, 2018

A linked chain to federate, a broken chain to break the link from the rest of the fediverse - so the broken chain of not federating has negative connotations and is a little discouraging.

I think the link/link broken is good, but I would like even more something like this one: https://www.iconfinder.com/icons/175933/connections_graph_network_relations_structure_icon
And one with the relations broken for the local only, I think it would convey better and avoid associations with links, but there's nothing like that in fontawesome, that's why I went for the link ones.

@renatolond renatolond force-pushed the masto-donte-com-br:instance_only_statuses branch to 694640e Aug 26, 2018
@Cassolotl
Copy link

@Cassolotl Cassolotl commented Aug 26, 2018

I think the link/link broken is good, but I would like even more something like this one: https://www.iconfinder.com/icons/175933/connections_graph_network_relations_structure_icon

Ohh that's a great one. Maybe in time fontawesome would have something like that. :)

@ggtea
Copy link

@ggtea ggtea commented Aug 28, 2018

Is post-to-local bad because only instance's user can see them and other instance's users can't see them?
I feel LTL have same problem and is already there.
It seems that root problem is other instance's user can't join to instance's LOCAL.
I wish I could join other instance's LOCAL from mine and remote follow and post to other instance's LTL.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 28, 2018

Local timelines are not private, they have all the public posts by local users and there are tools to browse them from outside the instance (I think at least one mobile app also implemented a similar function recently)

Local-only/instance-only posts would only be viewable, according to this implementation and glitch-soc one, to logged in users.

@PaperFixie
Copy link

@PaperFixie PaperFixie commented Aug 29, 2018

I am a member of multiple instances, one of which is purely for communication with my local community geographically. I think it would be beneficial to have local only posts for those of us who use another instance for socializing across the fediverse, but use a different one for talking to our literal neighbors.

@melissamcewen
Copy link

@melissamcewen melissamcewen commented Aug 29, 2018

I think this is beneficial and would improve my experience with Mastodon immensely. Also recently learned about a hostile instance where the admin may be using the DB to look at followers-only tweets (where the toot in question is on another instance, followers only, and the tooter follows/is followed by someone on the hostile instance. Instance only would be a barrier to such behavior. Especially for meta toots that have to deal with instance moderation this would be really nice.

@apiontek
Copy link

@apiontek apiontek commented Aug 29, 2018

I am in favor of this feature.

imagine mastodon.social gets this feature, and suddenly signing up somewhere else is not an option because you'd be missing out!

Why would someone make effort to post local-only content that I, a user at some other instance, need to see?

Imagine users on mastodon.social sending DMs to other users -- am I missing out? Imagine users I want to follow who have privacy settings requiring approval. Imagine posts that are locked.

Your posts go to your followers.

If I have a private account and approve follow requests, that means everyone who wants to follow me doesn't get to see all my content.

That's a restriction based on personal privacy consideration, which is good and important, but it doesn't serve the same purpose as instance-only toots would, which I feel is also a good and important option to have.

If for some reason I do an instance-only toot that seems so great it should go beyond my instance, I can always re-toot it, or someone on the local can always copy/paste it.

Much like private toots that one might later decide to make public, no one ever gets things perfect on the first go. It'd be nice to have this option available for use.

@a-dows
Copy link

@a-dows a-dows commented Aug 29, 2018

I run a geographically-oriented instance, and this would provide massive utility to us.

"Instance-only + followers" toots would help me make administrative announcements, and give peace-of-mind to users so they can feel comfortable posting classifieds or other personally-identifiable content, and in general facilitate the growth of a class of instance that is popping up all around the world.

I understand that it adds complexity, but if the feature is opt-in on an administrative level, it would improve my users' Mastodon experience tremendously.

@FlamingOntheNet
Copy link

@FlamingOntheNet FlamingOntheNet commented Aug 29, 2018

I'm a member of a regionally-focused instance, and there are many in the instance that would like a local_only feature. Personally, some posts are either more appropriate to the instance because they are very regionally-specific (meet-up, organizing, and local advocacy toots) or are related to trust issues from lessons learned from birdsite. I like to keep most of my toots in the federated timeline and my followers, but there are times when I simply want to avoid it but want it to be readable by the entire instance.

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Aug 29, 2018

"Instance-only + followers"

In this case you mean that the post doesn't leave your instance, but only goes to all the followers on your instance?

@a-dows
Copy link

@a-dows a-dows commented Aug 29, 2018

In this case you mean that the post doesn't leave your instance, but only goes to all the followers on your instance?

Yes, 100%.

@a-dows
Copy link

@a-dows a-dows commented Aug 29, 2018

If this feature is implemented, my followers who are not on my instance won't be bothered by locals-only content that doesn't apply to them—my general-interest toots will reach them, while my administrative and locals-only content won't. This is a better experience for both poster AND follower.

With this it becomes "unless you want ALL of their content, in which case you better sign up on the same server as them".

My followers have permission to see my toots; they do not have the right to see absolutely everything I post. With local-only posting as an option, my federated followers will still get 100% of my general-interest content.

@seanmpuckett
Copy link

@seanmpuckett seanmpuckett commented Aug 29, 2018

I am in favour of local-only statuses. It does not remove functionality from the core idea of Mastodon. It does add functionality that allows a more diverse set of use-cases. Including, what I believe is essential, allowing a user to not federate a message if they don't want to, thereby controlling its distribution. This can be critical for privacy and safety purposes. Please implement this function.

@giromide
Copy link

@giromide giromide commented Aug 29, 2018

The privacy levels on Mastodon are already a bit tricky, and I don't think this makes them any harder to understand. I see more benefit in the comfort of knowing one's posts won't leave their instance at all than detriment of more privacy confusion. Plainly explaining the feature is enough. Geographic instance users will love this.

@t54r4n1
Copy link

@t54r4n1 t54r4n1 commented Aug 29, 2018

Instance only meetups are fun and I want to plan more of them, as a mod of a geographic instance please let me have a tool to do this!

@rtxanson
Copy link

@rtxanson rtxanson commented Aug 29, 2018

I'm not on a region-specific instance, but follow a ton of people on a region-specific instance because I live in that region. I'd be in favor of "instance only + followers" if "followers" could also mean followers on other instances. Alternatively, would there be a way for me to filter a timeline to only see instance-specific tweets, meaning I could potentially make an account for the sole purpose of making sure I'm not missing important stuff?

I can see how there's a difference between a need for admins and mods of instances to communicate information that is only relevant to people on the instance (server updates, downtime, etc.), I see the need to address privacy concerns people laid out here, but what to do about regionally relevant information for people who aren't on a regional instance? I guess it might be up to posters to determine if their target audience also includes a selection of people who might need to see that content too.

@lawremipsum
Copy link

@lawremipsum lawremipsum commented Aug 29, 2018

I admin mspsocial.net (170 registered accounts). Here's my case for local-only status privacy:

  • It's intuitive. We have a local timeline, but currently no way to post to it without also posting to the federated timeline. My users ask how to do this frequently, and I have to explain they can't.
  • It gives users choice. More granular privacy controls are better, as a general rule, and add value for users who choose to take advantage of them.

The lack of local-only compels people to share posts more broadly than they would like to serve a network purpose ("feed more content to the federated timeline"). However,

  • Local-only will make the federated timeline better and more useful in most circumstances. The federated timeline on large instances is almost useless because it has too much content. Do users on other instances really need/want to wade through posts about parking in Poughkeepsie, or the level of bus service in Boston, or the weather in Walla-Walla? Why shouldn't the author of those statuses have the discretion to spare the world things that are not of general interest?

Mastodon has largely grown out of the need to programmatically generate content for the federated timeline. Some of it could be spared everyone's eyeballs and nobody will feel the loss. And I have no doubt that a lot of stuff that could be local only will still be posted under the default world-readable setting.

  • Adding the option puts the choice of whether to publish that information more widely in the hands of the author, not the code. Nobody would be required to post local-only. It would simply be an option so a user could avoid cluttering feeds with information unlikely to be of interest.

I'm neutral about whether it should be implemented as local-only or local-and-friends-only, but I suspect one or the other has deeper implications for the way federation works, so if that's the case the simpler implementation would be better than none at all.

mashirozx referenced this pull request in mashirozx/mastodon Sep 21, 2020
* Use backend from glitch-soc for instance-only toots

* Base frontend on privacy dropdown

* Add backend support for local_only on status create
Based off ThibG implementation on glitch tootsuite#502

* Add local_only indicator in the status action bar

* Add local_only indicator to detailed status

* Normalize translations
ran yarn build:development && i18n-tasks normalize && yarn manage:translations && i18n-tasks remove-unused

* Add local_only to admin screens

* Hide local statuses from user atom and from unlogged users

* Add local only icon to status page

* Fix issue with toggle in mobile

* Add default federation option to settings

* Fix wrong configuration fetched

* Change setting wording

* Check for empty "last_status" before sorting DM column (tootsuite#9207)

* Check for empty "last_status" before sorting

* Small touchups for codeclimate

* Update resolve_url_service.rb (tootsuite#9188)

* Increase default column width from 330px to 350px (tootsuite#9227)

* Optimize the process of following someone (tootsuite#9220)

* Eliminate extra accounts select query from FollowService

* Optimistically update follow state in web UI and hide loading bar

Fix tootsuite#6205

* Asynchronize NotifyService in FollowService

And fix failing test

* Skip Webfinger resolve routine when called from FollowService if possible

If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService

* Fix follow limit validator reporting lower number past threshold (tootsuite#9230)

* Fix follow limit validator reporting lower number past threshold

* Avoid floating point follow limit

* Fix form validation flash message color and input borders (tootsuite#9235)

* Fix form validation flash message color and input borders

* Fix typo

* Display amount of freed disk space in tootctl media remove (tootsuite#9229)

* Display amount of freed disk space in tootctl media remove

Fix tootsuite#9213

* Fix code style issue

* Add "Show thread" link to self-replies (tootsuite#9228)

Fix tootsuite#4716

* Fix nil error regression from tootsuite#9229 in tootctl media remove (tootsuite#9239)

Fix tootsuite#9237

* Improve ActiveRecord connection in on_worker_boot (tootsuite#9238)

This is how it looks in the example in the Puma README

* Check that twitter:player is valid before using it (tootsuite#9254)

Fixes tootsuite#9251

* Fix emoji update date processing (tootsuite#9255)

* Perform deep comparison for card data when receiving new props (tootsuite#9270)

Fixes tootsuite#9226

* Fix null error introduced in tootsuite#9270 (tootsuite#9275)

* Fix race condition causing shallow status with only a "favourited" attribute (tootsuite#9272)

Fixes tootsuite#9231

* Fix "tootctl media remove" can't count the file size (tootsuite#9288)

* Fixed an issue where "tootctl media remove" can not count the file size.

* Fixed the problem pointed out by codeclimate.

* Remove intermediary arrays when creating hash maps from results (tootsuite#9291)

* Prevent multiple handlers for Delete of Actor from running (tootsuite#9292)

* Fix filter ID not being a string in REST API (tootsuite#9303)

* Update Nginx config for Nanobox apps (tootsuite#9310)

The Nanobox files have gotten out of sync, a touch, with what Masto needs for Nginx settings. This PR updates them accordingly.

* WebSub: ATOM before RSS (tootsuite#9302)

Hello,
The ATOM feed contains the hub declaration for WebSub, but the RSS
version does not.
RSS/ATOM readers will typically pick whichever version comes first, and
will thus not see the WebSub feature.
I therefore suggest putting the ATOM version first, as it is more
feature-rich than its RSS counterpart is.

Clients not compatible with ATOM would not pick it anyway due to the
different type attribute.

A more complicated alternative would be to declare the WebSub feature in
the RSS version as well, using something like the following code, and
ensuring that clients subscribed to the RSS version would receive PuSH
updates just like those subscribed to the ATOM version.

````xml
<rss version="2.0" xmlns:webfeeds="http://webfeeds.org/rss/1.0"
xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<atom:link rel="self" type="application/rss+xml"
href="https://diaspodon.fr/users/test.rss"/>
<atom:link rel="hub" href="https://diaspodon.fr/api/push"/>
</channel>
</rss>
```

* Touch account on successful response, change char shown when culled (tootsuite#9293)

Just the color is not enough change since not everyone uses colored
terminals.
Touching the account makes it so that the account is not in the
threshold window in case of running again

* Ignore JSON-LD profile in mime type comparison (tootsuite#9179)

Ignore JSON-LD profile in mime type comparison

* Fix connect timeout not being enforced (tootsuite#9329)

* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec

* Bump version to 2.6.2

* add loglevel to ffmpeg in gif upload (tootsuite#9368)

* Allow hyphens in the middle of remote user names (tootsuite#9345)

Fixes tootsuite#9309

This only allows hyphens in the middle of a username, much like dots,
although I don't have a compelling reason to do so other than keeping
the changes minimal.

* Fix nil error when no DNS addresses are found for host (tootsuite#9379)

* Don't count suspended users in user count (tootsuite#9380)

Fix tootsuite#7637

* Skip deliveries to inboxes that have already been marked as unavailable (tootsuite#9358)

* Fix TLS handshake timeout not being enforced (tootsuite#9381)

Follow-up to tootsuite#9329

* Bump version to 2.6.3

* Remove npm-run-all dependency (tootsuite#9401)

Fix tootsuite#9359

* Bump version to 2.6.4

* Preload common JSON-LD contexts (tootsuite#9412)

Fixes tootsuite#9411

* Include replies to list owner and replies to list members in list statuses (tootsuite#9324)

* Bump version to 2.6.5

* Only stream local-only toots to logged-in users

* Normalize translations

* Bumps copyright year in README.md to 2019 (tootsuite#9939)

This is so incredibly small, but assuming this is a needed change. Might want to check year in other files.

* Fix link color in high-contrast theme, add underlines (tootsuite#9949)

Improve sorting of default themes in the dropdown

* Replace unlock-alt icon with unlock (tootsuite#9952)

* Allow most kinds of characters in URL query (fixes tootsuite#8408) (tootsuite#8447)

* Allow unicode characters in URL query strings

Fixes tootsuite#8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.

* Fix authorized applications list page design (tootsuite#9969)

* Fix not showing custom emojis in share page emoji picker (tootsuite#9970)

* [UI] Fix whitespace being applied to div instead of p (tootsuite#9968)

* fix large line breaks

* fix ascii art posts

* Hide misleading “You will be sent a confirmation e-mail” hint from admin view (tootsuite#9973)

Thanks @wryk for noticing this issue.

* Fix Tombstone.delete_all ArgumentError (tootsuite#9978)

* Only URLs extract with pre-escaped text (tootsuite#9991)

* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( tootsuite#9989 )

* Fix URL linkifier grabbing full-width spaces and quotations (tootsuite#9997)

Fix tootsuite#9993
Fix tootsuite#5654

* Fix IntersectionObserverArticle not hiding some out-of-view items (tootsuite#9982)

IntersectionObserverArticle is made to save on RAM by avoiding fully rendering
items that are far out of view. However, it did not work for items spawned
outside the intersection observer.

* Fix timeline jumps (tootsuite#10001)

* Avoid two-step rendering of statuses as much as possible

Cache width shared by Video player, MediaGallery and Cards at the
ScrollableList level, pass it down through StatusList and Notifications.

* Adjust scroll when new preview cards appear

* Adjust scroll when statuses above the current scroll position are deleted

* Don't focus spiler input when disabled spoiler (tootsuite#10017)

* Move sending account Delete to anyone but the account's followers to the pull̀ queue (tootsuite#10016)

* Add support for IPv6 only MXes in Email validation (tootsuite#10009)

* Add support for IPv6 only MXes

* Fixed email validator tests

* Save IP address used for sign-up, not only sign-in (tootsuite#10026)

Fixes tootsuite#9995

* Fix color of static page links in high contrast theme (tootsuite#10028)

* Fix hashtags select styling in default and high contrast themes (tootsuite#10029)

* Fix style regressions on landing page (tootsuite#10030)

* Add tight rate-limit for API deletions (tootsuite#10042)

Deletions take a lot of resources to execute and cause a lot of
federation traffic, so it makes sense to decrease the number
someone can queue up through the API.

30 per 30 minutes

* Fix hashtag column not subscribing to stream on mount (tootsuite#10040)

Fix tootsuite#9895

* Create Redisable#redis (tootsuite#9633)

* Create Redisable

* Use #redis instead of Redis.current

* Alternative handling of private self-boosts (tootsuite#9998)

* When self-boosting, embed original toot into Announce serialization

* Process unknown self-boosts from Announce object if it is more than an URI

* Add some self-boost specs

* Only serialize private toots in self-Announces

* Filter incoming Create activities by relation to local activity (tootsuite#10005)

Reject those from accounts with no local followers, from relays
that are not enabled, which do not address local accounts and are
not replies to accounts that do have local followers

* Filter incoming Announce activities by relation to local activity (tootsuite#10041)

* Filter incoming Announce activities by relation to local activity

Reject if announcer is not followed by local accounts, and is not
from an enabled relay, and the object is not a local status

Follow-up to tootsuite#10005

* Fix tests

* Add logging for rejected ActivityPub payloads and add tests (tootsuite#10062)

* Fix Announce activities of unknown statuses not fetching those statuses (tootsuite#10065)

Regression from tootsuite#9998

* Fix relay enabling/disabling not resetting inbox availability status (tootsuite#10048)

Fix tootsuite#10033

* Change robots.txt to exclude some URLs (tootsuite#10037)

- Exclude static assets
- Exclude uploaded files
- Exclude alternate versions of the profile page
- Exclude media proxy URLs

* Change robots.txt to exclude only media proxy URLs (tootsuite#10038)

* Revert "Change robots.txt to exclude some URLs (tootsuite#10037)"

This reverts commit 80161f4.

* Let's block media_proxy

/media_proxy/ is a dynamic route used for requesting uncached media, so it's
probably bad to let crawlers use it

* misleading comment

* Improve image description user experience (tootsuite#10036)

* Add image descriptions to searchable post content.

* Allow multi-line image descriptions.

* Request image descriptions in the same query as posts when creating the search index.

(see tootsuite#10036 (comment))

* perf: run node directly when streaming (tootsuite#10032)

* Fix breaks when opening a reply tree in WebUI (tootsuite#10046)

fix tootsuite#10045

* Change conversations to always show names of other participants (tootsuite#10047)

Fix tootsuite#9190

* Change buttons on timeline preview to open the interaction dialog (tootsuite#10054)

Fix tootsuite#9922

* Change error graphic to hover-to-play (tootsuite#10055)

Fix tootsuite#6060

* Add registrations attribute to instance entity in REST API (tootsuite#10060)

Fix tootsuite#9350

* Add vapid_key to the application entity in the REST API (tootsuite#10058)

Fix tootsuite#8785

* Fix mutes, blocks, domain blocks and follow requests not paginating (tootsuite#10057)

Regression from tootsuite#9581

* Fix crash on public hashtag pages when streaming fails (tootsuite#10061)

* Bump version to 2.7.2

* Do not leak local-only toots to remote mentioned users

* Add description on hover in media gallery (tootsuite#10713)

* Fix some colors of high contrast theme (tootsuite#10711)

* Fix "nothing here" text color of high contrast

* Fix counter border color of high contrast

* Bring back crossed eye icon on gallery (tootsuite#10715)

* Improve poll link accessibility (tootsuite#10720)

* Add distinction between hover and active/focus states
* Resolves tootsuite#10198

* Change icon and label depending on whether media is marked as sensitive (tootsuite#10748)

* Change icon and label depending on whether media is marked as sensitive

* WiP use a checkbox

* Fix some colors in light theme (tootsuite#10754)

* Fix typo in light theme

* Fix background color of empty column

* Adds click-able div that expands status (tootsuite#10733) (tootsuite#10766)

The clickable div is positioned under the account avatar and covers
all empty space below it to the end of the status.

* Minor performance improvements and cleanup in formatter (tootsuite#10765)

* Prevent from publicly boosting one's own private toots (tootsuite#10775)

* add og:image:alt for media attachments in embeds (tootsuite#10779)

* Add post-deployment migration script to delete public-boosts-of-private-toots (tootsuite#10783)

* fix `isSubmitting` prop case (tootsuite#10785)

* Fix “invited by” not showing up for invited accounts in admin interface (tootsuite#10791)

* Bump version to 2.8.3

* Retry ActivityPub inbox delivery on HTTP 401 and 408 errors (tootsuite#10812)

HTTP 401 responses returned by Mastodon's inbox controller may
be temporary if, for instance, the requesting user's actor/key json
could not be retrieved in a timely fashion. This changes allow retries
instead of dropping the message entirely.

Also added HTTP 408 as that error is by nature temporary.

* Move signature verification stoplight to the requests themselves (tootsuite#10813)

* Move signature verification stoplight to the requests themselves

This avoids blocking messages from known keys for 5 minutes when only one fails…

* Put the stoplight on the actual client IP, not a potential reverse proxy

* Fix possible race condition when processing statuses (tootsuite#10815)

* Improve streaming server security (tootsuite#10818)

* Check OAuth token scopes in the streaming API

* Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token

Inspired by kubevirt/kubevirt#1242

* Bump version to 2.8.4

* Fix merge issues

* Fix account URI in UpdatePollSerializer (tootsuite#11194)

* Fix account URI in UpdatePollSerializer

Fixes tootsuite#11185

* Add specs

* Fix swiping columns on mobile sometimes failing (tootsuite#11200)

Fixes tootsuite#9779

* Fix option to send e-mail notification about account action always being true (tootsuite#11242)

* Fix BackupService crashing when an attachment is missing (tootsuite#11241)

* Fix BackupService crashing when an attachment is missing

For various reasons such as admin error or out-of-sync media and
database backups, it might be possible for local attachments to be lost.

This commit allows the BackupService to continue its work even if some media
file is missing.

* Change error message

* Fix Status.remote scope matching *all* statuses (tootsuite#11265)

* Fix BlockService trying to reject incorrect follow request (tootsuite#11288)

Fixes tootsuite#11148

* Fix invites not being disabled upon account suspension (tootsuite#11412)

* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account

* Update fuubar dependency to 2.4.1 (tootsuite#11248)

See also: thekompanee/fuubar#111

* Fix support for MP4 files that are actually M4V files (tootsuite#11210)

Resolve tootsuite#11187

* Fix expiration date of filters being set to “Never” when editing them (tootsuite#11204)

When editing a custom filter, select the shortest preset duration that
still covers the remaining time of that filter.

Fixes tootsuite#9506

* Fix statsd UDP sockets not being cleaned up in Sidekiq (tootsuite#11230)

* Remove unused StatsD code and expose StatsD as a global variable (tootsuite#11232)

The instrumentation code was used for StatsD metrics collection
prior to the switch to the nsa gem and should have been removed
at that point as it no longer does anything at all

* Fix some flash notices/alerts staying on unrelated pages (tootsuite#11364)

* Fix `alerts` booleans not being typecast correctly in push subscription (tootsuite#11343)

* Fix `alerts` booleans not being typecast correctly in push subscription

Fix tootsuite#10789

* Fix typo

* Optimize makeGetStatus (tootsuite#11211)

* Optimize makeGetStatus

Because `ImmutableList.filter` always returns a new object and `createSelector`
memoizes based on object identity, the selector returned by `makeGetStatus`
would *always* execute.

To avoid that, we wrap `getFilters` into a new memoizer that memoizes based on
deep equality, thus returning the same object as long as the filters haven't
changed, allowing the memoization of `makeGetStatus` to work.

Furthermore, we memoize the compiled regexs instead of recomputing them each
time the selector is called.

* Fix memoized result being cleared too often

* Make notifications use memoized getFiltersRegex

* Memoize ancestorIds and descendantIds in detailed status view (tootsuite#11234)

* Fix boosting & unboosting preventing a boost from appearing in the TL (tootsuite#11405)

* Fix boosting & unboosting preventing a boost from appearing in the TL

* Add tests

* Avoids side effects when aggregate_reblogs isn't true

* Fix delete regression (tootsuite#11450)

Regression from ff789a7

* Apply filters to poll options (tootsuite#11174)

* Apply filters to poll options in WebUI

Fixes tootsuite#11128

* Apply filters to poll options server-side

* Add poll options to searchable text

* Fix unnecessary SQL query performed on unauthenticated requests (tootsuite#11179)

* Add message telling FTS is disabled when no toot can be found because of this (tootsuite#11112)

* Add message telling FTS is disabled when no toot can be found because of this

Fixes tootsuite#11082

* Remove info icon and reword message

* Display FTS warning based on actual search term, not the one being typed (tootsuite#11202)

Follow-up to tootsuite#11112

* Scroll to compose form rather than reply indicator on focus (tootsuite#11182)

* When sending a toot, ensure a CW is only set if the CW field is visible (tootsuite#11206)

In some occasions, such as the browser or a browser extension auto-filling
the existing but disabled/hidden CW field, a CW can be set without the user
knowing.

* When deleting & redrafting a poll, fill in closest expires_in (tootsuite#11203)

Use the smallest preset expires_in such that the new poll would
not expire before the old one.

In the typical case of a quick delete & redraft, this results in
using the same poll duration.

Fixes tootsuite#10567

* Only scroll to the compose form if it's not horizontally in the viewport (tootsuite#11246)

Avoids jumping the scroll around vertically when giving it focus and
editing long toots.

* Display custom emoji in bio field names (tootsuite#11350)

Already displayed in public pages, but not WebUI

* Play animated custom emoji on hover (tootsuite#11348)

* Play animated custom emoji on hover in status

* Play animated custom emoji on hover in display names

* Play animated custom emoji on hover in bios/bio fields

* Add support for animation on hover on public pages emojis too

* Fix tests

* Code style cleanup

* Fix animate on hover in poll options without CW (tootsuite#11404)

* Change domain block behaviour to prevent creation of accounts from suspended domains (tootsuite#11219)

* Change ActivityPub::DeliveryWorker to not retry HTTP 501 errors (tootsuite#11233)

* Change the retry limit in error of web push notification (tootsuite#11292)

- Change the maximum count of retry for web push notification (Default -> 5).
   - In case of high load of subscribe server, the retries will be repeated many times.
   - Because the retries occupy the default queue, maximum retry count should be reduced.

* Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (tootsuite#11302)

* Make puma bind address configurable with BIND env var (tootsuite#11326)

* Change terms and privacy policy pages to always be accessible (tootsuite#11334)

Fix tootsuite#11328

* Change language detection to include hashtags as words (tootsuite#11341)

* Fix only one middle dot being recognized in hashtags (tootsuite#11345)

Fix tootsuite#10934

* Change Dockerfile to bind to 0.0.0.0 instead of docker-compose.yml (tootsuite#11351)

* Fix sanitizing lists contents (tootsuite#11354)

* Add test

* Fix code for sanitizing nested lists stripping all tags

* Fix avatar animation on hover when not logged in (tootsuite#11349)

* Added logout to dropdown menu (tootsuite#11353)

* Added logout to dropdown menu

* Triggering build-and-test with empty commit as it seems it failed due to some internal failure

* Looks fine, ready to review

* Added changes from review

* method can be null without any problems

* Also target can be null

* Disallow numeric-only hashtags (tootsuite#11363)

* Add spec covering numeric-only hashtags

* Fix hashtag regex

* Change locale detection to run once per session (tootsuite#8657)

Fix tootsuite#6462

* Bind servers to 0.0.0.0 in Procfile (tootsuite#11378)

* Bind to 0.0.0.0

* Make Procfile common to main and streaming apps

* Change account domain block to clear out notifications and follows (tootsuite#11393)

* Remove pre from version, add extra suffix variable (tootsuite#11407)

* Remove timestamps from converted images to make them deterministic (tootsuite#11408)

* Prevent archiving when user set "noindex" (tootsuite#11421)

* Disable list title validation button when list title is empty (tootsuite#11475)

* Fix timestamp on featured tag (tootsuite#11477)

It resolves tootsuite#11338

* Trap tab in modals (tootsuite#11493)

* Fix privacy dropdown active state when dropdown is placed on top of it (tootsuite#11495)

* Improve dropdown menu keyboard navigation (tootsuite#11491)

* Allow selecting menu items with the space bar in status dropdown menus

* Fix modals opened by keyboard navigation being immediately closed

* Fix menu items triggering modal actions

* Add Tab trapping inside dropdown menu

* Give focus back to last focused element when status dropdown menu closes

* Improve keyboard navigation in privacy dropdown (tootsuite#11492)

* Trap tab in privacy dropdown

* Give focus back to last focused element when privacy dropdown menu closes

* Actually give back focus to the element that had it before clicking the dropdown

* Fix image uploads being perfectly white when canvas read access is blocked (tootsuite#11499)

Fixes tootsuite#11496

* Improve focus handling with dropdown menus (tootsuite#11511)

- Focus first item when activated via keyboard
- When the dropdown menu closes, give back the focus to
  the actual element which was focused prior to opening the menu

* Fix "cancel follow request" button having unreadable text in web UI (tootsuite#11521)

Fix tootsuite#11478

* Add GIF and WebP support for custom emojis (tootsuite#11519)

Fix tootsuite#11466

* Fix pinned statuses API returning pagination headers (tootsuite#11526)

Fix tootsuite#10227

* Fix crash when saving invalid domain name (tootsuite#11528)

Fix tootsuite#7629

* Fix “read more” button behing hidden (regression from tootsuite#11404) (tootsuite#11522)

* Fix “read more” button behing hidden (regression from tootsuite#11404)

This has the side-effect of putting the “Read more” button below possibly
trunctated polls instead of putting the poll below the “Read more”

* Remove dead code

* Bump version to 2.9.3

Co-authored-by: Renato "Lond" Cerqueira <renato@lond.com.br>
Co-authored-by: Steven Tappert <admin@dark-it.net>
Co-authored-by: m.b <mbajur@users.noreply.github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: ThibG <thib@sitedethib.com>
Co-authored-by: mayaeh <mayaeh@marimo-net.org>
Co-authored-by: Dan Hunsaker <danhunsaker@gmail.com>
Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>
Co-authored-by: valerauko <vale@valerauko.net>
Co-authored-by: Hugo Gameiro <hmgameiro@gmail.com>
Co-authored-by: Sam Schlinkert <sschlinkert@gmail.com>
Co-authored-by: Clar Charr <clar@charr.xyz>
Co-authored-by: Jakub Mendyk <jakubmendyk.szkola@gmail.com>
Co-authored-by: rinsuki <428rinsuki+git@gmail.com>
Co-authored-by: trwnh <a@trwnh.com>
Co-authored-by: abcang <abcang1015@gmail.com>
Co-authored-by: Hinaloe <hina@hinaloe.net>
Co-authored-by: Franck Zoccolo <franck@zoccolo.com>
Co-authored-by: ysksn <bluewhale1982@gmail.com>
Co-authored-by: nightpool <nightpool@users.noreply.github.com>
Co-authored-by: Ben Lubar <ben.lubar+github@gmail.com>
Co-authored-by: Nolan Lawson <nolan@nolanlawson.com>
Co-authored-by: nzws <git-yuzu@svk.jp>
Co-authored-by: Jeong Arm <kjwonmail@gmail.com>
Co-authored-by: Maciek Baron <thebezet@gmail.com>
Co-authored-by: nzws <git@nzws.me>
Co-authored-by: Neil Moore <dar13.dev@gmail.com>
Co-authored-by: Ben Lubar <ben.lubar@gmail.com>
Co-authored-by: Georg Gadinger <nilsding@nilsding.org>
Co-authored-by: han@highemelry <high.emerly.ytfin@gmail.com>
Co-authored-by: Daigo 3 Dango <zunda@users.noreply.github.com>
Co-authored-by: koyu <me@koyu.space>
Co-authored-by: Clar Fon <them@lightdark.xyz>
ThibG and others added 20 commits Sep 14, 2020
* Do not serve account actors at all in limited federation mode

When an account is fetched without a signature from an allowed instance,
return an error.

This isn't really an improvement in security, as the only information that was
previously returned was required protocol-level info, and the only personal bit
was the existence of the account. The existence of the account can still be
checked by issuing a webfinger query, as those are accepted without signatures.

However, this change makes it so that unallowed instances won't create account
records on their end when they find a reference to an unknown account.

The previous behavior of rendering a limited list of fields, instead of not
rendering the actor at all, was in order to prevent situations in which two
instances in Authorized Fetch mode or Limited Federation mode would fail to
reach each other because resolving an account would require a signed query…
from an account which can only be fetched with a signed query itself. However,
this should now be fine as fetching accounts is done by signing on behalf of
the special instance actor, which does not require any kind of valid signature
to be fetched.

* Fix tests
…sts (#14479)

* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
* Add support for inlined objects in activity audience

* Add tests
* use custom private boost icon for detail status

* only use className
Follow-up to #14359

In the case of limited toots, the receiver may not be explicitly part of the
audience. If a specific user's inbox URI was specified, it makes sense to
dereference the toot from the corresponding user, instead of trying to find
someone in the explicit audience.
* Add support for latest HTTP Signatures spec draft

https://www.ietf.org/id/draft-ietf-httpbis-message-signatures-00.html

- add support for the “hs2019” signature algorithm (assumed to be equivalent
  to RSA-SHA256, since we do not have a mechanism to specify the algorithm
  within the key metadata yet)
- add support for (created) and (expires) pseudo-headers and related
  signature parameters, when using the hs2019 signature algorithm
- adjust default “headers” parameter while being backwards-compatible with
  previous implementation
- change the acceptable time window logic from 12 hours surrounding the “date”
  header to accepting signatures created up to 1 hour in the future and
  expiring up to 1 hour in the past (but only allowing expiration dates up to
  12 hours after the creation date)
  This doesn't conform with the current draft, as it doesn't permit accounting
  for clock skew.
  This, however, should be addressed in a next version of the draft:
  httpwg/http-extensions#1235

* Add additional signature requirements

* Rewrite signature params parsing using Parslet

* Make apparent which signature algorithm Mastodon on verification failure

Mastodon uses RSASSA-PKCS1-v1_5, which is not recommended for new applications,
and new implementers may thus unknowingly use RSASSA-PSS.

* Add workaround for PeerTube's invalid signature header

The previous parser allowed incorrect Signature headers, such as
those produced by old versions of the `http-signature` node.js package,
and seemingly used by PeerTube.

This commit adds a workaround for that.

* Fix `signature_key_id` raising an exception

Previously, parsing failures would result in `signature_key_id` being nil,
but the parser changes made that result in an exception.

This commit changes the `signature_key_id` method to return `nil` in case
of parsing failures.

* Move extra HTTP signature helper methods to private methods

* Relax (request-target) requirement to (request-target) || digest

This lets requests from Plume work without lowering security significantly.
@blaine
Copy link

@blaine blaine commented Nov 5, 2020

Just bumping this - I want to add that most of the concerns here seem to be around forcing people to be on a single instance, and breaking the federated model. I want to add two things that haven't been discussed as possibilities on this issue as far as I can tell:

  1. A clear use-case for this feature beyond small niche networks is intra-company networks; if I wanted to set up a mastodon instance for my company, I would need to not federate, since clearly the goal would be to share private or commercially sensitive content. However, with local-only posts, it would be possible for employees to share some things publicly, but most things privately.

  2. If people are concerned with that loss of federation means that multiple accounts need to be maintained to access multiple networks, set up the activitypub equivalent of email forwarding. There's nothing in federation model that prevents this (indeed, it was a core design goal when we created webfinger). Where forwarding doesn't work (e.g., if you want to create a "local" community that is composed of people who are distributed across a variety of mastodon instances), the "Groups" model that has been mentioned in the comments on this issue and works for virtually all chat applications would be a logical extension of this behaviour.

There are plenty of legal and commercial reasons to implement this, and I'm happy to discuss those if it's relevant or interesting. I for one firmly believe that the Fediverse can only be truly successful if it manages to dislodge the incumbents' control over online sociality. The only way that can happen is if it's both possible and supported for the fediverse to be used in commercial contexts, and I sincerely hope that's coming. I'd rather a world without capitalism, but honestly I don't think that's Mastodon's fight to win.

If there's a better way to implement this sort of site-specific functionality, possibly through some plugin infrastructure or similar, I'd be keen to explore that. I'm not up to date with the current state of mastodon so it's entirely possible that I've missed something, but I need this or similar functionality and would prefer to not diverge massively from mainline mastodon, precisely for the reason of continuing support for federation! It'd be easy to run a non-federating instance to achieve my goals, but the downside is immense.

@GerryT11
Copy link

@GerryT11 GerryT11 commented Nov 13, 2020

I am in favor of this feature, for these reasons here: #861 (comment)

I agree with @Gargron that it doesn't make sense for general-purpose instances (e.g. mastodon.social) and would be counterproductive there, but it would allow local communities (in my case a local sports club) that have internal/irrelevant stuff and public toots. Thus, why not having this feature (built-in or as add-on) and disabling it by default, but the admin of an instance can switch it on?

@renatolond
Copy link
Collaborator Author

@renatolond renatolond commented Nov 20, 2020

I've merged the latest tag on this :)

Base automatically changed from master to main Jan 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.