Note: There is an important database migration in this release. Please read the upgrade notes carefully.

Features:

Add delete & redraft function (#7735)

If you noticed a typo or missing word on a toot that's already been sent, you can now choose to "delete & redraft" that toot. It pre-fills the compose area with all the data from your toot, including attached files, allowing you to edit and re-send it. It becomes a new toot, so boosts, favourites and replies to it will be reset.

Improved getting started column (#7676)

On mobile, the getting started column more prominently displays a link to your profile, preferences and security settings, and omits linking to timelines or columns that are anyway listed in the tab bar above.

On desktop, the getting started column now categorizes links into "Discovery" and "Personal" which should help new users orient themselves in the app. Links to blocked users, muted users and domain blocks have been moved to the dropdown menu on your own profile.

On both layouts, the bottom of the column has been restyled. Links to the FAQ, user guide and apps list have been replaced with links to "Hotkeys", "About this instance", "Terms of Service", "Documentation" and "Logout". Documentation, in turn, contains the FAQ, user guide and apps list.

Improved e-mail validation (#7631)

On sign up, entered e-mail address will be checked for the presence of an MX record, which indicates that it's potentially a real e-mail address. Furthermore, the MX record is checked against the e-mail domains blacklist, which allows blocking spammers who have many alias domains pointing to the same e-mail server.

Improve "Hide everything from {domain}" behaviour (#7765, #7773)

The confirmation dialog now explains what will happen when you block a domain. It will remove your followers from that domain, prevent new followers or follow requests from that domain, hide toots from the domain in public timelines, hide boosts of toots from the domain in your home timeline, and hide any mentions or notifications from that domain. It will not make you unfollow someone from that domain you've already been following, and those people will be able to appear in your home feed and notifications.

Filter out blocked/muted people from profile timelines (#7747)

Due to popular demand, boosts of blocked/muted people will no longer appear on profiles you view.

Other:

UI/UX additions:

• Public timelines now have a tab bar for choosing between all toots and only ones that contain media attachments (#7598)
• Submit report with ctrl+enter (#7729)
• Emoji pack upgraded, new unicode emojis available in the picker (#7746)
• Add "Edit profile" link to public profile page (#7754)

REST API additions:

• New API endpoint: GET /api/v2/search, which returns the same results as v1, except the hashtags are returned as objects with trend data instead of mere strings (#7661)

Performance improvements:

• Optimize direct timeline (#7614)
• Reduce wasted work in RemoveStatusService due to inactive followers (#7672)
• Improve counter caches on Status and Account (#7644)

Fixes:

Backend:

• Disable AMS logging (#7623)
• Catch ActionController::UnknownFormat and return HTTP 406 (#7621)
• Rescue Mastodon::DimensionsValidationError in Remoteable (#7662)
• Fix error when unmuting a domain without listing muted domains first (#7670)
• Deduplicate accounts and make unique username/domain index case-insensitive (#7658)
• Skip processing when HEAD method returns 501 (#7730)
• Detect file extension from Content-Type header in Remoteable (#7733)
• Migrate old web push subscriptions to ensure deliveries (#7764)

Deployments:

• Remove Puma pidfile before boot if container receives SIGTERM (#7052)
• Speed up some rake tasks by moving execution to Sidekiq (#7678)

ActivityPub:

• Ignore multiple occurrences of a hashtag within a status (#7606)
• Do not mark remote status as sensitive even if spoiler text is present (only apply that logic locally) (#7395)
• Do not accept ActivityPub follow requests from blocked user (#7756)

OStatus:

• Fix N+1 on AtomSerializer (#7669)

UI/UX:

• Fix caret position after selected suggestion and media upload (#7595)
• Fix lock icon position in account card (#7630)
• Don't use Object.assign with Notification, only display actions for mentions (#7632)
• Redirect / to home on mobile layout, to getting started on desktop (#7677)
• Put the CW field between the toot we are replying to and the toot field (#7508)
• Control the focus when clicking the CW button. (#7776)
• Display numbers in account header using shortNumberFormat for consistency (#7723)
• Remove unnecessary underline on admin accounts table (#7728)
• Preserve newlines in delete & redraft and desktop notifications (#7750)
• Improve emoji picker design in light theme (#7772, #7768)
• Fix some colors in light theme (#7722)

Other:

• Added the law requirements for the EU/EEA into the default privacy policy (#7605)

Upgrade notes:

As always, make sure you have backups of the database before performing any upgrades. If you are using docker-compose, this is how a backup command might look: docker exec mastodon_db_1 pg_dump -Fc -U postgres postgres > name_of_the_backup.dump

Please read: So far, Mastodon had been missing a database constraint for case-insensitivity (e.g. capital A vs lowercase a) due to an early mistake. Mostly it's been fine due to code outside of the database ensuring integrity, but that code has not always been perfect, and among other things sometimes failed due to race conditions. Long story short, your database may contain account records which share the same username/domain combination and are therefore unreachable by mentions, URLs, etc, and which sometimes lead to unexpected behaviour.

This release contains a fix for that situation. The database migrations will find affected accounts and either merge them (when possible and applicable) or delete them without trace, after which the database constraint will be created to ensure no data integrity issues of this nature occur in the future.

For remote accounts, the most recently active one will be the reference account into which others will be merged (but only as long as they have the same public key, which is definitive proof that it's really the same account).

For local accounts, the oldest account will be kept and others deleted. For people behind those duplicate accounts, Mastodon has never been fully functional, because they could never open their profile or receive mentions. However, if you want to manually deal with those accounts instead of having the migration delete them, before running the migration, you can use the rake mastodon:maintenance:find_duplicate_usernames task.

Non-Docker only:

• Dependency updates: yarn install

Both Docker and non-Docker:

• This release includes database migrations, that means you need to run RAILS_ENV=production bundle exec rails db:migrate (in Docker: docker-compose run --rm web rails db:migrate).
• This release includes changes to assets, that means you need to run RAILS_ENV=production bundle exec rails assets:precompile (in Docker: docker-compose run --rm web rails assets:precompile)

Contributors to this release:

@abcang
@akihikodaki
@ariasuni
@Gargron
@imbsky
@kedamaDQ
@kibitan
@lynlynlynx
@m4sk1n
@nightpool
@renatolond
@Reverite
@SerCom-KC
@shuheiktgw
@takayamaki
@tateisu
@ThibG
@unarist
@ykzts

Features:

Offline functionality: (#6876, #6886)

The Mastodon webapp is a Progressive Web App, and now it can run without an active internet connection, too. While many of the functions will not be available, already loaded content will remain accessible. Once a connection is re-established, a clickable gap will be displayed in the columns, allowing you to load things you might have missed while you were offline.

Direct messaging improvements (#6956, #7089, #4514, #7067)

You can now begin a direct message to someone from the dropdown menu on their profile or on their toots. A warning will be displayed that all mentioned users will be able to see the message. A new type of column is now available, which lists all of your direct message correspondence.

New profile metadata (#6645, #7288)

You can now set up to 4 custom properties on your public profile (label and value). For example, you could link to your website, your Patreon, list your e-mail address for inquires, your pronouns, or who drew your avatar.

RSS for users (#7259)

User profiles and hashtags now offer RSS feeds. The content inside the user profile feed is the same as on the Atom feed (public and unlisted toots), but in a more feed reader-friendly format.

Admin UI improvements (#7188, #7189, #7347, #7342)

The report screen has been revamped. Staff can leave notes on reports as well as individual accounts. A history of actions performed on the report is displayed right there and then. The reported toots are displayed in a more compact and polished manner.

The admin view of an account's toots no longer includes private and direct toots. However, if they are reported, they still show up on the report screen.

When a report comes from another server, the account associated with it is not actually the person who sent the report, but a representative account of the server it was sent from, e.g. an admin. Now the report UI reflects this to reduce confusion.

Updated privacy policy (#6666)

We were previously using the privacy policy from Discourse verbatim (which, in turn, is a verbatim copy of the one in WordPress). The policy contained a lot of protections for behaviour the Mastodon software was not exhibiting. The new policy is more narrow and explicit and explains in more detail what kind of data you can store in Mastodon and how it is used. Instead of 5 years, automatic scrubbing of old IP addresses in the database will occur every 12 months.

Bot accounts (#7391)

If you run bots on Mastodon, you can now opt-in to display a bot badge on your profile. This works with non-Mastodon software, too, if the ActivityPub actor is of the Service or Application type. In the future, more features might be implemented to filter bot accounts or opt-out of interactions with them.

Custom emojis in profiles (#6124, #7374)

You can now use custom emojis in your profile's bio, in your display name, and in the values of the profile metadata properties mentioned earlier.

Add preference to hide following/followers lists (#7532)

You can now choose to hide who you follow and who follows you from your public profile. The setting likewise hides this information from ActivityPub data and the REST API. Please mind that such information is an important discovery mechanism for other people for finding good content, but it can also be abused for profiling by association, which is why we are adding this option. Please also mind that the information could be stitched together under certain circumstances: a server where you have a number of followers will know about those followers, another server will know about its followers, etc.

Other:

UI/UX additions:

• Added missing management UI for user-hidden domains (#6628)
• Allow boosting own private toots to followers (#6157)
• Collapse overly long conversations on public pages, with controls for expanding (#7102)
• Added hotkey for revealing/hiding text behind a content warning (#7173)
• Added high contrast theme (#7213)
• Automatically resize images before upload in web UI to reduce bandwidth usage (#7223)
• "Administered by" information on the frontpage (#6984)
• Add search item to tab bar for mobile devices (#7072)
• Hide search from compose tab on mobile devices (#7077)
• Show media in a modal on public pages too (#6801)
• Added contact e-mail hint to 2FA login form (#7376)
• Added hint about 7 day cooldown for archive takeout (#7375)
• Show media modal on public timeline (#7413)

Administration additions:

• Ability to define a list of disallowed hashtags (#7176)
• Added "1 week" as expiry option for invites (#6872)
• Admins and moderators now have the ability to remove an account’s avatar (#6998)
• Ability to change the user’s email address (#7074)
• Ability to resend confirmation emails (#7378)
• Allow searching for custom emojis by incomplete shortcode in admin UI (#7099)

Deployment additions:

• Ability to specify Redis password during mastodon:setup (#7222)
• Enable ElasticSearch support by default on Nanobox (#6977)
• Support for running Mastodon as a hidden service (e.g. Tor) (#7134)
• Log when a rate limit is hit by someone (#7096)

REST API additions:

• Enable updating additional account information from user preferences via REST API (#6789)
• New rate limit for POST /api/v1/media to limit amount of data someone could upload in 24h to 10GB (#7337)
• Support explicitly supplying language code for status via REST API (#7389)
• Disable API access when login is disabled for the account (#7289)
• Return HTTP 410 for suspended accounts in GET /api/v1/accounts/:id (#7287)
• Add REST API for Web Push Notifications subscriptions (#7445)

Performance improvements:

• Improve performance of rendering mentions and custom emojis in text (#7271)
• Add support for a separate Redis server for volatile cache (#7272)
• Validate HTTP response length while receiving (#6891)
• Add a circuit breaker for ActivityPub deliveries to minimize 10s timeouts (#7053)
• Detect and prevent image bombs, max. processable dimension 4096^2 (#7229)
• Perform processing that does not use the database before connecting to the database in streaming API (#7168)
• Marginally optimize RAM usage (#7301)
• Reduce needlessly rendered data in ActivityPub (#7357)
• Store home feeds for 7 days instead of 14 (#7354)
• Marginally improve file/identify/convert/ffmpeg calls performance with posix-spawn (#7346)
• Improve performance of POST /api/v1/statuses (#7317)
• Improve performance when fetching conversation threads (#7321)
• Improve performance of rendering Webfinger response (#7319)
• Improve web UI load performance when there are a lot of custom emojis on the server (#7047)
• Support gzip encoding on HTTP requests (#7425)
• Disallow async function in service worker to allow minimizing the JS (#7482)
• Do not use permitted_for scope when querying pinned statuses (#7510)

Fixes:

Backend:

• Rescue SSL errors when processing mentions, remove useless line (#7184)
• Prevent animations in OpenGraph preview cards (#7109)
• Ensure SynchronizeFeaturedCollectionWorker is unique and clean up (#7043)
• Allow more than the max pinned toots if account is not local (#7105)
• Improve GIFV encoding params (#7098)
• Remove most behaviour disparities between blocks and mutes (#7231)
• Fix unpermitted parameters warning when generating pagination URLs (#6995)
• Rescue Mastodon::LengthValidationError in FetchLinkCardService (#7424)
• Catch Paperclip processing failures (fixes #6378) (#7439)
• Update session activation time (fixes #5605) (#7408)
• Raise Mastodon::RaceConditionError if Redis lock failed (#7511)

Deployments:

• Add missing OTP_SECRET in scalingo.json (#6917)
• Do not default SMTP verify mode to "peer", default to "none" (#6996)
• Improve OpenStack v3 compatibility (#7392)

REST API/API:

• Prevent suspended accounts from appearing in search results when it's an exact match (#7246)
• When creating status, if no sensitive param is given, use user's default (#7057)

ActivityPub:

• Support actors/statuses with multiple types (#7305)
• Store URIs of follows, follow requests and blocks for ActivityPub to pass them back correctly (#7160)
• Improve pagination for ActivityPub outbox, following and followers collections (#7356)
• Fix handling of malformed ActivityPub payloads when URIs are nil (#7370)
• Fix add/remove activities for pinned toots not being sent (#7393)
• Forward deletes on the same path as reply forwarding (#7058)
• Do not ignore unknown media attachments, only skip them (#6948)
• Fix hashtags not being federated together with mentions (fixes #6900) (#7406)
• Take the first recognized actor_type. (#7410)
• Fetch boosted statuses on behalf of a follower (fixes #7426) (#7459)
• Fix account URI not updating when updating ActivityPub account (#7488)
• HTTP signatures spec no longer requires algorithms field (#7525)
• User agent for WebFinger (#7531)
• Resolve unknown status from Add activity, skip Remove if unknown (#7526)
• Do not raise delivery failure on 4xx errors, increase stoplight threshold (#7541)

OStatus:

• The special handling of the "nsfw" hashtag is removed for everything except OStatus. Also, it is now only added to an outgoing status if any media is attached, rather than always when a content warning is present (#7398)
• Fix custom emoji handling in UpdateRemoteProfileService (OStatus) (#7501)

UI/UX:

• Improve relative timestamps in web UI, show year in dates older than a year (#7233)
• Place emoji picker top if it is closer to the bottom of the viewport (#7314)
• Place privacy dropdown menu top if it is closer to the bottom of the viewport (#7106)
• Fix esc hotkey behavior (#7199)
• Fix the hot key (j, k) does not function correctly when there is a pinned toot in account timeline. (#7202)
• Fix caret position after inserting emoji (#7167)
• Make scroll bars a bit wider on webkit browsers (#7060)
• Change icon for domain blocks (#7139)
• Remove duplicate frequently used emojis (#7064)
• Improve dropdowns accessibility (#7318)
• Set max-height to videos (and gif videos) on modals (#6914)
• Note if the user is already following the target when authorizing follow (#6325)
• Set Referrer-Policy to origin in web UI and public pages of private toots to obfuscate what you were viewing in web UI (#7162)
• When notification type is filtered, ignore live updates for it, preventing gradual emptying of the column (#7101)
• Optimize public/headers/missing.png (#7084)
• Fix text color in "show more" link inside boost confirmation modal (#7183)
• Able to deactivate invites if they aren't expired (#7163)
• Use randomized setTimeout when fallback-polling and re-add since_id (#7522)
• Skip pagination logic for pinned account timelines in reducer (#7540)
• Do not override the default push notification settings (#6037)
• In footer, replace text "Mastodon" with logo (#7545)
• Disables autocorrect/autocapitalize on remote username field. (#7549)
• Improve default background of public profile header (#7556)
• Use real container width in MediaGallery srcSet (#7571)

Other:

• Use RAILS_LOG_LEVEL to set log level of Sidekiq, too (#7079)

Upgrade notes:

As always, make sure you have backups of the database before performing any upgrades. If you are using docker-compose, this is how a backup command might look: docker exec mastodon_db_1 pg_dump -Fc -U postgres postgres > name_of_the_backup.dump

Non-Docker only:

• Dependency updates: bundle install and yarn install

Both Docker and non-Docker:

• This release includes database migrations, that means you need to run RAILS_ENV=production bundle exec rails db:migrate (in Docker: docker-compose run --rm web rails db:migrate).
• This release includes changes to assets, that means you need to run RAILS_ENV=production bundle exec rails assets:precompile (in Docker: docker-compose run --rm web rails assets:precompile)

Troubleshooting

• If you are on Ruby 2.3.x and lower you will get errors when uploading images. Please upgrade to Ruby 2.5.1 or at least 2.4.x
• If avatars and images in web UI are suddenly not loading, check if the server serves them with a CORS header, e.g. Access-Control-Allow-Origin: https://example.com/ where example.com is your domain. This is needed for the offline functionality of the webapp to work.
• If image uploads stopped working (images won't even begin uploading), it's likely you have a restrictive CSP (Content-Security-Policy) header set up and need to adjust it (allow blob: as img-src and connect-src). This is because overly large images will be downsized in the browser before upload to save bandwidth
• If you get 500 errors related to cache, this might be due to the upgrade from Rails 5.1 to Rails 5.2. The cache can be discarded by using RAILS_ENV=production bundle exec rails console: Rails.cache.delete
• If you are using Ceph for uploads, add S3_SIGNATURE_VERSION=s3

How to view logs

I get this question a lot so let's get this out of the way. The errors you see in the browser (with the disappointed elephant) are always reflected in the log of the web process. Here is how to view those logs:

• With Docker: docker logs mastodon_web_1 (add -f for live scroll)
• Non-Docker: journalctl -u mastodon-web (add -f for live scroll)

Furthermore, each request has a unique Request-Id header, which you can get from the Network tab of your web inspector in the browser. You can search the logs with that Request-Id value to find specifically the error message of that request.

Note: If the web process isn't actually started, the error will not be in those logs. It will most likely be in the Nginx error log, if you use Nginx, e.g.: /var/log/nginx/error.log by default. And it will most likely be about how the web process isn't running.

Contributors to this release:

@abcang
@akihikodaki
@ashfurrow
@beatrix-bitrot
@Dar13
@ekiru
@Gargron
@goofy-bz
@hcmiya
@hugogameiro
@imbsky
@jenkr55
@jumoru
@KScl
@luzi82
@lynlynlynx
@m4sk1n
@MasterGroosha
@matthiasbeyer
@maxolasersquad
@mayaeh
@nightpool
@petzah
@Quenty31
@renatolond
@retokromer
@SerCom-KC
@shuheiktgw
@stemid
@sts10
@Sylvhem
@takayamaki
@TakesxiSximada
@Technowix
@ThibG
@ThisIsMissEm
@unarist
@unleashed
@wiktor-k
@ykzts