RVM has a known issue with installing Ruby 2.5 on the version of Ubuntu
the Vagrant box is using: rvm/rvm#4291

This bug was preventing any gem installs in the vagrant box

Mitigation for CVE-2017-0889.

https://www.cvedetails.com/cve/CVE-2017-0889/
https://medium.com/in-the-weeds/all-about-paperclips-cve-2017-0889-server-side-request-forgery-ssrf-vulnerability-8cb2b1c96fe8

* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style

…signatures (mastodon#6225)

…todon#6236)

They were redundant, and also had a inconsistency; the button for
"other instances" had an icon for the external link in "more" page, but
it didn't in the other pages.

This unifies the implementation, and the external link icon is now shown
in all the about pages.

* moved save button

* added save back to the bottom

* Update show.html.haml

…ty) (mastodon#6207)

It is required for ApplicationController.

…don#6417)

…ion (mastodon#6369)

The database and Redis do not need external connections, so isolate them
and prevent unauthorized access.

* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales

* Add option to show only local toots in timeline preview
Right know, toots from all the known fediverse are shown in the main
page of an instance. That however doesn't reflect the instance itself.
With this option the admin may choose to display only local toots so
that users checking the instance get a better idea of internal
conversations.

* Fix issues pointed by codeclimate and eslint

* Add default message for community timeline

* Update pl.yml

* upgrade vagrant box to xenial

this allows the redis version to be upgraded to support the new redis
features used in the activity tracker

* add libpam0g package to vagrant box

this is required for native extensions of gems to build after the
addition of PAM support was added in mastodon#5303

)

…#6415)

the slight mismatch in hashtag regex between js and ruby was causing
hashtag warning to be displayed for unlisted tweets when an invalid
hashtag was entered

exact version of ruby regex not possible in js as POSIX bracket
expressions are not supported, this version approximates and doesn't
give same unicode support

* Added #link_header spec

* Added #perform_request spec

…#6447)

* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix mastodon#5880
Fix mastodon#4293
Fix mastodon#1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues

* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Fix password recovery

* Use “resource” instead of “current_user”

* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added #call spec

* Add better CLI prompt

* Add rake mastodon:setup interactive wizard

* Test db/redis/smtp configurations and add admin user at the end

* Test database connection even when database does not exist yet

In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed

The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

Reasoning: HTML title tag affects everyone. But OpenGraph only affects
when somebody is deliberately sharing the content, usually in an
environment where such content is expected. Hiding the content in
OpenGraph tags results in deceitful previews which inhibit the
shareability of the post.

Example: Somebody writes a clever post about politics but kindly
puts a "uspol" content warning on it. Mastodon users are thankful,
but sharing this post on another platform results in non-Mastodon
users believing the entire contents of the post is "uspol" and not
clicking through/reading and re-sharing.

* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test

Some available languages lack translations for notification mails. Now it
tests for two languages which is certain to have required translations:
German and English.

German is the language the current project owner, Eugen Rochko speaks, and
providing English translations for new messages is de facto mandatory.

…stodon#6484)

Time window of 6 hours

The cache store is explicitly used by some specs, but they were not
isolated and therefore not reliable. This fixes the issue by clearing
the cache after each specs.

* Cache relationships in API

* Fetch relationships for search results in UI

* Only save one account's maps in each cache item

…n#6500)

* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added call test case activitypub

- 4px rounded corners on media attachments
- Better colors/contrast for CW/media spoiler on public pages
- Fix vertical alignment of "Show more" button
- Fix layout jump when unhiding standalone media

…astodon#6508)

Also don't apply "-quality 80" option which is probably the reason
for slight color differences between original and remote image
(because it would apply it twice, once on original instance, and
again on the receiving instance)

* Set Docker permissions during the build process

* Remove docker_entrypoint.sh and use COPY with chown

…on (mastodon#6519)

…don#6515)

* Fix avatar and header issues by using custom geometry detector

Revert a part of mastodon#6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed

…it (mastodon#6521)

* Fix mastodon#201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue

It was merge when it needed to be deep_merge. And added some tests

mastodon#6529)

* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments

…mastodon#6528)

* Redesign landing page (again)

* Move login form in small version to the right column

* Display closed registrations message

* Add site setting for the hero image

* Fix test

* Increase spacing, maximum width, change call to action section

respond to!

* Update public.html.haml

* Update auth.html.haml

* Update modal.html.haml

…odon#6533 (mastodon#6538)

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

* document pam email extraction

* remove superfluous newline

…SAML strategies) (mastodon#6540)

…n#6488)

A complemental change for precompute_feed_service_spec.rb also fixes its
random failure which is caused by the Snowlake randomization of the order
of an original status and its reblog.

* Update Japanese translations.

* Add quotation marks.

UglifyJS2 is allowed to mangle function names, and function names can also
be duplicate if they are from different scopes. Therefore function names
are not reliable as identifiers.

Functions as keys for Map object is a cheaper and more reliable
alternative.

…vate (mastodon#6410)

…#6557)

* Ensure the app does not even start if OTP_SECRET is not set

* Remove PAPERCLIP_SECRET (it's not used by anything, actually)

Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them

* Fix prev/next links on public profile page

* Don't make pagination urls if no available statuses

* Fix empty check method

* Put left chevron before prev page link

* Add scope for pagination "starting at" a given id

* Status pagination try 2:

s/prev/older and s/next/newer
"older" on left, "newer" on right
Use new scope for "newer" link
Extract magic 20 page size to constant
Remove max_id from feed pagination as it's not respected

* Reinstate max_id for accounts atom stream

* normalize

- Add follow/unfollow/remote follow buttons
- Format the bio properly
- Always show username@domain, even for local accounts

* Use grid layout for the landing page

* Fix column settings

Set the ratio explicitly

* Improve information board

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

* Use grid layout for the landing page

* Use grid layout for tag pages

* Set 2 columns width as explicit percentage for tag pages

mastodon#6565)

* Some images can cause `convert` to fail, which crashes this whole task

* Add more specific exception

* Specify AGPLv3+

Since the documentation doesn't specify you can use Mastodon as AGPLv3 or any later version.

* Use newest version of SPDX AGPLv3+ identifier

* Fix mastodon#2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService