Merge branch 'develop' into feature/redesign-verifications

* develop: (35 commits)
  Install turbo-rails (#9881)
  Fix conference invitations (#9664)
  Fix invalid rendering of meeting and proposal body texts (#9764)
  Make documentation site work with multiple versions (#9917)
  Bump versions on install docs (#9916)
  Standardize CSV import formats and fix private users CSV import with invalid file (#9627)
  Fix: The i18n locales selector is showing a dropdown with 3 languages (#9902)
  Make Scopes field in debates translatable (#9903)
  Make ToS agreement translatable (#9909)
  Fix issues with a11y specs (#9929)
  Remove invitations badge (#9906)
  Make initiatives order translatable (#9905)
  Add missing active actions on admin navigation menu (#9904)
  Fix user sign up with invalid name (#9896)
  Remove duplication of LastActivity queries (#9895)
  Rename IgnoredMethods to AllowedMethods in Rubocop configuration (#9893)
  Exclude malformed file from codeclimate configuration (#9910)
  Fix correct resource linking for amendments (#9887)
  Fix superposition in admin's error forms (#9871)
  Add missing i18n key in Initiatives (#9892)
  ...

.codeclimate.yml

@@ -102,6 +102,7 @@ plugins:
       - "**/*/locales/*.yml"
       - "**/*/*.svg"
       - "decidim-dev/lib/decidim/dev/assets/iso-8859-15.md"
+      - "decidim-dev/lib/decidim/dev/assets/import_participatory_space_private_users_iso8859-1.csv"
 
   stylelint:
     # FIXME: after the webpacker packages changes, this is broken with this error:

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -28,7 +28,7 @@ If applicable, add the error stacktrace to help explain your problem.
  - Device OS: [e.g. iOS8.1, Windows 10]
  - Browser: [e.g. Chrome, Firefox, Safari]
  - Decidim Version: [e.g. 0.10]
- - Decidim installation: [e.g. MetaDecidim]
+ - Decidim installation: [e.g. Metadecidim]
 
 **Additional context**
 Add any other context about the problem here. For instance, add Metadecidim link. 

.github/ISSUE_TEMPLATE/Feature_request.md

@@ -1,14 +1,14 @@
 ---
 name: Feature request creation disabled
-about: Go to MetaDecidim :)
+about: Go to Metadecidim :)
 
 ---
 
-We've disabled the feature request's issue creation on this repository. If you want to discuss about a new feature, you can do this through MetaDecidim:
+We've disabled the feature request's issue creation on this repository. If you want to discuss about a new feature, you can do this through Metadecidim:
 
 * [Feature Request](https://meta.decidim.org/processes/roadmap)
 
-We're doing this because we're trying to eat our own dog food. Once your proposal is collaboratively reviewed and accepted by the Product Comitee, we'll create the new issue on GitHub and notify you through MetaDecidim.
+We're doing this because we're trying to eat our own dog food. Once your proposal is collaboratively reviewed and accepted by the Product Comitee, we'll create the new issue on GitHub and notify you through Metadecidim.
 
 Thanks for contributing to Decidim!
 

.rubocop_ruby.yml

@@ -220,7 +220,7 @@ Style/BlockDelimiters:
     # This looks at the usage of a block's method to determine its type (e.g. is
     # the result of a `map` assigned to a variable or passed to another
     # method) but exceptions are permitted in the `ProceduralMethods`,
-    # `FunctionalMethods` and `IgnoredMethods` sections below.
+    # `FunctionalMethods` and `AllowedMethods` sections below.
     - semantic
     # The `braces_for_chaining` style enforces braces around single line blocks
     # and do..end around multi-line blocks, except for multi-line blocks whose
@@ -259,7 +259,7 @@ Style/BlockDelimiters:
     - let!
     - subject
     - watch
-  IgnoredMethods:
+  AllowedMethods:
     # Methods that can be either procedural or functional and cannot be
     # categorised from their usage alone, e.g.
     #

CHANGELOG.md

@@ -109,6 +109,128 @@ You can read more about this change on PR [\#9480](https://github.com/decidim/de
 
 You can read more about Tailwind from the [Tailwind documentation](https://tailwindcss.com/docs/utility-first).
 
+#### 5.2. Automated authorization conflict handling for deleted users
+
+In previous Decidim versions authorization conflicts (i.e. authorizing the user with the same unique data as a previous user) needed to be always handled manually. Now these are automatically handled for cases where the original user had authorized their account, then deleted their account and finally authorized the new account with the same details as the previous account.
+
+This means that some participation data bound to the previous deleted user account is now automatically transferred over to the new account during the authorization process to prevent e.g. duplicate votes in budgeting votings. This includes any data that may or may not require an authorization through the component permissions because in Decidim we cannot be always perfectly sure when an authorization is required for the action or not. As an example, budget voting can start without an authorization and if the admin decides to configure an authorization for the component one day after the voting started, we need to assume that the all votes in that component required an authorization. Otherwise we would potentially allow multiple votes from the users that voted before the authorization was configured if they decided to create a new account to vote for a second time or deleted their original account and did that.
+
+The transferred data can differ between the different modules but the official modules handle the following data automatically:
+
+- **decidim-core**
+  - Amendments (meaning any amendments for amendable records in different modules, such as proposals at `decidim-proposals`)
+  - Coauthorships (meaning any coauthorable records in different modules, such as proposals and collaborative drafts at `decidim-proposals`)
+  - Endorsements (for any records, e.g. blog posts at `decidim-blogs`, debates at `decidim-debates` and proposals at `decidim-proposals`)
+- **decidim-blogs**
+  - Blog posts
+  - Endorsements for blog posts (through endorsement transfers at `decidim-core`)
+- **decidim-budgets**
+  - Budgeting votes (or orders as we call them in the code)
+- **decidim-comments**
+  - Comments
+  - Comment votes
+- **decidim-consultations**
+  - Consultation votes
+- **decidim-debates**
+  - Debates
+  - Endorsements for debates (through endorsement transfers at `decidim-core`)
+- **decidim-elections**
+  - Election votes
+  - Election form answers (through form answer transfers at `decidim-forms`)
+  - Feedback form answers (through form answer transfers at `decidim-forms`)
+- **decidim-forms**
+  - Form answers (for different forms, such as survey form answers at `decidim-surveys` or registration form answers at `decidim-meetings`)
+- **decidim-initiatives**
+  - Initiatives
+  - Initiative votes/signatures
+- **decidim-meetings**
+  - Meetings
+  - Meeting registrations
+  - Meeting poll answers
+  - Meeting registration form answers (through form answer transfers at `decidim-forms`)
+- **decidim-proposals**
+  - Proposal votes/supports
+  - Proposals (through coauthorship transfers at `decidim-core`)
+  - Collaborative drafts (through coauthorship transfers at `decidim-core`)
+  - Proposal amendments (through amendment transfers at `decidim-core`)
+  - Endorsements for proposals (through endorsement transfers at `decidim-core`)
+- **decidim-surveys**
+  - Survey form answers (through form answer transfers at `decidim-forms`)
+
+If external modules need to transfer records between accounts during the authorization transfers, module developers can define the following initializer at their modules (note that coauthorable records are automatically already handled):
+
+```ruby
+module Decidim
+  module YourModule
+    class Engine < ::Rails::Engine
+      # ...
+      initializer "decidim_your_module.authorization_transfer" do
+        Decidim::AuthorizationTransfer.register(:your_module) do |transfer, auth_hander|
+          # Define the record class as the first argument to be moved to the
+          # new user and the column name as the second argument that maps the
+          # record to the original user. This will update all records that match
+          # the old deleted account to the new user that was authorized using
+          # conflicting authorization data. If you need access to the
+          # authorization handler that caused the transfer to be initiated, it
+          # is available as the second yielded argument (auth_hander).
+          transfer.move_records(Decidim::YourModule::Foo, :decidim_author_id)
+        end
+      end
+      # ...
+    end
+  end
+end
+```
+
+By default you should handle transfer of all records that can require an authorization and leave instance implementers the possibility to disable those transfers if they want to as explained below.
+
+If you would like to disable the authorization transfers feature altogether, you can define the following code in your application class located at `config/application.rb` of your instance:
+
+```ruby
+module DecidimYourCity
+  class Application < Rails::Application
+    # ...
+    config.to_prepare do
+      Decidim::AuthorizationTransfer.disable!
+    end
+    # ...
+  end
+end
+```
+
+Note that when the functionality is disabled, the authorization transfers work as they used to, i.e. a conflict is registered, admin users are notified about the conflict situation and the conflict needs to be manually handled.
+
+If you would like to disable the authorization transfers only for specific modules, you can define the following code in your application class located at `config/application.rb` of your instance (pick only the modules you want to disable):
+
+```ruby
+module DecidimYourCity
+  class Application < Rails::Application
+    # ...
+    config.after_initialize do
+      Decidim::AuthorizationTransfer.unregister(:core) # any coauthorable records, e.g. proposals and collaborative drafts
+      Decidim::AuthorizationTransfer.unregister(:blogs) # blog posts
+      Decidim::AuthorizationTransfer.unregister(:budgets) # budgets
+      Decidim::AuthorizationTransfer.unregister(:comments) # comments
+      Decidim::AuthorizationTransfer.unregister(:consultations) # consultation votes
+      Decidim::AuthorizationTransfer.unregister(:debates) # debates
+      Decidim::AuthorizationTransfer.unregister(:elections) # elections
+      Decidim::AuthorizationTransfer.unregister(:forms) # form answers, e.g. survey form answers or meeting registrations
+      Decidim::AuthorizationTransfer.unregister(:initiatives) # initiatives and initiative votes/signatures
+      Decidim::AuthorizationTransfer.unregister(:meetings) # meetings, meeting registrations and meeting poll answers
+      Decidim::AuthorizationTransfer.unregister(:proposals) # proposal votes/supports
+
+      # Or if you want to unregister multiple modules at once
+      Decidim::AuthorizationTransfer.unregister(:blogs, :forms, :comments)
+    end
+    # ...
+  end
+end
+```
+
+Note that when unregistering an authorization transfer handler, the transfers will still work normally for the other transfer handlers and no conflicts are reported for the admin users in case of conflict situation between a new authorization and a previous authorization for a deleted user. In this case, the authorization is transferred to the new user normally but the unregistered transfer handlers are not called which means those records will not be transferred between the user accounts. For conflicts between normal registered users or managed users, the conflicts are still reported as before. The automated authorization transfers only happen in case the previously authorized conflicting user account was deleted.
+
+You can read more about this change at PR [\#9463](https://github.com/decidim/decidim/pull/9463).
+
 ### Detailed changes
 
 #### Added

Gemfile.lock

@@ -104,6 +104,7 @@ PATH
       rubyXL (~> 3.4)
       rubyzip (~> 2.0)
       seven_zip_ruby (~> 1.3)
+      turbo-rails (>= 1.3.0)
       valid_email2 (~> 4.0)
       webpacker (= 6.0.0.rc.5)
       webpush (~> 1.1)
@@ -752,6 +753,10 @@ GEM
     thor (1.2.1)
     tilt (2.0.10)
     timeout (0.3.0)
+    turbo-rails (1.3.0)
+      actionpack (>= 6.0.0)
+      activejob (>= 6.0.0)
+      railties (>= 6.0.0)
     tzinfo (2.0.4)
       concurrent-ruby (~> 1.0)
     uber (0.1.0)

config/i18n-tasks.yml

@@ -70,6 +70,7 @@ search:
   ##   %w(*.jpg *.png *.gif *.svg *.ico *.eot *.otf *.ttf *.woff *.woff2 *.pdf *.css *.sass *.scss *.less *.yml *.json)
   exclude:
     - decidim-dev/lib/decidim/dev/assets/iso-8859-15.md
+    - decidim-dev/lib/decidim/dev/assets/import_participatory_space_private_users_iso8859-1.csv
     - decidim-comments/app/assets/javascripts/decidim/comments/bundle.js
     - decidim-comments/app/assets/javascripts/decidim/comments/bundle.js.map
     - "*.jpeg"
@@ -252,6 +253,7 @@ ignore_unused:
   - decidim.amendments.accept.error
   - decidim.search.results
   - decidim.searches.filters.state.*
+  - decidim.security.selfxss_warning.*
   - decidim.assemblies.filter.*
   - decidim.initiatives.initiatives.votes_count.count.*
   - decidim.proposals.admin.participatory_texts.new_import.accepted_mime_types.*
@@ -326,6 +328,7 @@ ignore_unused:
   - decidim.initiatives.create_initiative.select_initiative_type.*
   - decidim.blogs.admin_log.*
   - decidim.templates.admin_log.*
+  - emojis.*
   - ransack.predicates.*
   - decidim.profiles.user.actions.*
   - decidim.forms.questionnaires.show.answer_questionnaire.*

decidim-accountability/app/services/decidim/accountability/results_csv_importer.rb

@@ -31,7 +31,7 @@ def import!
 
         ActiveRecord::Base.transaction do
           i = 1
-          csv = CSV.new(@csv_file, headers: true, col_sep: ";")
+          csv = CSV.new(@csv_file, headers: true, col_sep: Decidim.default_csv_col_sep)
           while (row = csv.shift).present?
             i += 1
             next if row.empty?

decidim-admin/app/commands/decidim/admin/process_participatory_space_private_user_import_csv.rb

@@ -5,7 +5,7 @@
 module Decidim
   module Admin
     class ProcessParticipatorySpacePrivateUserImportCsv < Decidim::Command
-      include Decidim::ProcessesFileLocally
+      include Decidim::Admin::CustomImport
 
       # Public: Initializes the command.
       #
@@ -34,10 +34,8 @@ def call
       private
 
       def process_csv
-        process_file_locally(@form.file) do |file_path|
-          CSV.foreach(file_path, encoding: "BOM|UTF-8") do |email, user_name|
-            ImportParticipatorySpacePrivateUserCsvJob.perform_later(email, user_name, @private_users_to, @current_user) if email.present? && user_name.present?
-          end
+        process_import_file(@form.file) do |(email, user_name)|
+          ImportParticipatorySpacePrivateUserCsvJob.perform_later(email, user_name, @private_users_to, @current_user) if email.present? && user_name.present?
         end
       end
     end

decidim-admin/app/commands/decidim/admin/process_user_group_verification_csv.rb

@@ -7,7 +7,7 @@ module Admin
     # A command with all the business logic when processing the CSV to verify
     # user groups.
     class ProcessUserGroupVerificationCsv < Decidim::Command
-      include Decidim::ProcessesFileLocally
+      include Decidim::Admin::CustomImport
 
       # Public: Initializes the command.
       #
@@ -35,11 +35,8 @@ def process_csv
         verifier = @form.current_user
         organization = @form.current_organization
 
-        process_file_locally(@form.file) do |file_path|
-          CSV.foreach(file_path) do |row|
-            email = row[0]
-            VerifyUserGroupFromCsvJob.perform_later(email, verifier, organization) if email.present?
-          end
+        process_import_file(@form.file) do |(email)|
+          VerifyUserGroupFromCsvJob.perform_later(email, verifier, organization) if email.present?
         end
       end
     end

decidim-admin/app/forms/decidim/admin/participatory_space_private_user_csv_import_form.rb

@@ -8,7 +8,7 @@ module Admin
     #
     class ParticipatorySpacePrivateUserCsvImportForm < Form
       include Decidim::HasUploadValidations
-      include Decidim::ProcessesFileLocally
+      include Decidim::Admin::CustomImport
 
       attribute :file, Decidim::Attributes::Blob
       attribute :user_name, String
@@ -20,11 +20,11 @@ class ParticipatorySpacePrivateUserCsvImportForm < Form
       def validate_csv
         return if file.blank?
 
-        process_file_locally(file) do |file_path|
-          CSV.foreach(file_path) do |_email, user_name|
-            errors.add(:user_name, :invalid) unless user_name.match?(UserBaseEntity::REGEXP_NAME)
-          end
+        process_import_file(file) do |(_email, user_name)|
+          errors.add(:user_name, :invalid) if user_name.blank? || !user_name.match?(UserBaseEntity::REGEXP_NAME)
         end
+      rescue CSV::MalformedCSVError
+        errors.add(:file, :malformed)
       end
     end
   end

decidim-admin/app/forms/decidim/admin/user_group_csv_verification_form.rb

@@ -6,10 +6,20 @@ module Admin
     #
     class UserGroupCsvVerificationForm < Form
       include Decidim::HasUploadValidations
+      include Decidim::ProcessesFileLocally
 
       attribute :file, Decidim::Attributes::Blob
 
       validates :file, presence: true, file_content_type: { allow: ["text/csv"] }
+      validate :validate_csv, unless: ->(f) { f.file.blank? }
+
+      def validate_csv
+        process_file_locally(file) do |file_path|
+          CSV.open(file_path, &:readline)
+        end
+      rescue CSV::MalformedCSVError
+        errors.add(:file, :malformed)
+      end
     end
   end
 end

decidim-admin/app/packs/stylesheets/decidim/admin/extra/_quill.scss

@@ -3,3 +3,10 @@
     background: #eee;
   }
 }
+
+.editor-container{
+  ul,
+  ol{
+    margin-bottom: 1rem;
+  }
+}

decidim-admin/app/packs/stylesheets/decidim/admin/modules/_forms.scss

@@ -111,11 +111,6 @@ label,
   }
 }
 
-.form-error{
-  margin-top: -$input-margin;
-  margin-bottom: $input-margin;
-}
-
 .form-input-extra-before{
   margin-bottom: $form-spacing * 1.5;
   display: block;

decidim-admin/app/views/decidim/admin/participatory_space_private_users_csv_imports/new.html.erb

@@ -25,7 +25,11 @@
   </div>
   <div class="card-section">
     <%= decidim_form_for(@form, url: participatory_space_private_users_csv_imports_path, html: { class: "form" }) do |form| %>
-      <p><%= t(".explanation") %></p>
+      <div class="mb-s">
+        <p><%= t(".explanation") %></p>
+        <p><%= t(".example_file") %></p>
+        <code class="code-block"><%= t(".explanation_example", csv_col_sep: Decidim.default_csv_col_sep) %></code>
+      </div>
       <div class="row column">
         <%= form.upload :file, optional: false %>
       </div>

decidim-admin/app/views/decidim/admin/user_groups_csv_verifications/new.html.erb

@@ -6,7 +6,11 @@
   </div>
   <div class="card-section">
     <%= decidim_form_for(@form, url: user_groups_csv_verification_path, html: { class: "form" }) do |form| %>
-      <p><%= t(".explanation") %></p>
+      <div class="mb-s">
+        <p><%= t(".explanation") %></p>
+        <p><%= t(".example_file") %></p>
+        <code class="code-block"><%= t(".explanation_example") %></code>
+      </div>
       <div class="row column">
         <%= form.upload :file, optional: false %>
       </div>