Lists #5703

Gargron · 2017-11-15T00:09:17Z

Fix #134

Database structure for storing lists
Adjust FeedManager to handle list timelines
Add lists to streaming API
API for managing lists

Web UI implementation should come in a different PR.

New APIs:

GET /api/v1/timelines/list/:id Timeline of a list
GET /api/v1/lists All your lists
POST /api/v1/lists Create a new list (allowed param: title)
GET/PUT/DELETE /api/v1/lists/:id
GET /api/v1/lists/:id/accounts All accounts in the list
POST/DELETE /api/v1/lists/:id/accounts Add or remove accounts to/from the list (array param: account_ids)

New streaming API:

/api/v1/streaming/list?list=:id Subscribe to list timeline

Other information:

Lists are private to their creator
Only people you follow can be added to your lists
If a follow stops existing, their list membership also stops existing

nightpool

needs an update to remove_status_service_spec for Feed -> HomeFeed.

needs a corresponding /documentation PR

nightpool · 2017-11-16T19:38:12Z

spec/controllers/api/v1/lists/accounts_controller_spec.rb

+  render_views
+
+  let(:user)  { Fabricate(:user, account: Fabricate(:account, username: 'alice')) }
+  let(:token) { Fabricate(:accessible_access_token, resource_owner_id: user.id, scopes: 'follow') }


wrong scopes

nightpool · 2017-11-16T19:40:40Z

app/models/list_feed.rb

+  def initialize(list)
+    @type    = :list
+    @id      = list.id
+  end


No from_database for lists?

Same question from me.

I'm fine punting on this for now.

nightpool · 2017-11-16T19:44:53Z

app/policies/list_policy.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class ListPolicy < ApplicationPolicy


Is this ever used by Doorkeeper? I couldn't find any indication that doorkeeper checked pundit policies....

nightpool · 2017-11-16T19:48:54Z

app/lib/feed_manager.rb

@@ -27,33 +27,41 @@ def filter?(timeline_type, status, receiver_id)
  end

  def push(timeline_type, account, status)


Considering that we only ever use push(:home) (as far as I can tell....) and we're adding a push_to_list anyway, we should probably make a push_to_home method, and then either get rid of push or make it private and get rid of any code duplication between it and push_to_list

(same with unpush)

cwebber · 2017-11-16T21:19:39Z

Any thought of exposing these as AS2 Collections?

- Creating and modifying lists merely requires "write" scope - Fetching information about lists merely requires "read" scope

nightpool · 2017-11-17T02:39:19Z

@cwebber right now lists are entirely private and no one else can see them. If we add other functionality then we might consider that

aschmitz

This looks reasonably good overall. The bulk endpoint is apparently new, so it would be good to make sure it's appropriately documented. A few minor comments for possible changes, but I don't know that any of them are really necessary.

aschmitz · 2017-11-17T02:38:58Z

app/services/fan_out_on_write_service.rb

@@ -30,15 +31,25 @@ def call(status)

  def deliver_to_self(status)
    Rails.logger.debug "Delivering status #{status.id} to author"
-    FeedManager.instance.push(:home, status.account, status)
+    FeedManager.instance.push_to_home(status.account, status)


What if you're in one of your own lists?

You can't follow yourself so you can't add yourself to a list.

aschmitz · 2017-11-17T02:40:02Z

app/lib/feed_manager.rb

-
-    PushUpdateWorker.perform_async(account.id, status.id) if push_update_required?(timeline_type, account.id)
-
+  def push_to_home(account, status)


Is there a good reason to call push_to_home and push_to_list separately outside of FeedManager? Should we just expose push_to_user and let FeedManager sort out lists? (This seems less efficient, but perhaps cleaner. I can see arguments on either side.)

As I started writing the code, the push behaviour was different enough that I had to create a separate push_to_list method. But as I refactored, they turned out pretty similar after all. However, I am absolutely OK with keeping them separate for now. As they say, the DRY principle kicks in for n >= 3

My thought was that it mirrors push_to_hashtag, which isn't in feed_manager but does need to happen

Huh? No, hashtags timelines are not stored in redis

Yes, but it mirrors the necessity of calling all of those different functions (see the calling site)

aschmitz · 2017-11-17T02:47:36Z

spec/controllers/api/v1/timelines/list_controller_spec.rb

+    allow(controller).to receive(:doorkeeper_token) { token }
+  end
+
+  context 'with a user context' do


It may be worth testing with a different user's context as well, to ensure the list isn't accidentally exposed.

aschmitz · 2017-11-17T02:49:06Z

spec/models/list_account_spec.rb

+require 'rails_helper'
+
+RSpec.describe ListAccount, type: :model do
+  pending "add some examples to (or delete) #{__FILE__}"


I'd either add tests or drop this (and list_spec.rb).

ClearlyClaire

I'm confused about the bigint VS integer things and I wonder about not having a method to load lists from database when Redis cache doesn't exist

ClearlyClaire · 2017-11-17T20:39:25Z

app/models/account.rb

@@ -3,7 +3,7 @@
 #
 # Table name: accounts
 #
-#  id                      :bigint           not null, primary key
+#  id                      :integer          not null, primary key
 #  username                :string           default(""), not null


Huh? Why would we go back to integer?

These model annotations are auto-generated. I've deleted the rest of your comments because they're not a concern.

Right. Still, noisy for no reason, and I don't understand why they would be generated that way? db/schema.rb seems fine, though.

see the conversation on #5461

Every db:migrate is gonna reset these to this. So I'd rather have them wrong than deal with noise in every PR. I shouldn't have merged #5461 but I did not realize annotate would not pick up the correct types, thought something just got forgotten because of the unusual way we performed the snowflake migrations.

ClearlyClaire · 2017-11-17T20:44:15Z

app/models/list_feed.rb

+  def initialize(list)
+    @type    = :list
+    @id      = list.id
+  end


Same question from me.

nightpool · 2017-11-17T21:21:30Z

Still needs remove_status_service_spec update.

Merge upstream (mastodon#5703)

* Add structure for lists * Add list timeline streaming API * Add list APIs, bind list-account relation to follow relation * Add API for adding/removing accounts from lists * Add pagination to lists API * Add pagination to list accounts API * Adjust scopes for new APIs - Creating and modifying lists merely requires "write" scope - Fetching information about lists merely requires "read" scope * Add test for wrong user context on list timeline * Clean up tests

aquariusdev · 2018-04-18T15:24:32Z

Can you add the ability to add/remove users to/from lists on their profile?

Mockup 1, borrowed a bit from Diaspora:

Mockup 2, using existing menus on Mastodon:

Mockup 3, using Twitter's Add to/Remove from Lists pop-up:

Cassolotl · 2018-04-18T15:47:03Z

@aquariusdev Since this is a pull request that was merged a while ago, it's probably best to create a new issue for your feature request!

Gargron added api REST API, Streaming API, Web Push API work in progress Not to be merged, currently being worked on labels Nov 15, 2017

Gargron force-pushed the feature-lists branch 2 times, most recently from 92d2690 to 305ee4c Compare November 16, 2017 17:08

Gargron added 3 commits November 16, 2017 18:20

Add structure for lists

e2a8acc

Add list timeline streaming API

22a77ff

Add list APIs, bind list-account relation to follow relation

112920a

Gargron force-pushed the feature-lists branch 2 times, most recently from f953418 to 983a648 Compare November 16, 2017 17:50

Gargron added 2 commits November 16, 2017 19:40

Add API for adding/removing accounts from lists

214966a

Add pagination to lists API

31bceab

Gargron force-pushed the feature-lists branch from 983a648 to 31bceab Compare November 16, 2017 18:53

Add pagination to list accounts API

a0e7c80

Gargron removed the work in progress Not to be merged, currently being worked on label Nov 16, 2017

nightpool requested changes Nov 16, 2017

View reviewed changes

nightpool reviewed Nov 16, 2017

View reviewed changes

nightpool requested review from nightpool and removed request for nightpool November 16, 2017 20:13

Adjust scopes for new APIs

95a4dfa

- Creating and modifying lists merely requires "write" scope - Fetching information about lists merely requires "read" scope

Gargron force-pushed the feature-lists branch from a01d3fb to 95a4dfa Compare November 16, 2017 23:16

aschmitz reviewed Nov 17, 2017

View reviewed changes

Add test for wrong user context on list timeline

3c616de

ClearlyClaire self-requested a review November 17, 2017 19:22

ClearlyClaire reviewed Nov 17, 2017

View reviewed changes

mastodon deleted a comment from ClearlyClaire Nov 17, 2017

Clean up tests

93857ff

nightpool approved these changes Nov 17, 2017

View reviewed changes

Gargron merged commit 24cafd7 into master Nov 17, 2017

Gargron deleted the feature-lists branch November 17, 2017 23:51

hannahwhy added a commit to glitch-soc/mastodon that referenced this pull request Nov 17, 2017

Merge pull request #224 from yipdw/merge-upstream

284e2cd

Merge upstream (mastodon#5703)

rinsuki mentioned this pull request Nov 22, 2017

Support Lists cinderella-project/iMast#72

Merged

Gargron mentioned this pull request Nov 27, 2017

Bump version to 2.1.0rc1 #5834

Merged

10 tasks

remram44 mentioned this pull request Dec 4, 2017

View content from a list of users #952

Closed

This was referenced Dec 5, 2017

Add list of lists component to web UI #5811

Merged

List timeline UI #5750

Merged

aquariusdev mentioned this pull request Apr 18, 2018

feature outline: Lists glitch-soc/mastodon#12

Open

aquariusdev mentioned this pull request Apr 18, 2018

"Add or remove from lists" option + multi-choice list selection on profile #7187

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lists #5703

Lists #5703

Gargron commented Nov 15, 2017 •

edited

nightpool left a comment

nightpool Nov 16, 2017

nightpool Nov 16, 2017

ClearlyClaire Nov 17, 2017

nightpool Nov 17, 2017

nightpool Nov 16, 2017

nightpool Nov 16, 2017 •

edited

cwebber commented Nov 16, 2017

nightpool commented Nov 17, 2017

aschmitz left a comment

aschmitz Nov 17, 2017

Gargron Nov 17, 2017

aschmitz Nov 17, 2017

Gargron Nov 17, 2017

nightpool Nov 17, 2017

Gargron Nov 17, 2017

nightpool Nov 17, 2017

aschmitz Nov 17, 2017

aschmitz Nov 17, 2017

ClearlyClaire left a comment

ClearlyClaire Nov 17, 2017

nightpool Nov 17, 2017

ClearlyClaire Nov 17, 2017

nightpool Nov 17, 2017

Gargron Nov 17, 2017

ClearlyClaire Nov 17, 2017

nightpool commented Nov 17, 2017

aquariusdev commented Apr 18, 2018 •

edited

Cassolotl commented Apr 18, 2018 •

edited

		@@ -0,0 +1,13 @@
		# frozen_string_literal: true

		class ListPolicy < ApplicationPolicy

		@@ -27,33 +27,41 @@ def filter?(timeline_type, status, receiver_id)
		end

		def push(timeline_type, account, status)


		PushUpdateWorker.perform_async(account.id, status.id) if push_update_required?(timeline_type, account.id)

		def push_to_home(account, status)

Lists #5703

Lists #5703

Conversation

Gargron commented Nov 15, 2017 • edited

nightpool left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nightpool Nov 16, 2017 • edited

Choose a reason for hiding this comment

cwebber commented Nov 16, 2017

nightpool commented Nov 17, 2017

aschmitz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ClearlyClaire left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nightpool commented Nov 17, 2017

aquariusdev commented Apr 18, 2018 • edited

Cassolotl commented Apr 18, 2018 • edited

Gargron commented Nov 15, 2017 •

edited

nightpool Nov 16, 2017 •

edited

aquariusdev commented Apr 18, 2018 •

edited

Cassolotl commented Apr 18, 2018 •

edited