Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom feeds #1001

Merged
merged 91 commits into from
May 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
7941e1e
actor type in profiles
dholms May 8, 2023
b37bd06
skeleton lex
dholms May 8, 2023
9a05145
more schemas
dholms May 9, 2023
d97e79e
first pass at feed composition
dholms May 9, 2023
82c9e74
cleanup services
dholms May 9, 2023
04c5890
more tidying
dholms May 9, 2023
e007507
fix up tests
dholms May 9, 2023
a7b9e04
bookmarking schemas
dholms May 9, 2023
7af005c
make feeds records instead of actors
dholms May 10, 2023
3d472a2
some cleanup
dholms May 10, 2023
4aadec1
fleshing out feeds as records
dholms May 10, 2023
06914dc
wip
dholms May 10, 2023
de97b20
working into cfg & dev-env
dholms May 10, 2023
0f0efcb
merge service auth & fix build
dholms May 10, 2023
6edd506
fix appview test util
dholms May 10, 2023
a351aad
Merge branch 'service-auth-tidy' into custom-feeds
dholms May 10, 2023
8be7529
bookmarking + views
dholms May 11, 2023
7ec7096
tidy
dholms May 11, 2023
f23c20f
add displayName
dholms May 11, 2023
7805578
db migration
dholms May 11, 2023
ba726c5
wip test
dholms May 11, 2023
654727b
blob processing
dholms May 11, 2023
e73633b
first test working!
dholms May 11, 2023
4954cbf
merge main
dholms May 11, 2023
dfa1043
Sync-up snapshots
devinivy May 11, 2023
0fafa58
wip
dholms May 11, 2023
d18bb30
Fix pds ctx type
devinivy May 11, 2023
325f348
Sync lexicons on pds and api
devinivy May 11, 2023
e8ce004
wip
dholms May 11, 2023
028a6c3
fully refactor
dholms May 11, 2023
b4bf2d8
Prep for open union in reply data in bsky appview, sync-up snapshots
devinivy May 11, 2023
dd4eacd
msgs & ports
dholms May 11, 2023
f429135
Make pds did cache sqlite-friendly
devinivy May 11, 2023
445e71f
Start scaffolding more feedgen tests
devinivy May 11, 2023
7f8d747
revamp appview tests
dholms May 12, 2023
445fc5a
revamp pds tests
dholms May 12, 2023
64e88e6
merge in dev env utils
dholms May 12, 2023
c6abbbd
ensure plc close
dholms May 12, 2023
34f5c21
fix merge issues
dholms May 12, 2023
612f6b5
add feed gen to dev-env
dholms May 12, 2023
44219c0
second feed gen
dholms May 12, 2023
036fa17
named feeds in mock
dholms May 12, 2023
c7a1fe3
fix issue with dev-env pds appview
dholms May 12, 2023
c0e52c4
update bsky snaps
dholms May 12, 2023
2e03ab9
Flesh-out some feed gen tests
devinivy May 12, 2023
ab4174c
Fix feed gen reply test
devinivy May 12, 2023
a2f6e3d
Test feed bookmarking
devinivy May 12, 2023
8201fbe
update skeleton defs
dholms May 12, 2023
97a324a
merge
dholms May 12, 2023
4a8dd88
Sync-up pds with updated feed skeleton contract
devinivy May 12, 2023
2fb41d6
Fix feed bookmark viewer state
devinivy May 12, 2023
f99cb85
add feed embeds
dholms May 12, 2023
69cb590
Test inaccurate feed gen repost
devinivy May 12, 2023
dbcaaa2
Tighten some error behaviors around feed gens
devinivy May 12, 2023
9d09b43
generator view in record embed
dholms May 12, 2023
885d165
test
dholms May 12, 2023
66e1dc2
bookmarked/subscribed -> saved
dholms May 12, 2023
e9ba4a7
updated snaps
dholms May 12, 2023
bcf66ea
name & descript on alices feed
dholms May 13, 2023
2bdf2c2
merge in main
dholms May 15, 2023
f39ff15
add like counts
dholms May 15, 2023
0d9c999
proposed lex
dholms May 15, 2023
dd7caf3
tweak
dholms May 15, 2023
65a24f4
fetch feed generator details
dholms May 15, 2023
e7e5695
add describeFeedGen to dev-env
dholms May 15, 2023
2729267
adds tests for getFeedGenerator
dholms May 15, 2023
c75c5c7
Update lexicons/app/bsky/feed/getFeedGenerator.json
dholms May 15, 2023
49d57de
update describe
dholms May 15, 2023
ae17f10
add cid to feed gen view
dholms May 15, 2023
f95d13d
add alice feed gen avatar
dholms May 17, 2023
695b4c7
generator displayName required
dholms May 18, 2023
af07541
update snaps
dholms May 18, 2023
8134b5c
Merge branch 'main' into custom-feeds
dholms May 18, 2023
900e91d
Fix dev env public url (#1058)
pfrazee May 18, 2023
73d766b
add defs for feed gen prefs & decrease displayName length
dholms May 18, 2023
87749b3
build branch
dholms May 18, 2023
df934b7
updated how saved feeds works
dholms May 18, 2023
ef52c70
cleanup lex
dholms May 18, 2023
1bfd214
mount route
dholms May 18, 2023
82047b3
Starter bsky algos (#1046)
dholms May 18, 2023
45c629b
v0.3.6
dholms May 18, 2023
eea4bdf
Merge branch 'main' into custom-feeds
dholms May 18, 2023
7fc4da6
v0.3.7
dholms May 18, 2023
3cb9a5d
v0.1.10
dholms May 18, 2023
4102fef
update name of skyline
dholms May 18, 2023
47c7cd0
build branch
dholms May 18, 2023
c8481b9
aws build
dholms May 19, 2023
d1fce54
getFeedGenerators test
dholms May 19, 2023
21867d2
For now only use new replyRef union for custom feeds, for compat
devinivy May 19, 2023
12431b1
Don't use replyRef union on new feeds yet
devinivy May 19, 2023
d697ba6
Merge remote-tracking branch 'origin/main' into custom-feeds
devinivy May 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build-and-push-pds-aws.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ on:
push:
branches:
- main
- custom-feeds
env:
REGISTRY: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_REGISTRY }}
USERNAME: ${{ secrets.AWS_ECR_REGISTRY_USEAST2_PACKAGES_USERNAME }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/build-and-push-pds-ghcr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ on:
push:
branches:
- main
- custom-feeds
env:
REGISTRY: ghcr.io
USERNAME: ${{ github.actor }}
Expand Down
23 changes: 22 additions & 1 deletion lexicons/app/bsky/actor/defs.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,8 @@
"type": "union",
"refs": [
"#adultContentPref",
"#contentLabelPref"
"#contentLabelPref",
"#savedFeedsPref"
]
}
},
Expand All @@ -111,6 +112,26 @@
"label": {"type": "string"},
"visibility": {"type": "string", "knownValues": ["show", "warn", "hide"]}
}
},
"savedFeedsPref": {
"type": "object",
"required": ["pinned", "saved"],
"properties": {
"pinned": {
"type": "array",
"items": {
"type": "string",
"format": "at-uri"
}
},
"saved": {
"type": "array",
"items": {
"type": "string",
"format": "at-uri"
}
}
}
}
}
}
10 changes: 9 additions & 1 deletion lexicons/app/bsky/embed/record.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,15 @@
"type": "object",
"required": ["record"],
"properties": {
"record": {"type": "union", "refs": ["#viewRecord", "#viewNotFound", "#viewBlocked"]}
"record": {
"type": "union",
"refs": [
"#viewRecord",
"#viewNotFound",
"#viewBlocked",
"app.bsky.feed.defs#generatorView"
]
}
}
},
"viewRecord": {
Expand Down
47 changes: 44 additions & 3 deletions lexicons/app/bsky/feed/defs.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"type": "object",
"required": ["post"],
"properties": {
"post": {"type": "ref", "ref": "app.bsky.feed.defs#postView"},
"post": {"type": "ref", "ref": "#postView"},
"reply": {"type": "ref", "ref": "#replyRef"},
"reason": {"type": "union", "refs": ["#reasonRepost"]}
}
Expand All @@ -50,8 +50,8 @@
"type": "object",
"required": ["root", "parent"],
"properties": {
"root": {"type": "ref", "ref": "app.bsky.feed.defs#postView"},
"parent": {"type": "ref", "ref": "app.bsky.feed.defs#postView"}
"root": {"type": "union", "refs": ["#postView", "#notFoundPost", "#blockedPost"]},
"parent": {"type": "union", "refs": ["#postView", "#notFoundPost", "#blockedPost"]}
}
},
"reasonRepost": {
Expand Down Expand Up @@ -89,6 +89,47 @@
"uri": {"type": "string", "format": "at-uri"},
"blocked": {"type": "boolean", "const": true}
}
},
"generatorView": {
"type": "object",
"required": ["uri", "cid", "creator", "displayName", "indexedAt"],
"properties": {
"uri": {"type": "string", "format": "at-uri"},
"cid": {"type": "string", "format": "cid"},
"did": {"type": "string", "format": "did"},
"creator": {"type": "ref", "ref": "app.bsky.actor.defs#profileView"},
"displayName": {"type": "string"},
"description": {"type": "string", "maxGraphemes": 300, "maxLength": 3000},
"descriptionFacets": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.richtext.facet"}
},
"avatar": {"type": "string"},
"likeCount": {"type": "integer", "minimum": 0 },
"viewer": {"type": "ref", "ref": "#generatorViewerState"},
"indexedAt": {"type": "string", "format": "datetime"}
}
},
"generatorViewerState": {
"type": "object",
"properties": {
"like": {"type": "string", "format": "at-uri"}
}
},
"skeletonFeedPost": {
"type": "object",
"required": ["post"],
"properties": {
"post": {"type": "string", "format": "at-uri"},
"reason": {"type": "union", "refs": ["#skeletonReasonRepost"]}
}
},
"skeletonReasonRepost": {
"type": "object",
"required": ["repost"],
"properties": {
"repost": {"type": "string", "ref": "at-uri"}
}
}
}
}
39 changes: 39 additions & 0 deletions lexicons/app/bsky/feed/describeFeedGenerator.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"lexicon": 1,
"id": "app.bsky.feed.describeFeedGenerator",
"defs": {
"main": {
"type": "query",
"description": "Returns information about a given feed generator including TOS & offered feed URIs",
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["did", "feeds"],
"properties": {
"did": {"type": "string", "format": "did"},
"feeds": {
"type": "array",
"items": {"type": "ref", "ref": "#feed"}
},
"links": {"type": "ref", "ref": "#links"}
}
}
}
},
"feed": {
"type": "object",
"required": ["uri"],
"properties": {
"uri": {"type": "string", "format": "at-uri"}
}
},
"links": {
"type": "object",
"properties": {
"privacyPolicy": {"type": "string"},
"termsOfService": {"type": "string"}
}
}
}
}
30 changes: 30 additions & 0 deletions lexicons/app/bsky/feed/generator.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"lexicon": 1,
"id": "app.bsky.feed.generator",
"defs": {
"main": {
"type": "record",
"description": "A declaration of the existence of a feed generator",
"key": "any",
"record": {
"type": "object",
"required": ["did", "displayName", "createdAt"],
"properties": {
"did": {"type": "string", "format": "did"},
"displayName": {"type": "string", "maxGraphemes": 24, "maxLength": 240},
"description": {"type": "string", "maxGraphemes": 300, "maxLength": 3000},
"descriptionFacets": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.richtext.facet"}
},
"avatar": {
"type": "blob",
"accept": ["image/png", "image/jpeg"],
"maxSize": 1000000
},
"createdAt": {"type": "string", "format": "datetime"}
}
}
}
}
}
33 changes: 33 additions & 0 deletions lexicons/app/bsky/feed/getActorFeeds.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"lexicon": 1,
"id": "app.bsky.feed.getActorFeeds",
"defs": {
"main": {
"type": "query",
"description": "Retrieve a list of feeds created by a given actor",
"parameters": {
"type": "params",
"required": ["actor"],
"properties": {
"actor": {"type": "string", "format": "at-identifier"},
"limit": {"type": "integer", "minimum": 1, "maximum": 100, "default": 50},
"cursor": {"type": "string"}
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["feeds"],
"properties": {
"cursor": {"type": "string"},
"feeds": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.feed.defs#generatorView"}
}
}
}
}
}
}
}
36 changes: 36 additions & 0 deletions lexicons/app/bsky/feed/getFeed.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"lexicon": 1,
"id": "app.bsky.feed.getFeed",
"defs": {
"main": {
"type": "query",
"description": "Compose and hydrate a feed from a user's selected feed generator",
"parameters": {
"type": "params",
"required": ["feed"],
"properties": {
"feed": {"type": "string", "format": "at-uri"},
"limit": {"type": "integer", "minimum": 1, "maximum": 100, "default": 50},
"cursor": {"type": "string"}
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["feed"],
"properties": {
"cursor": {"type": "string"},
"feed": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.feed.defs#feedViewPost"}
}
}
}
},
"errors": [
{"name": "UnknownFeed"}
]
}
}
}
29 changes: 29 additions & 0 deletions lexicons/app/bsky/feed/getFeedGenerator.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"lexicon": 1,
"id": "app.bsky.feed.getFeedGenerator",
"defs": {
"main": {
"type": "query",
"description": "Get information about a specific feed offered by a feed generator, such as its online status",
"parameters": {
"type": "params",
"required": ["feed"],
"properties": {
"feed": {"type": "string", "format": "at-uri"}
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["view", "isOnline", "isValid"],
"properties": {
"view": {"type": "ref", "ref": "app.bsky.feed.defs#generatorView"},
"isOnline": {"type": "boolean"},
"isValid": {"type": "boolean"}
Comment on lines +22 to +23
Copy link
Collaborator

@devinivy devinivy May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general we haven't been prefixing booleans with is (e.g. rebase, tooBig, muted, blockedBy, disabled, resolved). There is one place we do use it, though, which is on notifications isRead.

Copy link
Collaborator

@devinivy devinivy May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, what does isValid check for versus isOnline? Mostly wondering if both are satisfied by simply attempting to fetch the feed, or if one/both require a call to describeFeedGenerator.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm down to strip out the is 👌

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as for the second part of this, I basically do the check against describeFeedGenerator, but maybe I should just be doing a sample call 🤔

I have the code up for this now too if you want a look but basically:

  • isOnline is "does the service respond at all"
  • isValid is "is the did in the record correct & does this service declare that they support the given feed uri?"

}
}
}
}
}
}
33 changes: 33 additions & 0 deletions lexicons/app/bsky/feed/getFeedGenerators.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"lexicon": 1,
"id": "app.bsky.feed.getFeedGenerators",
"defs": {
"main": {
"type": "query",
"description": "Get information about a list of feed generators",
"parameters": {
"type": "params",
"required": ["feeds"],
"properties": {
"feeds": {
"type": "array",
"items": {"type": "string", "format": "at-uri"}
}
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["feeds"],
"properties": {
"feeds": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.feed.defs#generatorView"}
}
}
}
}
}
}
}
36 changes: 36 additions & 0 deletions lexicons/app/bsky/feed/getFeedSkeleton.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"lexicon": 1,
"id": "app.bsky.feed.getFeedSkeleton",
"defs": {
"main": {
"type": "query",
"description": "A skeleton of a feed provided by a feed generator",
"parameters": {
"type": "params",
"required": ["feed"],
"properties": {
"feed": {"type": "string", "format": "at-uri"},
"limit": {"type": "integer", "minimum": 1, "maximum": 100, "default": 50},
"cursor": {"type": "string"}
}
},
"output": {
"encoding": "application/json",
"schema": {
"type": "object",
"required": ["feed"],
"properties": {
"cursor": {"type": "string"},
"feed": {
"type": "array",
"items": {"type": "ref", "ref": "app.bsky.feed.defs#skeletonFeedPost"}
}
}
}
},
"errors": [
{"name": "UnknownFeed"}
]
}
}
}
2 changes: 1 addition & 1 deletion packages/api/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@atproto/api",
"version": "0.3.4",
"version": "0.3.7",
"main": "src/index.ts",
"scripts": {
"codegen": "lex gen-api ./src/client ../../lexicons/com/atproto/*/* ../../lexicons/app/bsky/*/*",
Expand Down
Loading