Skip to content

Commit

Permalink
feat: make Listings iterable
Browse files Browse the repository at this point in the history 
BREAKING CHANGE: This renames Listing.each to Listing.forEach, and
it is now no longer the preferred method of iteration. Instead you
should probably use `for await` loops.
  • Loading branch information
@thislooksfun
thislooksfun committed Apr 13, 2021
1 parent 51642ca commit 868d58d
Show file tree
Hide file tree
Showing 2 changed files with 104 additions and 71 deletions.
68 changes: 17 additions & 51 deletions dochome.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,12 @@ without an application means you have a _much_ lower ratelimit.
Print a comment tree to stdout.

```ts
async function printTree(c: Comment, indent: string = "") {
const body = c.body.replace(/\n/g, "\n" + indent);
console.log(`${indent}(${c.id}) ${c.author}: ${body}`);
await c.replies.each(r => printTree(r, indent + " "));
async function printTree(cmt: Comment, indent: string = "") {
const body = cmt.body.replace(/\n/g, "\n" + indent);
console.log(`${indent}(${cmt.id}) ${cmt.author}: ${body}`);
for await (const reply of cmt.replies) {
await printTree(reply, index + " ");
}
}

(async () => {
Expand All @@ -86,8 +88,7 @@ the largest:
1. Objects are not lazy loaded.
1. Promise chaining properties is not allowed.
1. All parameters are camelCase, not snake_case.
1. Listings are not arrays. Instead use [.each()][l-each] to iterate through a
listing.
1. Listings are not arrays, but they can be iterated using `for await`.
1. Sub-objects are not auto-populated (like `Post` and `Comment`'s `author`). In
snoowrap the `author` field is a user object, but in snoots it's just a
username.
Expand All @@ -111,57 +112,22 @@ const title = post.title;
```ts
// snoowrap
const posts = await client.getSubreddit("funny").getNew().fetchAll();
posts.map(p => console.log(p.author.name));
for (const post of posts) {
console.log(post.author.name);
}

// snoots (literal translation)
const sub = await client.subreddits.fetch("funny");
const posts = sub.getNewPosts();
await posts.each(p => console.log(p.author));
for await (const post of posts) {
console.log(p.author);
}

// snoots (preferred method, 1 fewer api call)
const posts = client.subreddits.getNewPosts("funny");
await posts.each(p => console.log(p.author));
```

### Listings

[Listings][listing] in snoots are _not_ iterables. This is by design. Reddit's
listings come in many forms, ranging from fully-populated arrays to completely
empty lists with instructions on where to look for more items. Trying to expose
this as an array or other iterable leaves only two options: fetch the entirety
of every listing before giving it back to you, or make you responsible for
fetching more. Both solutions suck. In the former case you are forced to deal
with dozens or even hundreds of extra api calls that eat up your ratelimit, and
in the latter it's really easy to forget to fetch more and make your code run
perfectly but skip most of the items.

Rather than deal with all this mess, snoots' Listing class is an opqaue wrapper
around the implementation details of the actual data population, instead only
exposing clean, easy to understand methods to interact with the data as a whole.
The main way you'll likely interact with a Listing is via the [.each()][l-each]
method. If you want to run your logic on a whole page at a time you can use
[.eachPage()][l-eachpage]. Both of these take in functions that allow for early
breaking of the loop. If the callback returns or resolves to (returns a Promise
that then becomes) `false`, the iteration will be stopped and no more fetching
will occur. For example, to print out all the comments on a post until one of
them is too old, you can use the following:

```ts
const post = await client.posts.fetch("<post id>");
const timestamp = some old timestamp;
await post.comments.each(c => {
console.log(c.body);
return c.createdUtc > timestamp;
});
```

There are times when you only need to know if _something_ matches some criteria
in a Listing. For that we have [.some()][l-some]. This behaves just like the
array method of the same name, except that it is fully asynchronous.

```ts
const post = await client.posts.fetch("<post id>");
const autoModDidComment = await post.comments.some(c => c.author === "AutoModerator"));
for await (const post of posts) {
console.log(p.author);
}
```

<!-- Links -->
Expand All @@ -170,6 +136,6 @@ const autoModDidComment = await post.comments.some(c => c.author === "AutoModera
[ua]: ./interfaces/clientoptions.html#useragent
[creds]: ./interfaces/credentials.html
[listing]: https://thislooks.fun/snoots/docs/latest/classes/listing.html
[l-each]: https://thislooks.fun/snoots/docs/latest/classes/listing.html#each
[l-foreach]: https://thislooks.fun/snoots/docs/latest/classes/listing.html#foreach
[l-eachpage]: https://thislooks.fun/snoots/docs/latest/classes/listing.html#eachpage
[l-some]: https://thislooks.fun/snoots/docs/latest/classes/listing.html#some
107 changes: 87 additions & 20 deletions src/listings/listing.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,27 @@ export abstract class Pager<T> implements Fetcher<T> {
* intentionally exposes as little as possible about the internal workings to
* minimize the amount of boilerplate needed to interact with them.
*
* @note Since Reddit's responses are paged, Listings only implement asyncInterator.
* This means that if you want to loop over it you have to use `for await`:
* ```ts
* const posts = await client.subreddits.getNewPosts("funny");
* for await (const post of posts) {
* console.log(post.title);
* }
* ```
*
* @note If you want to iterate using callbacks, you can use {@link forEach}:
* ```ts
* const posts = await client.subreddits.getNewPosts("funny");
* await posts.forEach(post => console.log(post.title));
* ```
*
* @note If you want to get an entire page at a time, use {@link eachPage}:
* ```ts
* const posts = await client.subreddits.getNewPosts("funny");
* await posts.eachPage(page => console.log(page.length));
* ```
*
* @template T The type of items this Listing holds.
*/
export default class Listing<T> {
Expand Down Expand Up @@ -129,33 +150,46 @@ export default class Listing<T> {
const res = await fn(page.arr);
if (res === false) return;

if (page.next) {
page = page.next;
} else if (page.fetcher) {
page = await page.fetcher.fetch(this.ctx);
} else {
page = null;
}
page = await Listing.nextPage(page, this.ctx);
} while (page != null);
}

/**
* Execute a function on each element of the listing.
*
* @note This is an enhanced version of the default Array.forEach. It allows
* for asynchronous callbacks and breaking.
*
* @example Async
* ```ts
* async function slowAsync(post: Post): Promise<void> {
* // do something slow
* }
*
* const posts = await client.subreddits.getNewPosts("funny");
* await posts.forEach(post => slowAsync(post));
* ```
*
* @example Breaking
* ```ts
* const posts = await client.subreddits.getNewPosts("funny");
* await posts.forEach(post => {
* console.log(post.title);
* // Break if the post was more than 5 minutes old.
* return post.createdUtc >= Date.now() - 5 * 60 * 1000;
* });
* ```
*
* @param fn The function to execute. If this returns or resolves to `false`
* the execution will be halted.
*
* @returns A promise that resolves when the listing has been exausted.
* @returns A promise that resolves when the iteration is complete.
*/
async each(fn: AwaitableFn<T, boolean | void>): Promise<void> {
await this.eachPage(async page => {
for (const el of page) {
// If the function returns false at any point, we are done.
const res = await fn(el);
if (res === false) return false;
}
return true;
});
async forEach(fn: AwaitableFn<T, boolean | void>): Promise<void> {
for await (const el of this) {
const res = await fn(el);
if (res === false) break;
}
}

/**
Expand All @@ -169,8 +203,41 @@ export default class Listing<T> {
* element in the listing, or `false` if it reached the end of the listing.
*/
async some(fn: AwaitableFn<T, boolean>): Promise<boolean> {
let found = false;
await this.each(async el => !(found = await fn(el)));
return found;
for await (const el of this) {
if (await fn(el)) return true;
}
return false;
}

private static async nextPage<T>(
page: Listing<T>,
ctx: Context
): Promise<Listing<T> | null> {
if (page.next) {
return page.next;
} else if (page.fetcher) {
return await page.fetcher.fetch(ctx);
} else {
return null;
}
}

[Symbol.asyncIterator]() {
return {
page: this as Listing<T>,
ctx: this.ctx,
index: 0,

async next(): Promise<IteratorResult<T>> {
if (this.index >= this.page.arr.length) {
const nextPage = await Listing.nextPage(this.page, this.ctx);
if (!nextPage) return { done: true, value: undefined };
this.page = nextPage;
this.index = 0;
}

return { done: false, value: this.page.arr[this.index++] };
},
};
}
}

0 comments on commit 868d58d

Please sign in to comment.