Option to auto download oldest episodes first

[ykneisel]

Auto download always downloads newest episodes first. So, if the episode cache size is smaller than the number of unplayed items, older episodes are never downloaded.
Another problem is that if there are multiple new episodes from the same feed, the newer episodes end up at the top of the queue, while the other way around would make more sense.
Of course I can manually reorder the queue, but it would be nice if the order of new episodes was retained automatically.

[mfietz]

So, if the episode cache size is smaller than the number of unplayed items, older episodes are never downloaded.

So, if there is an option to download the oldest episode first, wouldn't that mean that if the number of old episodes is larger than the cache size that new episodes are never downloaded? ^^

the newer episodes end up at the top of the queue

You probably have "Enqueue at the front" enabled. See the preferences, under the Queue category

[ykneisel]

So, if there is an option to download the oldest episode first,
wouldn't that mean that if the number of old episodes is larger than
the cache size that new episodes are never downloaded? ^^

Of course they would eventually be downloaded. Each time an episode is
removed from the queue a new episode is downloaded, but currently this
will always be the newest one. And since new episodes are published all
the time, those will always be downloaded first.

You probably have "Enqueue at the front" enabled. See the preferences,
under the /Queue/ category

No, the option "Enqueue at front" is unchecked. See the following example:

Queue:
a1
a2
New episodes:
b2
b1
Episode cache:
3

b1 and b2 are episodes from the same feed where b2 is newer than b1.
Now, auto download will download b2 first since it is newer.
Queue will then be:
a1
a2
b2

Next, a1 is removed and the next episode will be downloaded.
Queue:
a2
b2
b1

See what I mean? Unless I change the order of b1 and b2 manually, b2
will be played before b1, which is annoying if the chronology of
individual episodes is important.

[mfietz]

Are you really sure you would want AP to always download the oldest episode first? In my experience, for almost all podcasts you would want AP to download the newest episode first. Think Daily Show/Colbert Report (newest first) vs Breaking Bad/Game of Thrones (you wouldn't want to start in season 3, would you?).

So, for me it would make sense to have a feed setting what tells AP the correct order of the episodes, e.g. the feed overview would show the oldest episode on top.
But I have no idea yet how the auto download algorithm would handle such a mixed environment where you would want it to download the newest episodes first on some of the podcasts and the oldest ones first on others.
Easiest solution: Just press the download button manually for these unusual podcasts ^^

@TomHennen any good ideas?

[TomHennen]

I can see the utility in something like this, but as @mfietz points out, you'd really want it to be feed specific. Unfortunately the auto download algorithm just gets all of the new episodes from all feeds ordered by date, descending. BUT, before it does that, it downloads everything you have in the queue and it downloads those things in the order specified in the queue.

I can't think of a rational way to do some feeds descending and others ascending in a single list for autodownload.

So, I feel like the solution is to add the things you want downloaded in a specific order to the queue in that order. For feeds where you want to do this in order the 1.3 release (currently in beta) has the gear widget thingy, which will let you sort all the episodes from that feed in many different ways (including by date ascending) and then add them to the queue in that order.

Would that do what you want?

[ykneisel]

I see... well out of the 53 podcast feeds I have added to AP, there is
not a single one for which I would NOT want the oldest (unplayed)
episode to always be downloaded first (admittedly, only one of them is
published daily).

Unfortunately the auto download algorithm just gets all of the new
episodes from all feeds ordered by date, descending.

For me, an option to order new episodes by date, ascending instead would
be perfect. But from what you're saying I suspect that I may not be the
typical user and this option may therefore not be useful to anyone else.

the gear widget thingy, which will let you sort all the episodes from
that feed in many different ways (including by date ascending) and
then add them to the queue in that order.

Would that do what you want?

Not really since this would mean having to do this manually and
regularly for all the feeds - instead of having it done automatically
with auto download.

[ChaoticMind]

for what it's worth, i'm on the @ykneisel camp.

[corecode]

Downloading oldest new episodes first is least likely to surprise the user. Consider two related scenarios:

1. Device is permanently connected and auto downloads are possible. User listens to queue as podcasts appear, as soon as possible. In this scenario no podcast has multiple "new" episodes.
2. Device is occasionally connected. User listens to queue in bursts (some days/weeks less time to listen to podcasts, other days/weeks more time). In this scenario, new episodes occasionally accumulate (because queue is full); it is possible that there are multiple new episodes for a single podcast.

Ideally, the user should observe the same sequence of podcasts in both scenarios. However, as it is right now, scenario (2) will lead to reversals within a podcast, if there are multiple new (yet unqueued) episodes for a podcast.

For podcasts that do not refer to previous episodes, the order is not relevant. However, sequential podcasts (e.g. Serial, StartUp, podcasts with occasional part 1, 2, ... episodes) suffer greatly from episodes being reordered.

[mfietz]

We should probably have a setting for this, as both approaches have their justification.
Probably, before long, someone will ask for this setting per feed. While I accept that there are podcasts where oldest first makes more sense, I still think that most current first makes most sense for most podcasts.
The question is: How to combine both approaches? How would an algorithm rank episodes from podcasts with different policies?

[corecode]

Maybe we can start by describing the use case and reasoning for either way. Essentially I already started with my comment above. Could you add a description of the podcasts which should be downloaded newest first? I think this way we can find a distinguishing characteristic.

[mfietz]

Basically, every podcasts that talks about (tech) "news" (in the broadest sense of the word). Examples from television probably make most sense: Say you haven't watched your favorite daily late night show for a week. Would you rather watch the 7 days old episode (about something that happened 8 days ago) or last nights show? ;)
Even for podcasts like the fragmented podcasts, it makes most sense to me to still listen to the most current episode even if an older episode is far from being outdated.

[corecode]

I don't know what the fragmented podcasts are - could you maybe supply a link?

To summarize what we have so far:

• downloading old-to-new produces the same listening sequence as avid, always-connected listening does
• many podcasts benefit from or even require correct sequential listening
• news-style podcast episodes, particularly daily/frequent podcasts, may lose appeal as they age.
• for such news podcasts, newer episodes are more interesting; for those downloading newest first appears to be more appropriate

[corecode]

For news-style podcasts, maybe there should be a way to limit the number of podcast episodes that are considered "new"? That way if you get behind in the news, the "olds" would drop out of the "new" category. @mfietz would that make sense?

[mfietz]

I don't know what the fragmented podcasts are - could you maybe supply a link?

http://lmgtfy.com/?q=fragmented+podcast

[mfietz]

For news-style podcasts, maybe there should be a way to limit the number of podcast episodes that are considered "new"? ... would that make sense?

We don't want to surprise or confuse users. I have not idea how this would look in the GUI. A checkbox that suddenly activates several policies in the background (things that are not completely obvious to the user) won't cut it. The user would have to know, when she does makes a particular setting or a feed has a particular attribute, that, for instance, episodes older than X will automatically loose their newness.

[tralamazza]

Can we have an "Advanced Options" tab?

[corecode]

I did not suggest a checkbox that activates several policies.

I have the impression that you are trying hard to drive your own agenda, instead of objectively evaluating the suggestion of downloading podcasts old-to-new. I'll supply objective data below, and I hope you will produce evidence that supports your view.

I tallied up the top 50 iTunes podcasts, as listed on http://www.itunescharts.net/us/charts/podcasts/2015/10/26:

• 6 daily podcasts, out of which 2 are clearly not news, and one is clearly news
• 33 less frequent podcasts (twice a week or less);
• 8 fixed sequence podcasts
• 3 podcasts too new to judge

This data shows that the popular podcasts consist mostly of infrequent and serialized podcasts. News-style podcasts only make up a small portion. This data suggests that old-to-new downloads should be the default.

New-to-old, the current behavior, never makes sense: sequential (serialized) podcasts clearly suffer, and for news-style podcasts, listening to the news in reverse order makes no sense either.

[mfietz]

I did not suggest a checkbox that activates several policies.

Also, I never said you did, did I? I was just thinking aloud how the limiting could be implemented in the GUI. I like to think about the implementation early in the process, because what good is finding a theoretical solution when it the implementation fails?

I have the impression that you are trying hard to drive your own agenda

The way I see it: When you want to change a behavior that has been there for ages, you have to justify that change. Not the other way around (What you asked me to do...)

I'll supply objective data below, ...

Look, I never said that "newest first" only applies to news podcasts. Those were meant as an example. I also said that applies for me to other podcasts like fragmented. To be honest, I don't know most of the podcasts in the top 50. Yes, they objectively belong in those categories (more or less), that doesn't mean that for news podcasts the only "right" thing is the old behavior and for all others the behavior you suggested. And because there are more of the later, you are right.

New-to-old, the current behavior, never makes sense

Still disagree. Yes, the current behavior doesn't make sense in every case, but so doesn't your suggestion in my opinion.

To make it clear (because the way you are kind of attacking me suggests you misunderstood): I don't say your suggestion is not justified. I say both approaches are. I don't want to discuss about what is better or what makes more sense, but how we can have both - That's why I suggested a setting.

[mfietz]

For the sake of completeness: Somewhat related to #132 and #745

[corecode]

Thanks for adding these references. Thinking about it - shouldn't the "add to front of queue" option cover the newest-first behavior?

I.e.:

• "add to front" enabled: episodes are downloaded new-to-old, added to front of queue.
• "add to front" disabled: episodes are downloaded old-to-new, added to end of queue.

That way we don't need another option, and both behaviors can be implemented.

[mfietz]

I'm thinking more in the direction to replace add to front with a more general setting. This would set what the default assumption for podcasts would be, the default "natural" order (old to new or new to old).
This setting would basically do what you say. If the default order is old to new, we would both download older episodes first and make sure the older episode is higher up in the queue (I don't think this is guaranteed, currently; depending on which download finishes faster, you could get a wrong order).

There might be podcasts where it also makes sense to display the episodes oldest to newest, for instance audio books or radio plays. But this probably would have to be another setting, downloading the oldest new episode first does not really mean you also want to sort the episodes that way, does it?

[keunes]

I wrote a bunch of things on the process, but cutting the crap, I have one remark:
I think @mfietz already is thinking about it (talking about replacing 'add to front with a more general setting'), but I'm not sure. I think there's two different issues at play that we somehow need to combine:

• order of downloading
• order of ending up in the queue

This ties in to the remark made earlier by @corecode about two download/listening scenarios (continuous downloading & listening vs bursts).

What I'm concerned about (a problem that I have to solve manually now), is the following scenario which I wish was possible:

• per feed I can set new-to-old (let's call them the newsy feeds) or old-to-new (let's call them the sequential feeds)
• of the newsy feeds I always want to listen to the most recent episode; they should be added before any episodes of the same feed that may be already in the queue.
• however there's more episodes published that I can listen to, and I don't necessarily want the newsy feeds to always be at the top of the queue, because than I'll never get to listen to my sequential feeds.

It seems my scenario would be addressed by the abandoned #151. I want to highlight this issue here, because I feel it doesn't make a lot of sense to create a setting that separates old-to-new (sequential) feeds and new-to-old (newsy) feeds, when they end up somewhere in the queue in the wrong order anyway if the listener isn't connected to the internet all-time or if they listen to podcasts in bursts. (wrong order in the queue = when the not-latest episode is listened to earlier than the latest episode, because the not-latest was added to the queue earlier in an earlier download batch)

Don't know if I made myself clear at all; happy to try to answer any questions.

[mfietz]

Let's see if I understand you correctly.
Basically, one has to think about two levels of queue order. At the moment, we have a single level: add to top or add to bottom. Depending on the type of feed, you might want to have some kind of second level order.
Example: consecutive podcast. You add episodes at the top. Episode #2 comes out, the unplayed episode #1 sits at queue position #1. You probably don't want the episode to be added before #1, although the setting says so.
From a dev perspective: When adding an episode to the top/bottom of the queue, check if that current first/last episode is from the same podcast. If so, bring these two episodes in the right order.
(Maybe there are even cases I cannot currently think of where adding to the queue in general should make sure of the right order which the user could change manually, of course)

[mfietz]

Just came up with a scenario myself: Podcast 1 is any random podcast, podcast 2 is meant to be listen to sequencually
Queue:

1. Podcast 1, Episode 1
2. Podcast 2, Episode 2
   3...) Other episodes...
   [Time goes by]
   User adds Podcast 2, Episode 1

If new episodes are added at the top: nothing to worry about
If new episodes are added at the bottom: Episode should be added at position 2)

[corecode]

Do you mean the user manually adds the episode to the queue? I didn't think we were talking about this at all - this issue is about auto download of new episodes.

[mfietz]

Doesn't really matter if the episode is manually or automatically added to the queue. Still have to get the order right.

[corecode]

This issue is about which episodes to download first. I think more generic queue order concerns should go into a different issue.

[mfietz]

This would be a new issue introduced by the changes we discussed so far. I'd like to get this right at one go and not introduce an new issue by solving this one. Not going to create a hypothetical issue.

[corecode]

Okay, let's fix the issue at hand then: my sequential podcasts are downloaded/queued out of sequence. "Enqueue at Front" is disabled. #1298 provides a simple fix that makes my problem go away.

Should we modify #1298 so that it only downloads in podcast sequence when "Enqueue at Front" is disabled? This would cater to the sequential podcast crowd (have "Enqueue at Front" disabled), and to the news podcast crowd ("Enqueue at Front" enabled).

[keunes]

I was talking about automatic adding to the queue. Correct me if I'm wrong, but generally, adding to the queue and downloading go hand in hand. Whether that's done automatically or manually, you want to get the order right automatically.

As I see it, the whole point of this setting we're discussing is listening to the right episode. Downloading the right episode is merely a step in the process, rather than the end goal. Adding to the queue is another step in this process. Either you want to listen to the most recent episode of a feed, or you want to listen to all items in sequential order. @corecode: What would be the point of managing download order correctly, if the order of the episodes gets messed up later on by the queue. (I thus agree with @mfietz about integrating this discussion.)

In my opinion, ideally, there is one all-encompassing setting that covers both downloading and queue order. Ideally, this would eliminate the 'add to top/bottom' setting.

@mfietz: I think you're understanding what I tried to say.

[corecode]

I think there are several different behaviors that are being asked for, and we should tease them apart. This issue has been meandering for the last 5 years, with no clear solution in sight. One problem is that as people talk about their sort-of-related preferences, simple improvements are being discounted as not enough/not the right thing.

The latest conversation seems to be about starting a podcast on a backlog, instead of on the latest episode; I think essentially you're asking for all episodes to be marked as NEW, and be downloaded bit by bit as the oldest ones are being played and finished.

This issue originally was about a very simple thing: don't download NEW episodes starting from the newest NEW, but from the oldest NEW, because otherwise you get order reversals, or you might never download some NEW episodes, because additional NEW episodes keep coming in, which are newer, and therefore are being downloaded before the older NEW episodes. That's what this issue was about, originally.

I suggest opening a new issue for any behavior that goes beyond this simple download order change.

[antennapod-bot]

This issue has been mentioned on AntennaPod Forum. There might be relevant details there:

https://forum.antennapod.org/t/auto-download-is-downloading-episodes-in-backward-order/206/2

[Dunc0]

Hi All, I just stopped by as a user to say that this issue was why I left the app many years ago. I want to listen to all my podcasts from oldest to newest and it drove me insane that this functonality was not available. All I want to do is be able to view a podcast and have that podcase ONLY play from beginning to end. I always ended up with other podcast episodes mixed through from unrelated podcasts and I hated it. As a user, what I expect is a spotify/iTunes like experience where I can just go to the podcast I want and then sort from old to new.

[orionlee]

I haven't been able to get the time to finish the serial support (PR #3504).

In the meantime, I'd share my experience of listening to serial podcasts on AntennaPod, having used a branch based on PR #3504 for almost a year.

1. I have listened to more serial podcasts: I was more inclined to subscribe to new serial podcasts with the serial support added. I realize my choice of podcasts is actually influenced by the feature/UI of AntennaPod.

2. An issue is that it is hard for users to easily browse serial podcast episodes, say, if an users want to download them manually.

• The serial podcast episodes are buried in the Episode > New tab.
• Serial podcasts themselves are buried in the podcast subscriptions.

So irrespective of automatic download support, allowing users to filter subscriptions to see only serial podcasts would be helpful, which could be approximated by an user filtering subscriptions so see only those sorted from oldest-to-newest only (related to issue #3065).

[tonytamsf]

Thanks for this. Would it make sense if people sort the episodes for a podcast feed from old to new, that is indicating they want to down download from oldest to newest?

[Dunc0]

Hi Tonytamsf, not 100% sure if this was directed at me or orionlee or both, but from my point of view, yes. If I sort from old to new I expect old episodes at the top of the page and new at the bottom.

[antennapod-bot]

This issue has been mentioned on AntennaPod Forum. There might be relevant details there:

https://forum.antennapod.org/t/keep-x-latest-episodes/234/2

[Ethanol6]

If the original issue was about fixing the download order of new episodes such that the oldest new episode is downloaded before the newest new episode, then I argue that the functionality of downloading chronologically is nearly there. If an episode could be manually marked as new, which predates any current new episodes, then the current functionality would be to download that older episode that we manually marked as "new". Now if we could do a multi-select action and apply "new" to all or a large selection of episodes then that could work.

[keunes]

If the original issue was about fixing the download order of new episodes such that the oldest new episode is downloaded before the newest new episode

That's an interesting point. I don't think it is - I feel like this 'oldest episode first' should apply even to 'not-new episodes'. If I add a new podcast (then only the latest x episodes is marked as new), I wouldn't want to manually have to mark all previous episodes as 'new'.

[dcjkfgdjhd]

Any news on this?

As long as AntennaPod does not support serial podcasts as well as listening to the full back catalogue of episodical podcasts (such as the BBC's "In Our Time" or "A History of the World in 100 Objects") it is not a "proper" podcatcher...

[dcjkfgdjhd]

Thanks for this. Would it make sense if people sort the episodes for a podcast feed from old to new, that is indicating they want to down download from oldest to newest?

I don't think so. There are two sort settings: The sort order for viewing the list, and the sort order for downloading.

The first is something that I may want to change from time to time when browsing the podcast, depending on what is currently convenient depending on what I want in this moment.

The second is a permanent property of the podcast that should be stored in each podcast's settings page.

[kevincox]

Hi, I would like to offer a $100 CAD reward to a PR that gets merged with the following requirements.

1. A persistent option per-podcast that sets the preference to oldest-to-newest.
2. Episodes will be downloaded oldest-to-newest respecting download limits.
3. Episodes will play in the queue oldest-to-newest (should just work based on download order).
4. Once an episode is downloaded that should enable the next downloads to start as is currently the case.

The PR must be submitted in a respectful manor and be accepted by the AntennaPod devs. If the work is split among multiple PRs I will split (or at my choice duplicate) the rewards among the contributors. At the recipients request I will instead donate the bounty to charity or the AntennaPod project.

[ByteHamster]

The code for the auto-download feature is unfortunately pretty messy, currently. I think before implementing the feature, auto-downloads in general need a lot of love. See also my comment in this other issue: #2495 (comment)

[kevincox]

I will happily split the amount to include the required work there.

[joeytwiddle]

I would also like to offer an additional $200 CAD bounty for a PR meeting kevincox's requirements.

Hopefully this extra bounty will help to cover some work on the auto-downloads code. But that is entirely optional.

(If you complete the task but I am not responding on GitHub, please come find me on Twitter.)

[ByteHamster]

Thanks for the bounty @joeytwiddle. In case anyone is interested, please note that auto-downloads need a lot of love before this can be implemented properly. Just glueing it on top of the barely working existing logic will likely not get merged. So if you are interested please comment/discuss the ideas you have before starting to work on it.

[e-t-l]

Reading through the algorithm is a bit above my paygrade, unfortunately, and rewriting it myself is out of the question. I'd just like to add, though, that kevincox's requirements seem like they could be met by having a boolean that's a sub-option, perhaps a checkbox, EITHER under "Auto Delete Episode" that reads "Auto download next episode," OR under "Include in auto downloads" that reads "Download oldest first." Either way, it would function as follows:

When the currently playing episode is 90% played, the next most recent undownloaded episode is downloaded.

Example use case:
Episode 1 - played, not downloaded
Episode 2 - played, not downloaded
Episode 3 - CURRENTLY PLAYING, downloaded
Episode 4 - unplayed, downloaded
Episode 5 - unplayed, downloaded
Episode 6 - unplayed, not downloaded
Episode 7 - unplayed, not downloaded
etc
When Episode 3 is almost done playing, Episode 6 starts downloading. When Episode 3 finishes playing, it will be deleted* so the net amount of storage used by this podcast stays approximately constant (*If the user does not have "Auto Delete Episode" selected, presumably they don't have any issue with limited storage, so there shouldn't be an issue with downloading the next episode anyway. The user can always delete episodes manually as desired).

[Phoenix2683]

Jumping in to this convo late but if I could put my comment as a vote for how I'd like to see it.

As I see the convo has pointed out, there are two types of podcasts. Those which are more current event related. Think a news or political podcast, maybe a tech podcast. Then there are those that are serial. Maybe a book review/discussion a show.

If I'm starting a show after it's long been aired and want to follow along with the podcast, even if old, then yes I want the oldest episodes first. Currently even though i can set to oldest first, it's only going with the oldest NEW episode first which really defeats the purpose. This only works properly if I'm starting the podcast brand new. It feels like the assumption is still "why would you want to listen to old episodes".

To me rather than having to mark all those old ones as new, wouldn't it make sense to use a function that already exists for the reverse case? If someone wants oldest first but only for new episodes, they can mark all the old episodes as played and with the current features, the problem is solved. I am guessing (not a programmer) that it would be easier to make the setup play oldest first (regardless of new status) than create a new feature to mark old episodes as new?

The other option is when adding a new subscription all existing episodes are considered new, if you don't ever want to listen to the old ones, just once again mark them as unplayed.

I love the app, love supporting open source work, this is the one feature that really gets at me when trying to listen to my podcasts.

Thanks for the work!

[mruessler]

It feels like the assumption is still "why would you want to listen to old episodes".

True. I think it is good to point this out because then you can think whether the assumption can be generalised or is itself just one usecase.
I am discovering more and more series podcast. You wouldn’t want to start Game of thrones in season 7 episode 4 just because that is the last episode considered as ›new‹ (I am making this up as an example).

[dcjkfgdjhd]

Without this feature, this is not a fully usable audio feed player.

Lots of podcasts require it. Probably actually it is the best podcasts that do, because what is the point of podcasts? It is not to listen to shortlived news stories (we get more than enough of that on the radio or on TV) but to listen to longer, more complex, in-depth reports on subjects of longer lasting relevance.

BeyondPod has had that function since forever. You can simply choose between "Download episodes in order" or "Download newest episodes".

[antennapod-bot]

This issue has been mentioned on AntennaPod Forum. There might be relevant details there:

https://forum.antennapod.org/t/proposal-for-auto-download-redesign/1291/31

https://forum.antennapod.org/t/episodic-versus-sequential/3256/5

[keunes]

With 103 comments, this is the second-longest issue discussion ever for AntennaPod. It's on our radar, we have plenty of use-cases discussed above. We are now discussing the best course of action on our forum.

I will lock this issue now to avoid further reading work for anyone analysing this difficult matter.

If you are interested in contributing to the structured problem review in video-calls (regardless your coding interest or experience), please do let us know on the forum.

[keunes]

Dear followers of this long-standing issue.

We are running some user research related to the topic of this issue! This research will help us understand more about you and how you listen to podcasts, allowing us to ensure that we’re always building things in a way that makes sense for you, our users.

If you’d like to take part, then please complete this short survey (<5 minutes) and let us know on there if you’d be happy to take part in an interview to help us learn more.

Quick note: for GDPR reasons please only take part if you are over 18.

https://survey.antennapod.org/index.php/657295