Skip to content

@omarroth omarroth released this Jul 30, 2019 · 152 commits to master since this release

Minor fix to video extractor after changes by YouTube.

It is recommended to update to this version immediately.

Assets 2

@omarroth omarroth released this Jul 13, 2019

Hello again everyone! Focus this month has mainly been on improving playback performance, along with a couple new features I'd like to announce. There have been 109 commits this past month from 10 contributors.

This past month has seen the addition of Chinese (zh-CN) and Icelandic (is) translations. I would like to give a huge thanks to their respective translators, and again an enormous thanks to everyone who helps translate the site.

I'm delighted to mention that FreeTube 0.6.0 now supports 1080p thanks to the Invidious API. I would very much recommend reading the relevant post for some more information on how it works, along with several other major improvements. Folks that are interested in adding similar functionality for their own projects should feel free to get in touch.

This past month there has been quite a bit of work on improving memory usage and improving download and playback speeds. As mentioned in the previous release, some extra hardware has been allocated which should also help with this. I'm still looking for ways to improve performance and feedback is always appreciated.

Along with performance, a couple quality of life improvements have been added, including author thumbnails and banners, clickable titles for embedded videos, and better styling for captions, among some other enhancements.

Communities

Support for YouTube's communities tab has been added. It's a very interesting but surprisingly unknown feature. Essentially, providing comments for a channel, rather than a video, where an author can post updates for their subscribers.

It's commonly used to promote interesting links and foster discussion. I hope this feature helps people find more interesting content that otherwise would have been overlooked.

For Developers

For accessing channel communities, an /api/v1/channels/comments/:ucid endpoint has been added, with similar behavior and schema to /api/v1/comments/:id, with an extra attachment field for top-level comments. More info on usage and available data can be found in the wiki.

An /api/v1/auth/feeds endpoint has been added for programmatically accessing a user's subscription feed, with options for displaying notifications and filtering an existing feed.

An /api/v1/search/suggestions endpoint has been added for retrieving suggestions for a given query.

For Administrators

It is now possible to disable more resource intensive features, such as downloads and DASH functionality by adding disable_proxy to your config. See #453 and the Wiki for more information and example usage. I expect this to be a big help for folks with limited bandwidth when hosting their own instances.

Finances

Donations

  • Patreon : $38.39
  • Liberapay : $84.85
  • Crypto : ~$0.00 (converted from BCH, BTC)
  • Total : $123.24

Expenses

  • invidious-load1 (nyc1) : $10.00 (load balancer)
  • invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
  • invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node5 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node6 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node7 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node8 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node9 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node10 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
  • Total : $105.00

The goal on Patreon has been updated to reflect the above expenses. As mentioned above, the main reason for more hardware is to improve playback and download speeds, although I'm still looking into improving performance without allocating more hardware.

As always I'm grateful for everyone's support and feedback. I'll see you all next month.

Assets 2

@omarroth omarroth released this Jun 6, 2019 · 281 commits to master since this release

Hope everyone has been doing well. This past month there have been 97 commits from 10 contributors. For the most part changes this month have been on optimizing various parts of the site, mainly subscription feeds and support for serving images and other assets.

I'm quite happy to mention that support for Greek (el) has been added, which I hope will continue to make the site accessible for more users.

Subscription feeds will now only update when necessary, rather than periodically. This greatly lightens the load on DB as well as making the feeds generally more responsive when changing subscriptions, importing data, and when receiving new uploads.

Caching for images and other assets should be greatly improved with #456. JavaScript has been pulled out into separate files where possible to take advantage of this, which should result in lighter pages and faster load times.

This past month several people have encountered issues with downloads and watching high quality video through the site, see #532 and #562. For this coming month I've allocated some more hardware which should help with this, and I'm also looking into optimizing how videos are currently served.

For Developers

viewCount is now available for /api/v1/popular and all videos returned from /api/v1/auth/notifications. Both also now provide "type" for indicating available information for each object.

An /authorize_token page is now available for more easily creating new tokens for use in applications, see this comment in #473 for more details.

A POST /api/v1/auth/notifications endpoint is also now available for correctly returning notifications for 150+ channels.

For Administrators

There are two new schema changes for administrators: views for adding view count to the popular page, and feed_needs_update for tracking feed changes.

As always the relevant migration scripts are provided which should run when following instructions for updating. Otherwise, adding check_tables: true to your config will automatically make the required changes.

Native Notifications

It is now possible to receive Web notifications from subscribed channels.

You can enable notifications by clicking "Enable web notifications" in your preferences. Generally they appear within 20-60 seconds of a new video being uploaded, and I've found them to be an enormous quality of life improvement.

Although it has been fairly stable, please feel free to report any issues you find here or emailing me directly at omarroth@protonmail.com.

Important to note for administrators is that instances require use_pubsub_feeds and must be served over HTTPS in order to correctly send web notifications.

Finances

Donations

  • Patreon : $49.73
  • Liberapay : $100.57
  • Crypto : ~$11.12 (converted from BCH, BTC)
  • Total : $161.42

Expenses

  • invidious-load1 (nyc1) : $10.00 (load balancer)
  • invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
  • invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node5 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node6 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
  • Total : $85.00

See you all next month!

Assets 2

@omarroth omarroth released this May 6, 2019 · 378 commits to master since this release

Hello everyone! This past month there have been 130 commits from 11 contributors. Large focus has been on improving the player as well as adding API access for other projects to make use of Invidious.

There have also been significant changes in preparation of native notifications (see #195, #469, #473, and #502), and playlists. I expect to see both of these to be added in the next release.

I'm quite happy to mention that new translations have been added for Esperanto (eo) and Ukranian (uk). Support for pluralization has also been added, so it should now be possible to make a more native experience for speakers in other languages. The system currently in place is a bit cumbersome, so for any help using this feature please get in touch!

For Administrators

A check_tables option has been added to automatically migrate without the use of custom scripts. This method will likely prove to be much more robust, and is currently enabled for the official instance. To prevent any unintended changes to the DB, check_tables is disabled by default and will print commands before executing. Having this makes features that require schema changes much easier to implement, and also makes it easier to upgrade from older instances.

As part of #303, a cache_annotations option has been added to speed up access from /api/v1/annotations/:id. This vastly improves the experience for videos with annotations. Currently, only videos that contain legacy annotations will be cached, which should help keep down the size of the cache. cache_annotations is disabled by default.

For Developers

An authorization API has been added which allows other applications to read and modify user subscriptions and preferences (see #473). Support for accessing user feeds and notifications is also planned. I believe this feature is a large step forward in supporting syncing subscriptions and preferences with other services, and I'm excited to see what other developers do with this functionality.

Support for server-to-client push notifications is currently underway. This allows Invidious users, as well as applications using the Invidious API, to receive notifications about uploads in near real-time (see #469). An /api/v1/auth/notifications endpoint is currently available. I'm very excited for this to be integrated into the site, and to see how other developers use it in their own projects.

An /api/v1/storyboards/:id endpoint has been added for accessing storyboard URLs, which allows developers to add video previews to their players (see below).

Player

Support for annotations has been merged into master with #303, thanks @glmdgrielson! Annotations can be enabled by default or only for subscribed channels, and can also be toggled per video. I'm extremely proud of the progress made here, and I'm so thankful to everyone that has made this possible. I expect this to be the last update with regards to supporting annotations, but I do plan on continuing to improve the experience as much as possible.

The Invidious player now supports video previews and a corresponding API endpoint /api/v1/storyboards/:id has been added for developers looking to add similar functionality to their own players. Not much else to say here. Overall it's a very nice quality of life improvement and an attractive addition to the site.

It is now possible to select specific sources for videos provided using DASH (see #34). I would consider support largely feature complete, although there are still several issues to be fixed before I would consider it ready for larger rollout. You can watch videos in 1080p by setting Default quality to dash in your preferences, or by adding &quality=dash to the end of video URLs.

Finances

Donations

  • Patreon : $49.73
  • Liberapay : $63.03
  • Crypto : ~$0.00 (converted from BCH, BTC)
  • Total : $112.76

Expenses

  • invidious-load1 (nyc1) : $10.00 (load balancer)
  • invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
  • invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node5 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
  • Total : $80.00

That's all for now. Thanks!

Assets 2

Hello again! This past month has seen 116 commits from 13 contributors and a couple important changes I'd like to announce.

A privacy policy is now available here. I've done my best to explain things as clearly as possible without oversimplifying, and would very much recommend reading it if you're concerned about your privacy and want to learn more about how Invidious uses your data. Please let me know if there is anything that needs clarification.

I'm also very happy to announce that a Spanish translation has been added to the site. You can use it with ?hl=es or by setting es as your default locale. As always I'm extremely grateful to translators for making the site accessible to more people.

For Administrators

Invidious now supports server-to-server push notifications. This uses PubSubHubbub to automatically handle new videos sent to an instance, which is less resource intensive and generally faster. Note that it will not pull all videos from a subscribed channel, so recommended usage is in addition to channel_threads. Using PubSub requires a valid domain that updates can be sent to, and a random string that can be used to sign updates sent to the instance. You can enable it by adding use_pubsub_feeds: true to your config.yml. See Configuration for more info.

Unfortunately there are a couple necessary changes to the DB to support liveNow and premiereTimestamp in subscription feeds. Migration scripts have been provided that should be used automatically if following the instructions here.

You can now configure default user preferences for your instance. This allows you to set default locale, player preferences, and more. See #415 for more details and example usage.

For Developers

The fields API has been added with #429 and is now supported on all JSON endpoints, thanks @afrmtbl! Syntax is straight-forward and can be used to reduce data transfer and create a simpler response for debugging. You can see an example here. I've been quite happy using it and hope it is similarly useful for others.

An /api/v1/annotations/:id endpoint has been added for pulling legacy annotation data from this archive, see below for more details. You can also access annotation data available on YouTube using ?source=youtube, although this will only return card data as legacy annotations were deleted on January 15th.

A couple minor changes to existing endpoints:

  • A premiereTimestamp field has been added to /api/v1/videos/:id
  • A sort_by param has been added to /api/v1/comments/:id, supports new, top.

More info is available in the documentation.

Annotations

I'm pleased to announce that annotation data is finally available from the roughly 1.4 billion videos archived as part of this project. They are accessible from the Internet Archive here or as a 355GB torrent, see here for more details. A corresponding /api/v1/annotations/:id endpoint has been added to Invidious which uses the collection from IA to provide legacy annotations.

Support for them in the player is possible thanks to this plugin developed by @afrmtbl. A PR for adding support to the site is available as #303. There's also an extension for overlaying them on top of the YouTube player (again thanks to @afrmtbl), and an extension for hooking into code still present in the YouTube player itself, developed by @tech234a.

I would recommend reading the official announcement for more details. I would like to again thank everyone that helped contribute to this project.

Finances

Donations

  • Patreon : $42.42
  • Liberapay : $70.11
  • Crypto : ~$1.76 (converted from BCH, BTC, BSV)
  • Total : $114.29

Expenses

  • invidious-load1 (nyc1) : $10.00 (load balancer)
  • invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
  • invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node5 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
  • Total : $80.00

This past month the site saw a couple abnormal peaks in traffic, so an additional webserver has been added to match the increased load. The goal on Patreon has been updated to match the above expenses.

Thanks everyone!

Assets 2

@omarroth omarroth released this Mar 6, 2019 · 624 commits to master since this release

The project has seen quite a bit of activity this past month. Large focus has been on fixing bugs, but there's still quite a few new features I'm happy to announce. There have been 133 commits from 15 contributors this past month.

As a couple miscellaneous changes, a couple nice screenshots have been added to the README, so folks can see more of what the site has to offer without creating an account.

The footer has also been cleaned up quite a bit, and now displays the current version, so it's easier to know what features are available from the current instance.

For Administrators

This past month there has been a minor release - 0.14.1 - which fixes a breaking change made by YouTube for their polymer redesign.

There have been several new features that unfortunately require a database migration. There are migration scripts provided in config/migrate-scripts, and the wiki has instructions for automatically applying them. I'll do my best to keep those changes to a minimum, and expect to see a corresponding script to automatically apply any new changes.

Administrator preferences have been added with #312, which allows administrators to customize their instance. Administrators can change the order of feed menus, change the default homepage, disable open registration, and several other options. There's a short 'how-to' here, and the new options are documented here.

An /api/v1/stats endpoint has been added with #356, which reports the instance version and number of active users. Statistics are disabled by default, and can be enabled in administator preferences. Statistics for the official instance are available here.

For Developers

/api/v1/channels/:ucid now provides an autoGenerated tag, which returns true for topic channels, and larger genre channels generated by YouTube. These channels don't have any videos of their own, so latestVideos will be empty. It is recommended instead to display a list of playlists generated by YouTube.

You can now pull a list of playlists from a channel with /api/v1/channels/playlists/:ucid. Supported options are documented in the wiki. Pagination is handled with a continuation token, which is generated on each call. Of note is that auto-generated channels currently have one page of results, and subsequent calls will be empty.

For quickly pulling the latest 30 videos from a channel, there is now /api/v1/channels/latest/:ucid. It is much faster than a call to /api/v1/channels/:ucid. It will not convert an author name to a valid ucid automatically, and will not return any extra data about a channel.

Preferences

In addition to administrator preferences mentioned above, you can now change your preferences without an account (see #42). I think this is quite an improvement to the usability of the site, and is much friendlier to privacy-conscious folks that don't want to make an account. Preferences will be automatically imported to a newly created account.

Several issues with sorting subscriptions have been fixed, and /manage_subscriptions has been sped up significantly. The subscription feed has also seen a bump in performance. Delayed notifications have unfortunately started becoming a problem now that there are more users on the site. Some new changes are currently being tested which should mostly resolve the issue, so expect to see more in the next release.

Channel Playlists

You can now view available playlists from a channel, and auto-generated channels are no longer empty. You can sort as you would on YouTube, and all the same functionality should be available. I'm quite pleased to finally have it implemented, since it's currently the only data available from the above mentioned auto-generated channels, and makes it much easier to consume music on the site.

There's also more discussion on improving Invidious for streaming music in #304, and adding support for music.youtube.com. I would appreciate any thoughts on how to improve that experience, since it's a very large and useful part of YouTube.

Finances

Donations

Patreon : $42.42
Liberapay : $30.97
Crypto : ~$0.00 (converted from BCH, BTC)
Total : $73.39

Expenses

invidious-load1 (nyc1) : $10.00 (load balancer)
invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
Total : $75.00

It's been very humbling to see how fast the project has grown, and I look forward to making the site even better. Thank you everyone.

Assets 2
Feb 25, 2019
Pull dash URL from player response

@omarroth omarroth released this Feb 6, 2019 · 757 commits to master since this release

This last month several contributors have made improvements specifically for the people using this project. New pages have been added to the wiki, and there is now a Matrix Server and IRC channel so it's easier and faster for people to ask questions or chat. There have been 101 commits since the last major release from 8 contributors.

It has come to my attention in the past month how many people are self-hosting, and I would like to make it easier for them to do so.

With that in mind, expect future releases to have a section for For Administrators (if any relevant changes) and For Developers (if any relevant changes).

For Administrators

This month the most notable change for administrators is releases. As always, there will be a major release each month. However, a new minor release will be made whenever there are any critical bugs that need to be fixed.

This past month is the first time there has been a minor release - 0.13.1 - which fixes a breaking change made by YouTube. Administrators using versioning for their instances will be able to rely on the latest version, and should have a system in place to upgrade their instance as soon as a new release is available.

Several new pages have been added to the wiki (as mentioned below) that will help administrators better setup their own instances. Configuration, maintenance, and instructions for updating are of note, as well as several common issues that are encountered when first setting up.

For Developers

There's now a pretty=1 parameter for most endpoints so you can view data easily from the browser, which is convenient for debugging and casual use. You can see an example here.

Unfortunately the /api/v1/insights/:id endpoint is no longer functional, as YouTube removed all publicly available analytics around a month ago. The YouTube endpoint now returns a 404, so it's unlikely it will be functional again.

Wiki

There have been a sizable number of changes to the Wiki, including a list of public Invidious instances, the list of extensions, and documentation for administrators (as mentioned above) and developers.

The wiki is editable by anyone so feel free to add anything you think is useful.

Matrix & IRC

Thee is now a Matrix Server for Invidious, so please feel free to hop on if you have any questions or want to chat. There is also a registered IRC channel: #invidious on Freenode which is bridged to Matrix.

Features

Several new features have been added, including a download button, creator hearts and comment colors, and a French translation.

There have been fixes for Google logins, missing text in locales, invalid links to genre channels, and better error handling in the player, among others.

Several fixes and features are omitted for space, so I'd recommend taking a look at the compare tab for more information.

Annotations Update

Annotations were removed January 15th, 2019 around15:00 UTC. Before they were deleted we were able to archive annotations from around 1.4 billion videos. I'd very much recommend taking a look here for more information and a list of acknowledgements. I'm extremely thankful to everyone who was able to contribute and I'm glad we were able to save such a large part of internet history.

There's been large strides in supporting them in the player as well, which you can follow in #303. You can preview the functionality at https://dev.invidio.us . Before they are added to the main site expect to see an option to disable them, both site-wide and per video.

Organizing this project has unfortunately taken up quite a bit of my time, and I've been very grateful for everyone's patience.

Finances

Donations

  • Patreon : $49.42
  • Liberapay : $27.89
  • Crypto : ~$0.00 (converted from BCH, BTC)
  • Total : $77.31

Expenses

invidious-load1 (nyc1) : $10.00 (load balancer)
invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
Total : $75.00

As always I'm grateful for everyone's contributions and support. I'll see you all in March.

Assets 2

@omarroth omarroth released this Jan 19, 2019 · 854 commits to master since this release

Add missing "avg_rating"
Assets 2

@omarroth omarroth released this Jan 6, 2019 · 858 commits to master since this release

I hope everyone had a happy New Year! There's been a couple new additions since last release, with 44 commits from 9 contributors. It's been quite a year for the project, and I hope to continue improving the project into 2019! Starting off the new year:

Translations

I'm happy to announce support for translations has been added with a160c64. Currently, there is support for:

  • Arabic (ar)
  • Dutch (nl)
  • English (en-US)
  • German (de)
  • Norwegian Bokmål (nb_NO)
  • Polish (pl)
  • Russian (ru)

Which you can change in your preferences under Language. You can also add &hl=LANGUAGE to the end of any request to translate it to your preferred language, for example https://invidio.us/?hl=ru. I'd like to say thank you again to everyone who has helped translate the site! I've mentioned this before, but I'm delighted that so many people find the project useful.

Annotations

Recently, YouTube announced that all annotations will be deleted on January 15th, 2019. I believe that annotations have a very important place in YouTube's history, and announced a project to archive them.

I expect annotations to be supported in the Invidious player once archiving is complete (see #110 for details), and would also like to host them for other developers to use in their projects.

The code is available here, and contains instructions for running a worker if you would like to contribute. There's much more information available in the announcement as well for anyone who is interested.

Tor

I unfortunately missed the chance to mention this in the previous release, but I'm now happy to announce that you can now view Invidious through Tor at the following links:

kgg2m7yk5aybusll.onion
axqzx4s6s54s32yentfqojs3x5i7faxza6xo3ehd4bzzsg2ii4fv2iid.onion

Invidious is well suited to use through Tor, as it does not require any JS and is fairly lightweight. I'd recommend looking here and here for more details on how to use the onion links, and would like to say thank you to /u/whonix-os for suggesting it and providing support setting setting them up.

Popular and Trending

You can now easily view videos trending on YouTube with a16f967. It also provides support for viewing YouTube's various categories categories, such as News, Gaming, and Music. You can also change the region parameter to view trending in different countries, which should be made easier to use in the coming weeks.

A link to /feed/popular has also been added, which provides a list of videos sorted using the algorithm described here. I think it better reflects what users watch on the site, but I'd like to hear peoples' thoughts on this and on how it could be improved.

Finances

Donations

  • Patreon: $64.63
  • Liberapay : $30.05
  • Crypto : ~$28.74 (converted from BCH, BTC)
  • Total : $123.42

Expenses

  • invidious-load1 (nyc1) : $10.00 (load balancer)
  • invidious-update1 (s-1vcpu-1gb) : $5.00 (updates feeds)
  • invidious-node1 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node2 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node3 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-node4 (s-1vcpu-1gb) : $5.00 (web server)
  • invidious-db1 (s-4vcpu-8gb) : $40.00 (database)
  • Total : $75.00

What will happen with what's left over?

I believe this is the first month that all expenses have been fully paid for by donations. Thank you! I expect to allocate the current amount for hardware to improve performance and for hosting annotation data, as mentioned above.

Anything that is left over is kept to continue hosting the project for as long as possible. Thank you again everyone!

I think that's everything for 2018. There's lots still planned, and I'm very excited for the future of this project!

Assets 2
You can’t perform that action at this time.