IPFS to store videos

[alxlg]

I think that what limits PeerTube adoption is that instances are perfect for personal/organization use but not to build a free service like YouTube where everyone can upload videos without limits. The issue is that storage has a cost and videos make the necessary storage grow quickly.

IPFS (InterPlanetary File System) can be used to solve the storage issue because every user can store the files by himself but it doesn't have a way to browse and interact with them. PeerTube instead has an awesome UI that can be used by everyone.

Would it be possible to combine PeerTube and IPFS? Ideally the instance administrator would limit the classic upload for each user but eventually let users upload videos by specify an IPFS address. I guess when a second user browse a PeerTube instance and want to watch a video hosted on IPFS, PeerTube provides it by reading from IPFS and not from its local storage. PeerTube instances would cache IPFS contents like IPFS users and admins would monitor IPFS cache impact on their storage. If a PeerTube user wants to be sure its video is available he just have to keep it on IPFS with his machine. This could have another advantage: if the used PeerTube instance won't be available anymore its users won't need to upload videos on other PeerTube instances if they are on IPFS: they would just "upload" the IPFS addresses.

I will be grateful to those who answer by denying or confirming my assumptions.

[rigelk]

Ideally the instance administrator would limit the classic upload for each user but eventually let users upload videos by specify an IPFS address. I guess when a user […] wants to watch a video hosted on IPFS, PeerTube provides it by reading from IPFS and not from its local storage.

@alxlg indeed that's more or less how I envisioned the potential use of IPFS. But then there's the fact an IPFS endpoint is not a webseed[¹] nor holds versions of different quality. In other words, IPFS would only be a second class citizen feature-wise.

¹: let me extend a bit on that issue. The fact is that we use WebTorrent (BitTorrent/WebRTC) on the client side to watch videos. It provides a handy pool of content seeders and direct browser connection. Watching a video via IPFS would mean to replace entirely that component with an IPFS client in the browser. So it's not just thinking of a different storage/upload mechanism.

If you have any ideas as to how to solve these problems, I'm all ears :)

P.S.: we have also not heard a lot about IPFS performance-wise when it comes to streaming videos.

[rigelk]

Ideally the instance administrator would limit the classic upload for each user but eventually let users upload videos by specify an IPFS address. I guess when a user […] wants to watch a video hosted on IPFS, PeerTube provides it by reading from IPFS and not from its local storage.

@alxlg indeed that's more or less how I envisioned the potential use of IPFS. But then there's the fact an IPFS endpoint is not a webseed[¹] nor holds versions of different quality. In other words, IPFS would only be a second class citizen feature-wise.

¹: let me extend a bit on that issue. The fact is that we use WebTorrent (BitTorrent/WebRTC) on the client side to watch videos. It provides a handy pool of content seeders and direct browser connection. Watching a video via IPFS would mean to replace entirely that component with an IPFS client in the browser. So it's not just thinking of a different storage/upload mechanism.

If you have any ideas as to how to solve these problems, I'm all ears :)

P.S.: we have also not heard a lot about IPFS performance-wise when it comes to streaming videos.

[alxlg]

@rigelk thanks for your reply!

I had not thought of different quality versions of videos. Since IPFS is really low in the stack the only solution I can think of is storing a different IPFS file for each version. The user should be able to specify the IPFS address for each quality version he wants to maintain... This doesn't seem user-friendly but with a desktop client that automatically manage versions on IPFS it could gain adoption... Ideally the desktop client could use some API to upload the video to PeerTube by specifying many IPFS addresses. Desktop client's users should just pick a video from their HDD and the desktop client would generate different versions, upload them to IPFS and load the addresses to a PeerTube instance.

It seems like a big amount of work but promising to me and the idea could get many contributors.

[alxlg]

@rigelk thanks for your reply!

I had not thought of different quality versions of videos. Since IPFS is really low in the stack the only solution I can think of is storing a different IPFS file for each version. The user should be able to specify the IPFS address for each quality version he wants to maintain... This doesn't seem user-friendly but with a desktop client that automatically manage versions on IPFS it could gain adoption... Ideally the desktop client could use some API to upload the video to PeerTube by specifying many IPFS addresses. Desktop client's users should just pick a video from their HDD and the desktop client would generate different versions, upload them to IPFS and load the addresses to a PeerTube instance.

It seems like a big amount of work but promising to me and the idea could get many contributors.

[rigelk]

@alxlg we're not even close to writing a Desktop client. This is a non-option considered our resources.

I was considering leaving the video uploaded without transcoding, thus leaving a single quality available. It's always better than no video at all.

---

But now that I come to think of it, about ¹: do we even have to replace the webtorrent client for ipfs videos? If we could manage to mark ipfs endpoints as WebSeed, we could just use them under the hood of webtorrent by making the webtorrent client aware of them.

[rigelk]

@alxlg we're not even close to writing a Desktop client. This is a non-option considered our resources.

I was considering leaving the video uploaded without transcoding, thus leaving a single quality available. It's always better than no video at all.

---

But now that I come to think of it, about ¹: do we even have to replace the webtorrent client for ipfs videos? If we could manage to mark ipfs endpoints as WebSeed, we could just use them under the hood of webtorrent by making the webtorrent client aware of them.

[alxlg]

@rigelk in fact I did not intend to replace the video player. I thought that the server could run both PeerTube and a IPFS node, and the PeerTube instance see the file cached with IPFS like local files... I hope it has sense now...

This feature doesn't depend on a desktop client, but it would just help normal PeerTube users to automatically store their videos locally.

I would be happy to store some videos with IPFS on my HDD without a PeerTube instance that need much more maintenance.

I think the change would mostly be in PeerTube UI providing a way to upload a video through a IPFS address and not uploading the entire file to the PeerTube instance. Of course the server admin should configure it to run IPFS...

[alxlg]

@rigelk in fact I did not intend to replace the video player. I thought that the server could run both PeerTube and a IPFS node, and the PeerTube instance see the file cached with IPFS like local files... I hope it has sense now...

This feature doesn't depend on a desktop client, but it would just help normal PeerTube users to automatically store their videos locally.

I would be happy to store some videos with IPFS on my HDD without a PeerTube instance that need much more maintenance.

I think the change would mostly be in PeerTube UI providing a way to upload a video through a IPFS address and not uploading the entire file to the PeerTube instance. Of course the server admin should configure it to run IPFS...

[Openmedianetwork]

Would not a webRTC torrent app running in the users PC do the same thing when she/he leave it seeding the video? This also allows user to create "seedboxs" using RSS auto downloading torrent apps. Easy simple/KISS way of "distributing" the video hosting.

https://github.com/Openmedianetwork/visionOntv/wiki/Seedbox

[Openmedianetwork]

Would not a webRTC torrent app running in the users PC do the same thing when she/he leave it seeding the video? This also allows user to create "seedboxs" using RSS auto downloading torrent apps. Easy simple/KISS way of "distributing" the video hosting.

https://github.com/Openmedianetwork/visionOntv/wiki/Seedbox

[alxlg]

@Openmedianetwork good point! I think your proposal is easier to implement but using IPFS too could have some advantages. For example, I'm pretty sure that if an instance of PeerTube is no longer available an user can reupload his/her IPFS videos on another instance just sharing IPFS addresses, much better than reuploading video files! Do you think this could be achieved with torrents/WebTorrents too? Would changes to PeerTube be needed? Maybe uploading a *.torrent file instead of a video file?

[alxlg]

@Openmedianetwork good point! I think your proposal is easier to implement but using IPFS too could have some advantages. For example, I'm pretty sure that if an instance of PeerTube is no longer available an user can reupload his/her IPFS videos on another instance just sharing IPFS addresses, much better than reuploading video files! Do you think this could be achieved with torrents/WebTorrents too? Would changes to PeerTube be needed? Maybe uploading a *.torrent file instead of a video file?

[rigelk]

@alxlg using an IPFS address or a *.torrent file yields the same import capabilities. See #102. The only advantage of IPFS I see is that there are pinning brokers. (For BitTorrent too? I didn't check)

[rigelk]

@alxlg using an IPFS address or a *.torrent file yields the same import capabilities. See #102. The only advantage of IPFS I see is that there are pinning brokers. (For BitTorrent too? I didn't check)

[Openmedianetwork]

issues with testing webrtc http://hamishcampbell.com/index.php/2018/05/18/testing-peertube-webrtc-seedboxes/

[Openmedianetwork]

issues with testing webrtc http://hamishcampbell.com/index.php/2018/05/18/testing-peertube-webrtc-seedboxes/

[rigelk]

@Openmedianetwork this has nothing to do with IPFS. Please find a related issue and detail your problem there, not in a blog post.

[rigelk]

@Openmedianetwork this has nothing to do with IPFS. Please find a related issue and detail your problem there, not in a blog post.

[Openmedianetwork]

Sorry this was a update for alexig that my suggestion for webRTC torrent as an alternative for seeding dues not appear to work after actually testing. Will start a new thread after further of tests.

[Openmedianetwork]

Sorry this was a update for alexig that my suggestion for webRTC torrent as an alternative for seeding dues not appear to work after actually testing. Will start a new thread after further of tests.

[Chocobozzz]

Closing this issue since we have no really use cases for IPFS for now.

[Chocobozzz]

Closing this issue since we have no really use cases for IPFS for now.

[NightA]

@rigelk
IPFS can serve as a backup for local storage and dedicated seeding pools, as it effectively transfers each newly added files to an entire pre-existing network of 300+ peers from the get-go. In such an instance, PeerTube might not necessarily even have to double as a WebRTC-based IPFS node but simply run alongside side a regular one (which in-itself can be optional), first to ensure the files initially propagate through the network and secondly to provide an optional method for retrieving them through a local gateway. However simply linking to an IPFS address should suffice if it will be possible to configure a PeerTube instance to use external public gateways for retrieval.

In a case where a PeerTube instance should go down with no pre-existing seeding pools in place, as long as the videos are still present on IPFS, it should be possible to retrieve them by simply following each video's address (that presumably was shared beforehand with other federated instances). This way each video will remain accessible and therefor could be later conventionally reseeded via a different instance.

As a by-product, if it'll be possible to authenticate each user's identity, perhaps it might also be possible to use this method for transferring channels between different PeerTube instances.

[NightA]

@rigelk
IPFS can serve as a backup for local storage and dedicated seeding pools, as it effectively transfers each newly added files to an entire pre-existing network of 300+ peers from the get-go. In such an instance, PeerTube might not necessarily even have to double as a WebRTC-based IPFS node but simply run alongside side a regular one (which in-itself can be optional), first to ensure the files initially propagate through the network and secondly to provide an optional method for retrieving them through a local gateway. However simply linking to an IPFS address should suffice if it will be possible to configure a PeerTube instance to use external public gateways for retrieval.

In a case where a PeerTube instance should go down with no pre-existing seeding pools in place, as long as the videos are still present on IPFS, it should be possible to retrieve them by simply following each video's address (that presumably was shared beforehand with other federated instances). This way each video will remain accessible and therefor could be later conventionally reseeded via a different instance.

As a by-product, if it'll be possible to authenticate each user's identity, perhaps it might also be possible to use this method for transferring channels between different PeerTube instances.