Gaideo - A decentralized Youtube with encrypted HLS video content stored on users Gaia storage

[models-72]

We prefer clear, concrete, and concise applications. If an application is incomplete or unclear, we will request an update to the application.

Background

What problems do you aim to solve? How does it serve the mission of a user owned internet?

The current video/social media platforms like YouTube/Facebook all centralize and censor their content. The idea behind this platform is to allow each user to own their videos/pictures wherever they choose to have their Gaia storage. The content owner can decide how public the data is (private or community), as well as the search terms to use for discovery. All of the data will be encrypted and completely controlled by the content owner. This gets video content back into the hands of the creators where it belongs without a big centralized company involved.

Project Overview

What solution are you providing? Who will it serve?

UPDATED OVERVIEW 9/10/2020

The solution is a React application that allows users to upload encrypted video and images to Gaia. Encryption must be done outside of the React application. A public docker image is available to users to encapsulate all of the heavy lifting of encrypting videos. Users will be able to create a Community and control other blockstack users to be in their community. Videos/images made available to a community will have the images and the video "key" encrypted using a shared password using AES instead of the normal Gaia encryption. Community owners will be able to control the shared password and membership.

Please checkout https://gaideo.com to see this in action. It is very much a work in progress but already has the capability to upload and view encrypted video.

One of the recent comments suggested using ffmeg.js. I will investigate to see if it can be a replacement for needing to run a docker image.

Scope

What are the components or technical specs of the project? What will the final deliverable look like? How will you measure success?

I have already gotten a lot of this working on gaideo.com. I will update below the done and planned work.

Budget and Milestones

What grant amount are you seeking? How long will the project take in hours? If more than 20, please break down the project into milestones, with a clear output (e.g., low-fi mockup, MVP with two features) and include the estimated work hours for each milestone.

I am seeking the full $5000 grant amount.

I have really been excited about this project and have worked through the long weekend to reach what I can consider the first milestone. Now that I have built the app out a little bit, I have a much more concrete vision on the ultimate finished project. I will attempt to outline in bullet form the steps that I have already undergone and the future steps I plan to take. Hopefully I can get some grant money along the way 👍

Completed Work:
Learn React. I had never developed with React before and it took a bit to get up to speed, but I love it now since it is making it so easy to host and is so powerful.
Learn how to encrypt an mp4 video using ffmpeg into HLS format
Build docker image that does all of the video encryption work and makes it easy to upload
Build a phase one React application that allows the user to publish encrypted videos using blockstack's login and Gaia storage
Implement a way to list in the UI all of the videos that have been uploaded
Implement an hls.js video player to play the encrypted video with a hook to retrieve the encrypted key file from Gaia storage
Implement a way to delete an existing video
Host https://gaideo.com and deploy the web site
Host the video encryption image on docker hub. https://hub.docker.com/repository/docker/gaideo/gaideo-encrypt
Revisit the UI to make it more responsive to smaller form factors like mobile
Figure out how to make the video player re-sizable to the parent
Implement a way to edit the title/description/keywords for an existing video

That was a lot of time and effort and some money to do that so far. Feel free to check it out for yourself to see how it works but I have bigger plans for this now that I see the potential. So here is the next set of steps:

Planned Work:
Come up with a cool logo for Gaideo (might need to hire out for this)
Create a file system cache for faster load times and sort-ability of images and videos
Add the ability to upload and encrypt images
Add the ability to browse encrypted images
Allow users to create albums and associate one or more video/image with in order to organize and filter better
Add the ability to create a Community with a shared password
Allow user to update profile information for things like a friendly name and avatar icon
When publishing either images or videos, allow user to specify "Personal" or select a "Community"
If images or videos are part of a community use the Community password to encrypt the video key file, video title/desc, and the images
Improve UI for browsing images and videos to show by each Community and Personal
Create a slide show UI for images
Add a way to invite another blockstack user to your Community. The invited user, if accepting the invitation, would need to supply the Community password the first time viewing the Community's data
Implement a comments section for a Community video/image encrypted by the Community password
Really focus on tightening up the UI and making it look as professional as possible (might need to hire out for this)
Tell everyone to stop using Facebook and YouTube to get back control of your personal data

Who is building this? What relevant experience do you bring to this project? Are there skills sets you are missing that you are seeking from the community?

I am building this alone for the most part. In phase three I will seek to find help with making the UI look and function better. I have 26 years of development experience across multiple languages: C#, angular, typescript/javascript, C++, and Java. I am the CTO of Assert security (https://assertsecurity.io). I have strong track record of executing my ideas.

Risks

What dependencies or obstacles do you anticipate? What contingency plans do you have in place?

I am relying on the getFileUrl for streaming the encrypted video from Gaia storage. That is the only part I have not prototyped enough to be sure it works they way I think it does.

Community and Supporting Materials

Do you have previous projects, code commits, or experiences that are relevant to this application? What community feedback or input have you received? How do you plan to share your plan to the community over time and as the final deliverable?

I have recently started to engage with the Discord community more to get up to speed. I have talked some about this idea with community members in the Discord voice chat. I would look for feedback from the community for the best way to communicate progress.

[friedger]

I like the project and a decentralized video platform is on all blockstack app wish lists 👍

See below for more comments:

Project Overview

Uploaded videos will be queued for processing, which will involve encrypting the video, storing the encrypted video data in Gaia storage,

Blockstack apps usually do not give servers access to their gaia storage. As a first product, I would do the processing on the client (or offer the encoding as a service on a server and the deliver the processed file to the client), then the client will store the files on their gaia and signal the server the location of the video

and access control data in a database, and finally deleting the unencrypted video.
Access control is usually done through encryption

Scope
For a first project, I would reduce the scope by removing the network part and focus on private streaming.

Budget and Milestones
The technical feasibility study you mention in the risks is a good first milestone. Something like: upload a hls folder based on https://github.com/REBL-Stack/Filebrowser/ and playback using hls.js

Please attach budgets to each milestone (@blocks8 or how are grants paid out?)

Community and Supporting Materials
An link to the app wish list:
https://forum.blockstack.org/t/big-list-of-app-ideas/6816/11

Some technical links:
https://qvault.io/2020/07/28/hls-video-streaming-with-node-js-a-tutorial/
https://hls-js-dev.netlify.app/api-docs/

[models-72]

I made some adjustments based upon your feedback

[friedger]

Here is a demo of unencrypted stream hls video hosted on gaia: https://hls-js-dev.netlify.app/demo/?src=https%3A%2F%2Fgaia.blockstack.org%2Fhub%2F1NrUHmxV2ABHTugfDCyEcJVdQUNxCt99Wy%2Fvideo%2Findex.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsImR1bXBmTVA0IjpmYWxzZSwibGV2ZWxDYXBwaW5nIjotMSwibGltaXRNZXRyaWNzIjotMX0=

[models-72]

I made some good progress this weekend. I have so far been able to test streaming an unencrypted HLS video using Gaia, as well as a fully encrypted video hosted on my local machine. I'm trying to tie it all together with some reasonable UI but it is coming along nicely. It looks like I will be able to achieve everything with one React application and a companion docker image that does the work of encrypting .mp4 videos. I have some ideas around how to index the information so that the videos can be shared more easily, but I would like to get some ideas from the community around that. I would welcome advice on the best way to communicate progress and if there is any more information lacking in the grant proposal. Also on discord my user name is shardeman.

[blocks8]

Review Committee Feedback from 9/8/2020: We'd like more information on your proposal.

Thank you for your application! You have a clear vision of an application that you can bring to the world and we're excited to see your plans. We need a bit more information about your plans to make sure we have the clear deliverables.

We suggest breaking down the first milestone and associated grant funding into a smaller scope, as was mentioned by @friedger above too. That milestone would need the associated funding and the technical specs laid out for the deliverable. For example, you could break the deliverable down into specific pieces:

1. Identify or build open source video player that can decrypt/encrypt video
2. Build the video players allows upload and playback of end to end encrypted video.
3. Video player has a good user experience running various tests (ie: no lag, or issues with frame rate).

The above are suggestions, so please scope to your project, but the level of detail is important to help scope the deliverable and know when it’s successfully completed. After the initial milestone and deliverable are met, you could consider applying for additional funding to build on the initial piece.

If you would like to update your proposal with the above feedback, you may do so as a reply to this comment or edit the original application entry above.

[models-72]

I have really been excited about this project and have worked through the long weekend to reach what I can consider the first milestone. Now that I have built the app out a little bit, I have a much more concrete vision on the ultimate finished project. I will attempt to outline in bullet form the steps that I have already undergone and the future steps I plan to take. Hopefully I can get some grant money along the way 👍

• Learn React. I had never developed with React before and it took a bit to get up to speed, but I love it now since it is making it so easy to host and is so powerful. (Complete)
• Learn how to encrypt an mp4 video using ffmpeg into HLS format (Complete)
• Build docker image that does all of the video encryption work and makes it easy to upload (Complete)
• Build a phase one React application that allows the user to publish encrypted videos using blockstack's login and Gaia storage (Complete)
• Implement a way to list in the UI all of the videos that have been uploaded (Complete)
• Implement an hls.js video player to play the encrypted video with a hook to retrieve the encrypted key file from Gaia storage (Complete)
• Implement a way to delete an existing video (Complete)
• Host https://gaideo.com and deploy the web site (Complete)
• Host the video encryption image on docker hub. https://hub.docker.com/repository/docker/gaideo/gaideo-encrypt (Complete)

That was a lot of time and effort and some money to do that so far. Feel free to check it out for yourself to see how it works but I have bigger plans for this now that I see the potential. So here is the next set of steps:

• Come up with a cool logo for Gaideo (might need to hire out for this)
• Revisit the UI to make it more responsive to smaller form factors like mobile
• Figure out how to make the video player re-sizable to the parent
• Implement a way to edit the title/description/keywords for an existing video
• Add the ability to upload and encrypt images
• Add the ability to browse encrypted images
• Add the ability to create a group with a shared password
• When publishing either images or videos, allow user to specify "Personal" or select a "Group"
• If images or videos are part of a group use the group password to encrypt the video key file, video title/desc, and the images
• Improve UI for browsing images and videos to show by each Group and Personal
• Create a slide show UI for images
• Add a way to invite another blockstack user to your group. The invited user, if accepting the invitation, would need to supply the group password the first time viewing the group's data
• Implement a comments section for a group video/image encrypted by the group password
• Really focus on tightening up the UI and making it look as professional as possible (might need to hire out for this)
• Tell everyone to stop using Facebook and YouTube to get back control of your personal data

[MarvinJanssen]

I see that you updated your description per @friedger's feedback. I am wondering how you are going to deal with the Gaia upload? As pointed out by @friedger, a server component will not have write access to a user's Gaia bucket. You mention that you have produced a Docker image to facilitate the HLS encryption process. I assume this means that anyone that wishes to upload a video will have to set up Docker and go through the motions? Requiring Docker would raise the bar for content creators significantly.

Have you looked at WebAssembly builds of what you need? For example:

• https://github.com/Kagami/ffmpeg.js/
• https://github.com/flash1293/aes-wasm

[models-72]

In the original write-up I had not built much and did not fully understand the technologies I was dealing with. This is why I thought I needed a server component. Now I no longer plan to have a server component at all outside of hosting the React application. I will checkout ffmpeg.js and see what can be done there. It would be cool if it could be done without docker.

[friedger]

I assume this means that anyone that wishes to upload a video will have to set up Docker and go through the motions? Requiring Docker would raise the bar for content creators significantly.

Solving that could be a proposal by its own.

@models-72 As the MVP is already functioning, I suggest to update this proposal (or create a new one) with the focus on one or two topics:

• group sharing and invitations
• UI improvements
• comments
• marketing

With 2e3c555 it is now clarified that only forward looking projects are awarded a grant.

[friedger]

@models-72 Where do you host the source code?

[models-72]

Its only on my local machine at the moment. I just looked at the history and the project is 8 days old so I feel pretty good about my progress so far in that amount of time. I have been spending 10-14 hour days working on it though instead of my day job. I need to find a place to host it though because I would be bummed if my machine died and I lost it all

[models-72]

I have created a public github repository for all of the code to date for gaideo.com:

https://github.com/gaideo/gaideo.git

[RaffiSapire]

Review Committee Feedback from 9/15/2020: We'd like more information on your proposal.

Hi @models-72, I'm Raffi, I am working with the foundation on the grant program. I also used to work at livepeer, which is building p2p video encoding infrastructure so I'm excited to see this. I think it would be easiest if we hop on a call and talk through and scope this proposal. Can you please let me know your email address and share times you are available on Friday for a call? We'll schedule time to sync up.

[models-72]

Hi Raffi. I replied with my contact info but did not realize it was going into the comments so I hid the comment. Hopefully you still got the email with my response. If not let me know.

[RaffiSapire]

Hello! Thanks very much for taking the time to talk about this over the phone today. We're super excited about the prospect of building an encrypted video-sharing app that anyone can use. In the spirit of transparency, we wanted to share a summary from our phone call today. We talked about really honing in on this piece to start. As a result, during this research phase, a key question is, do we need a server, or not? Can we take ffmpeg.js and use it on the client? Can I produce the same artifacts that I'm making in the docker image? We talked about starting with a web app before working on mobile compatibility, and moving into the sharability features (which you've already made great progress on) as a 2nd phase milestone.

[models-72]

Updated Scope for 1st Milestone

• Investigate viability of using ffmpeg.js to encrypt an .mp4 video file into encrypted HLS on a desktop computer
• Implement solution if possible
• If not possible, enhance the existing docker image to have a server component so that files can be uploaded and encrypted
• The encrypted data would need to be temporarily hosted so that the client app could download the files and upload to the user's Gaia storage.
• Find a cloud provider to host the docker image

[models-72]

Progress Update

I have been able to build a special version of ffmpeg.js, which has allowed me to prove the ability to encrypt .mp4 files in the browser. The performance is a good bit slower but it works. I still have work to do to get it out on gaideo.com but the proof point is there that it can be done without having to host the encryption on a server.

[MarvinJanssen]

@models-72 I find it really impressive how fast you are moving forward with this. I think you should scope out your final grant proposal and then wait for approval. Grants do not normally cover work that has already been done ("Grants are generally forward looking. Work created in the past is not eligible for a grant.") You have more than proved that you can deliver on this project.

Regarding performance, did you try multi-threading? (ffmpegwasm supports this with parameter -threads NUM).

[models-72]

Thanks Marvin for the compliment. I have never really understood the rules of the grant game since I started this project. I always felt like demonstrating progress is better than waiting for money and its more fun. My impression from the conversations I have had is they are still figuring out the criteria for how to pay these grants and are probably trying to be extra careful not to waste the money. Anyways, I have not tried setting the multi-threading parameter, which I will be doing after I finish typing this!

[MarvinJanssen]

I should have probably worded my comment better. By "waiting", I definitely did not mean to imply to sit around and do nothing. It is due to the fact that some rules are a bit open to interpretation. I am certain that you will receive a grant if I can base myself on the responses you have gotten so far.

[models-72]

Performance Update

So initially I had support for ffmpeg.js which is pure javascript built from ffmpeg using emscripten. After Marvin's comment I implemented support for ffmpeg as a web assembly to check the performance benefits and play around with thread counts. I found that there is some improvement to performance with web assembly. For a 17 second high definition video with a size of 34 MB the web assembly finished in 7 minutes versus 10 minutes for the pure javascript. The thread count in web assembly did not seem to make any difference. The native ffmepg was still significantly faster finishing in seconds instead of minutes.

Also while testing I found that the crypto happening in ffmpeg whether from javascript or web assembly was messing up after the first segment. Using crypto from the docker image which runs ffmpeg natively did not have this problem. That was kind of a bummer but thankfully I was able to make it work by letting ffmpeg produce all segments unencrypted and then using aes via the blockstack library to encrypt the segments and update the .m3u8 before uploading to Gaia.

Overall, I like the three pronged approach to encrypting videos. For compatibility, it is good to have pure javascript and would be adequate for smaller videos. For medium-sized videos, you would want to at least be using a browser that supports web assembly and pthreads. And then if someone was wanting to do movie length high def videos it would be silly not to learn how to run the docker image to save time.

[RaffiSapire]

Thanks for the updates along the way. Sorry it took me a while to respond, I've been moving cross country + jewish holidays. Based on our call last week, and feedback from Jude, below is the scope of the grant that is approved, oriented around functionality and features, vs research. The next step will be to send you over a contract so we can disburse payment at the given milestones, please email grants@stacks.org with your contact information.

M1: $2,000 Encrypted upload. Take any video file on the computer as a MP4, encrypt the file client-side, and upload the encrypted file to Gaia. All encoding and encryption must happen in the browser. The uploaded data should be structured such that a client can stream the encrypted video as it downloads it, in order to meet the criteria in Milestone 2.

M2. $2,000 Playback. The browser client can download, decrypt, and play a previously-uploaded file, given a URL to it. The download should be streamed, and should be seek-able — the client should not be required to download and encrypt the entire file before viewing it. This implies using a data representation that is (1) chunked, and (2) encoded at multiple bit-rates.

M3. UI. $1,000 UX. The app allows individual users to create and see playlists with a video thumbnail. Given another user’s BNS username, the browser client can load up and render a page with that user’s list of videos and any playlists they have created (playlists may link to other peoples’ videos, not just their own). The user may opt to create unlisted videos that do not show up in this page, but may be loaded directly via a URL (per milestone 2).

[blocks8]

Funded $2,000 for development of Milestone 1.
M1: $2,000 Encrypted upload. Take any video file on the computer as a MP4, encrypt the file client-side, and upload the encrypted file to Gaia. All encoding and encryption must happen in the browser. The uploaded data should be structured such that a client can stream the encrypted video as it downloads it, in order to meet the criteria in Milestone 2.

[models-72]

As of this morning I have completed all three milestones. Below is a re-post from discord with more details:

FYI: I just did a pretty big push to gaideo.com. I had to change the data access enough to where you should use the people menu to delete all of your existing data if you ever made any on the site. The new release has encryption from browser, you can add friends, share your media with them, and create playlists from media spanning your content and your friends content. All of the data is stored encrypted only on the content creator's Gaia storage and access is fully controlled by them. Also, I pushed a bunch of code to https://github.com/gaideo/gaideo. If you are curious about ffmpeg.js, web workers, indexdb caching, public/private key sharing of data, and probably some other stuff I can't recall check it out. The UI is responsive so should work on mobile but there is currently a known issue with not being able to play HLS on iPhone which after doing some research will require a WebView native app to solve. I think I will propose that as another grant if people are interested.

[RaffiSapire]

Hey @models-72 could you please share the link where anyone here can 1) encrypt a MP4 file with 1 click, (not using the command line or any code) 2) share the file with someone (not using command line)?

[models-72]

https://gaideo.com/

Encrypt

1. Click on Publish
2. Enter a title for your video
3. Click Next
4. Either click the upload circle and choose a .mp4/.mov/.avi file, or on a desktop you can drag the files onto the circle
5. Click Finish

There is progress as it encrypts and uploads. When it is done you should be able to go to Videos and play your video.

Sharing

Adding a friend

1. When it is the first time sharing with someone you will need to add them as a friend
2. Click on the Account menu in the upper right corner and select the Friends menu
3. Click on the plus icon
4. You have to enter a blockstack user name. If the name ends with ".id.blockstack" you can omit that part and it will automatically figure that out.
5. Click the Add button to finishing adding your new friend. Note that your friend must have logged into gaideo.com before or it will not allow it. This limitation is part of the blockstack framework's lookupProfile api since users are only tracked once they have authenticated with an application.

Sharing

Once you have a friend you can share stuff with them.

1. Find the video you want to share in the Videos section.
2. Click on the vertical dots menu and click Share
3. A dialog will popup with a list of your friends
4. Check the friend(s) you want to share this video with and click OK

Viewing a shared video

1. Click on the Videos section
2. Click on the Account menu and click the Friends menu
3. In the drop down you can select one or more friends you want to view videos from
4. Choose friend(s) who have shared a video with you

You can add any video to a playlist, which will let you group videos together from your friends list and your own content. To do this use the vertical dots menu to Add to playlist. You will need to create a playlist first using the Account menu and then clicking on Playlists. The UI flow is similar to adding a friend when adding a playlist.

Also remember that video playback will not work on an iPhone due to IOS not supporting Media Source Extensions. We need to make a small web view app to solve this limitation. And finally, I am happy to receive any feedback on bugs or usability. Some of the UI is still under churn. I plan on changing Publish to Upload and will be adding some searching capability to make it easier to locate videos and not have to browse through a long list if there are lots of videos.

[ryanarndtcm]

Hey there! We are getting technical review done on our end but I wanted to update you here so you knew what was happening. Apologies for the delay @models-72 !

[RaffiSapire]

Hello @models-72 , thank you for presenting last week and for your awesome contribution. The remainder of the grant will be disbursed if it hasn't already. Are you interested in building on this project? If so, you can submit a new grant.

[models-72]

I have really enjoyed working on this project and I have several ideas for ways to add more value to what is currently built. I would like to get a basic idea on the most interesting thing to start with from the community before writing up the full grant request. I can post this in the discord chat as well but here is a list of items I think would be good to focus on:

• Separate re-usable code into their own repositories
  - gaia-video-player,
  - video encrypt/transcode utilities
  - library for saving files to gaia that allow for searching and sharing with others (this might need to be worked out with existing solutions like Radiks)
• IOS web view application to support running on an iPhone. Currently there is a problem showing encrypted videos on IOS since it does not support Media Source Extensions. An IOS web-view application would allow for intercepting requests to the media files including the key file which we could then request instead of the "browser" and decrypt using our stacks application private key
• More configuration for video transcoding. gaideo.com makes a lot of assumptions about the quality of video that gets transcoded in order to optimize transcoding time. Currently videos that have a vertical resolution higher than 720 will get set to 720 during transcoding. We can make this more flexible in the profile settings allowing the user to pick the quality they want.
• Create comment/emojii section for media. There is no way to respond to another user's media and let them know what they think either with a comment or emojii. This feature has its own challenges since each comment/reaction would be owned by the user in their gaia storage and would need to be aggregated together
• Desktop app (electron?) for backup/restore, native ffmpeg. A native application would allow for better access to third party tools like ffmpeg and direct file system access. Transcoding/encryption could be performed much faster in this mode and the user would be able to backup/restore all of their gaideo.com media which would open up offline sharing support for their media files. If gaideo.com ever got taken down people could have backups of all of their data in a format that would still be usable on other domains using the same gaideo.com source code or simply view locally on the native application.

[dantrevino]

@models-72 have you looked into using Radiks for the social/sharing parts? Also, I think you mentioned being able to share audio as well. I might be making that up, but we tried last week and it wasn't working. If audio sharing is not on the roadmap, is that something you can add?

[ryanarndtcm]

@models-72 https://forum.stacks.org/c/Stacks-Foundation/stacks-grants/56 forum is a great spot for brainstorming and community feedback too - especially as soon we will be moving over to grant.io thanks!

[models-72]

I have posted my wall of text in the new forum. I will answer dantrevino's questions there

[RaffiSapire]

Thanks @models-72 - let us know if its helpful to get on the phone and chat through anything, or if you want to submit another grant to work on any of what you've proposed, then we're excited to continue to support you!

[models-72]

Some of the feedback I received from dantrevino and tarun was wanting to be able to use Gaideo for encrypted music streaming. I had not thought of this in my original list of ideas but it does seem like a very useful media type to have. Is this something that the Stacks foundation would support for a grant? If so I will write up a grant for that since the need was expressed by 2 of the community members. If not I will pick something from my original list.

[MarvinJanssen]

👍 for audio streams.

[vanesvibes]

@models-72 Vane here! I am supporting the Grants in updating information. Could you let us know your discord username?

[models-72]

shardeman#7211

[will-corcoran]

Hi @models-72

I am in the process of pulling together an index of all completed grants and links to the work (code repos, presentations, etc) created as a result of the grant. Would you be so kind as to post links here to the work you created as a result of this grant?

Thanks!
Will Corcoran
Stacks Foundation Grant Program Manager