Telegram YouTube Notifier

Introduction

System created using the FastAPI + Python Telegram BOT libraries, to subscribe users on YouTube and notify them if any video on the specified channel is available.

First steps

Some steps are essential for the system to work properly. We will go through each one of them below.

Pubsubhubbub

Notifications are made available through Pubsubhubbub. It establishes a series of rules for you to be notified. You can learn more about it by reading the following articles:

• https://developers.google.com/youtube/v3/guides/push_notifications

• https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html

Environment variables

There are some environment variables that are necessary for the system to communicate, with external "services" (like Telegram and PostgreSQL, for example). All the environment variables that the system uses will be specified below with a brief description of why it is needed:

• TELEGRAM_TOKEN => When creating a bot in Telegram, you will be given a unique token that is necessary to authorize the bot and send requests to the Bot API. Keep your token safe and store it safely, it can be used by anyone to control your bot. You can read a little more about creating bots on Telegram by clicking here.

• CALLBACK_URL => The Pubsubhubbub service requires you to specify a subscriber's callback URL where notifications are to be delivered. It is worth noting that you must also specify the path of the method you are using to handle calls, for example: Imagine that our RESTful API has the URI "http://telegramytnotifier.com", in the case for If we define the environment variable CALLBACK_URL, we must pass the path where the treatment is performed, which could be "http://telegramytnotifier.com/feed". If you don't have a server to host your application for testing, you can use NGROK.

• DATABASE_URL => A simple way to connect to PostgreSQL is to use its "URL". For example, instead of specifying each field separately, we can use it in the following format: "postgres://PostgresUserName:PostgresPassword@PostgresHostname:5432/PostgresDatabaseName". Another interesting fact is that as we use Heroku as a platform to host our project, it requires that access to the bank be done through DATABASE_URL. You can read a little more about it here.

• PORT => Here we specify the port on which our service will operate. Like DATABASE_URL, Heroku also generates a specific port on which our system will run, which is not predictable for us for security reasons.

• POSTGRES_PASSWORD (OPTIONAL) => This variable is more to make the PostgreSQL that we are using in Docker work. If you use our docker-compose.yml for development, you don't have to worry about this variable.

Development

To make your life a little easier if you decide to play with the system a little, everything is ready with Docker. In fact, we use Docker and Docker Compose. Below are some references to help you:

To better understand the tools:

• Get Started with Docker
• Overview of Docker Compose

To install them, just click on the links below:

• Install Docker
• Install Docker Compose

Now that we understand a little better how Docker works, let's move on to the settings. First, you will need a TELEGRAM_TOKEN . You can follow this documentation to learn how to generate one: https://core.telegram.org/bots#6-botfather.

We also need a way to expose our RESTful API to the web. The way I suggest initially is to use NGROK . Its configuration is very easy, just follow this tutorial: https://dashboard.ngrok.com/get-started/setup. Remembering that from the address generated by NGROK, we will be able to set the environment variable CALLBACK_URL . For example: CALLBACK_URL=http://m4ur0gg.ngrok.io/feed . PS: "http://" is required!.

The environment variables DATABASE_URL , PORT and POSTGRES_PASSWORD do not need to be specified if you are using Docker Compose.

Open the docker-compose.yml file and configure in the environment section of the bot , the values ​​we generated from TELEGRAM_TOKEN and CALLBACK_URL . If you do not already have the address generated by NGROK to set CALLBACK_URL , follow these steps:

Start PostgreSQL:

$ docker-compose up --build -d db

Launch the RESTful API:

$ docker-compose up --build api

Use NGROK to expose the RESTful API and obtain the URL:

$ ngrok http 5000

In your terminal, you will see an output very similar to this one:

ngrok by @inconshreveable                                                                                                            (Ctrl+C to quit)
                                                                                                                                                     
Session Status                online                                                                                                                 
Account                       Mauro de Carvalho (Plan: Free)                                                                                         
Version                       2.3.35                                                                                                                 
Region                        United States (us)                                                                                                     
Web Interface                 http://127.0.0.1:4040                                                                                                  
Forwarding                    http://4750cd3fd33b.ngrok.io -> http://localhost:5000                                                                  
Forwarding                    https://4750cd3fd33b.ngrok.io -> http://localhost:5000

Based on the above output, our CALLBACK_URL would be http://4750cd3fd33b.ngrok.io/feed .

There, we already have PostgreSQL and the RESTful API running. We now need to initialize the bot. This step will be as simple as the others:

$ docker-compose up --build bot

Now everything is working for development! Just send messages to the bot you created on Telegram with /subscribe <YouTube Channel ID> and you will be notified as soon as a new video on the desired channel arrives!

Deploy

Our deployment example is centered on the Heroku platform. If you want to use another way, if possible, I ask that you open a PR here in the repository specifying the steps you took to deploy. This will help other people a lot.

Without further ado, let's start learning how to bring the entire system up to production using Heroku!

First of all, create an account on the platform. You can access the following URL for this: https://signup.heroku.com/.

After you have created your account and verified it, we will create a new app.

• Click on the "Create new app" button;
• Create a very cool name for your app;
• Click on "Create app".

Now that the app is created, we’ll be on the app’s dashboard. We will now add the PostgreSQL addon to our project.

• Click on the "Resources" tab;
• In the search bar with "Quickly add add-ons from Elements", search for "Heroku Postgres";
• Select the "Hobby Dev - Free" plan (If you choose paid, no problem, just change);
• Click on "Submit Order Form".

We now have a database! However, we still need to configure some environment variables.

• Click on the "Settings" tab;
• Click on "Reveal Config Vars";
• Note that DATABASE_URL is already present in the environment variables, very practical, right ?;
• Add TELEGRAM_TOKEN as a key and the token generated by Botfather as a value;
• Also add CALLBACK_URL along with your app's URL. Heroku always uses the default: https://<APP NAME>.herokuapp.com/. In our case, remember that we also need to specify the route. CALLBACK_URL will look like this: https://<APP NAME>.herokuapp.com/feed ("/feed" is mandatory because it is the route we use in the RESTful API to deal with Pubsubhubbub!);
• We don't necessarily need to specify the PORT environment variable because heroku already does this "injection "automatically when the project goes into production. Do not worry.

Environment variables successfully configured! We just need to deploy now.

• Go to the terminal;
• Go to the root directory of this project that you have cloned;
• Use the heroku login command to authenticate to heroku;
• Use the command heroku git:remote -a <APP NAME> to "link" the app remotely;
• As we also use Docker to deploy (see file heroku.yml), we need to execute the following command also heroku stack:set container;
• Now we just have to upload the application: git push heroku main;
• The above command will take a while, right after it is finished, we need to "scale" our services;
• First use heroku ps:scale web=1 and then heroku ps:scale bot=1;
• You can follow the logs of your application using the command heroku logs --tail.

That's it. Now we have our service running at full speed on Heroku!

Contributing

Feel free to do whatever you want with this project. :-)

Reference

• https://devcenter.heroku.com/articles/build-docker-images-heroku-yml

• https://betterprogramming.pub/how-to-containerize-and-deploy-apps-with-docker-and-heroku-b1c49e5bc070

• https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html#rfc.section.5

• https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html#verifysub