CloneBot V2 is inspired from MsGsuite's CloneBot, which got out-dated and having too many errors in it. We both created it to keep the legacy of CloneBot alive! The bot who helped thousands for cloning their data.❤️
1. The Powerful Telegram Bot based on Gclone to clone Google Drive's Shared Drive data easily.⚡
2. CloneBot V2 usage Service Accounts to easily clone TBs of data without hitting 750GB Upload/Clone limit of Google Drive.♻️
3. It is most lightweight and performs only server-sided cloning to have very less load on system and don't use your own bandwidth.🗃️
4. Just provide the sharing link of a particular Shared Drive/folder or file and set multiple destination folders to clone data.🔗
Easily navigate through out the guide and learn about Powerful CloneBot V2 and terms related to it.
->🐳Build or Deploy using Docker
->🧿Deploy using GitHub Actions
1.Gclone upgraded to v1.58.1 (latest)!😉
2.Deployment on Okteto & Scalingo Added!🤩
3.Port Error fixed for Okteto deployments!😄
4.Bot can be deployed as both Web or Worker on Heroku.🙃
5.Mitigated Time Zone Error while deployment.🤗
6.Addressed GitHub Workflow Error!😙
7.Docker Image changed!😌
8.Fixed error where CloneBot V2 is unable to show progress!😘
1.You may need account for Heroku/Okteto/Scalingo while deploying CloneBot V2 on respected platforms.
2.Service Accounts are mandatory to use CloneBot V2, because it uses Service Accounts to prevent hitting 750GB Upload/Clone limit of Google Drive while cloning large amount of data.
3.VPS or your local machine (PC or Laptop or Mobile) should have Python 3 and Git installed in order to run CloneBot V2.
4.CloneBot V2 don't use your bandwidth or Internet connection while cloning data but it can if hosted on your local machine or VPS for calling required Telegram APIs to update the progress or to generate required response.
5.This Project comes with GNU License, please consider reading it before using this.
CloneBot V2 is very straight forward and easy to use bot. If you deployed your CloneBot V2 then consider adding commands in it through @BotFather to make it easy for other users to know bot commands, here is the commands list to be set in @BotFather:
1.First convert accounts folder of your Service Accounts into accounts.zip then send it to bot and write /sa in caption or send /sa as reply to accounts.zip file. Don't have Service Accounts? Learn here how to create
2.Now Send /folders to your CloneBot V2 and then bot will show Shared Drives name in which you added your Service Accounts's Google Group, select Shared Drive or directory available in it as destination. Not added Service Accounts in Google Group? Learn here how to do.
3.Your CloneBot V2 is now ready to be used! just send any Google Drive sharing link and select the Destination folder you set before to clone data in it.
4.Additionally, /ban and /unban command is to unauthorize or authorize user again and /id command is to get your Telegram User ID.
⛔NOTE: Each allowed user have to upload their own accounts.zip to use CloneBot_V2.
Deployment of CloneBot V2 is as simple as its usage! Their are many methods listed below to deploy CloneBot easily, but before you deploy it, you need some values listed below and how to get it:
path_to_gclone - Path to gclone file, by default it is gclone or change it if you using different one.
telegram_token - Get your bot's Telegram API Token from BotFather.
user_ids - Telegram User IDs of users who can use your CloneBot_V2. Separate them using , and first User ID is Admin.
group_ids - Telegram Group IDs of Groups in which CloneBot can be used otherwise keep it -1. Separate them using ,
gclone_para_override - Keep it blank if you don't know what it is.
CONFIG_FILE_URL is URL to config.ini file which contains values of variables discussed above, lets see how to get your CONFIG_FILE_URL easily:
1.First open sample-config.ini file then copy its code.
2.Now open GitHub Gist and create a new gist and paste the code you just copied and name it as config.ini and now fill below values carefully:
path_to_gclone - Keep it as gclone.
telegram_token - Enter Telegram Bot Token, get it from @BotFather.
user_ids - Enter User IDs, which you want to allow your CloneBot V2. Separate IDs by , and first ID is Admin.
group_ids - Enter Group IDs, in which you want to allow your CloneBot V2 to be used otherwise keep it -1. Separate IDs by ,
gclone_para_override - Keep it blank if you don't know what it is.
3.Then press Create Secret Gist then click Raw, it will open a New Tab in your Browser. Just copy the URL of that New Tab
Before:
https://gist.githubusercontent.com/UserName/0ee24eXXXXXXXXXXXXXXX6b/raw/Commit_ID/config.ini
After:
https://gist.githubusercontent.com/UserName/0ee24eXXXXXXXXXXXXXXX6b/raw/config.ini
CloneBot V2 can be deployed almost everywhere using Docker, either you can create your own Docker Image using Build Tool provided in the Workflow including Docker-Code. While CloneBot V2 also have ready to use Docker image for systems based on Linux AMD 64.
->docker pull ghcr.io/thecaduceus/clonebot_v2:main
->FROM ghcr.io/thecaduceus/clonebot_v2:main
3.Want to build own docker image? alright! here is the guide.
⛔NOTE: Docker Image only accepts CONFIG_FILE_URL
Heroku is very famous PaaS (Platform as a Service) platform and it provides very simple user environment as well as you can deploy CloneBot V2 very quickly on Heroku to run it continuously for 24 Hours.
App Name - Give a unique name to your Bot.
CONFIG_FILE_URL - Enter CONFIG_FILE_URL which you made here.
3.Click "Deploy" button and once it got deployed, click Manage App then go to Resource Tab and enable the dyno as shown in the image.
⛔NOTE: Heroku provides 550 running hours (dyno hours) per month and it restarts deployed app each 24 Hours.
This methods is really amazing and simple!🥰 You can deploy your CloneBot V2 on Heroku using GitHub Actions itself by just providing your HEROKU API KEY HEROKU EMAIL and HEROKU APPNAME.
1.First fork this repository, now open the settings of your Forked Repository and click Secrets->Actions.
HEROKU_API_KEY - Enter your Heroku API Key as value.
HEROKU_EMAIL - Your Heroku Email ID.
HEROKU_APP_NAME - A unique app name in small letters only.
CONFIG_FILE_URL - CONFIG_FILE_URL you made above for Heroku deployment!
3.Go to Actions Tab then click Deploy to Heroku and Run Workflow. Now it will be automatically got deployed on given Heroku Account!😉
⛔NOTE: Deploying as web? change deploy.yml (Docker Process Type) and add PORT as Repository secret, value can be 8080.
Okteto is also very famous Kubernetes development platforms and used by many users and it is ideal for lightweight apps and it is perfect for CloneBot V2, Okteto don't have any running hours limit like Heroku but your CloneBot V2 will go to sleep if the ENDPOINT provided by Okteto for your CloneBot V2 untouched. Don't worry, I have solution too and some special arrangements.
1.First Create your Okteto Account, You need one GitHub account as okteto supports only one Method to either Create or Login: Create/Login on Okteto
CONFIG_FILE_URL - Enter CONFIG_FILE_URL, which you just made here.
5.Okteto make your deployed app to sleep if provided ENDPOINT (Allotted URL) remain untouched for 24 Hours. So lets setup a simple cron-job to keep your app active.
6.First copy your app's ENDPOINT as shown in the image and go to Cron-Job.org and sign up!
7.Done? Nice! now click "CREATE CRONJOB" button and provide your copied ENDPOINT URL that you just copied and change execution schedule to every 5 minutes.Finally! click "CREATE" and you done it! 😌 Relax and use CloneBot V2 freely.
⛔NOTE: Don't forget to setup Cron-Job for Okteto otherwise your deployed bot will go into sleep and you have to active it from Okteto Dashboard, while Cron-Job doing it on your behalf.
Running CloneBot V2 on your PC or VPS is very simple and takes very less efforts! It have very less load on your System and don't use your bandwidth or Internet connection for cloning Google Drive data but only for calling Telegram APIs to update the progress or to generate required response.
->Python 3 or above with pip
->Git
->git clone https://github.com/TheCaduceus/CloneBot_V2
->Or Download from Here
->cd CloneBot_V2
->pip install -r requirements.txt
->Go to Gclone Library and download Gclone file as per your Operating System and place it in "telegram_gcloner" folder.
->Website provides direct download link, so you can also use Command-line to download Gclone.
Linux:
->curl download_link_here >> telegram_gcloner/gclone
Windows:
->curl download_link_here >> telegram_gcloner/gclone.exe
->Open config.ini file in any text editor and enter the values of variables as written here
Or you can download your config.ini file from external source using CONFIG_FILE_URL by using Command-line:
->curl CONFIG_FILE_URL >> telegram_gcloner/config.ini
->cd CloneBot_V2
->python telegram_gcloner/telegram_gcloner.py
->Press CTRL + C keys
Termux is a best app for running and using Command-line tools on Mobile, CloneBot can also be deploy on your Mobile using Termux itself, don't worry because CloneBot V2 is very lightweight and designed to be deployed even on low-end systems and thus it will not cause heavy load on your Mobile.
1.Download Termux app: Download Here
2.Choose specific code from here based on architecture of your phone.
CloneBot V2 is also deployable to Scalingo cloud, Just deploy Scalingo Branch.
Switch to Scalingo Branch for guide.
Service Accounts are just like normal Google Account and thus have same Upload or Download limits as Google Account which is 750GB Upload and 10TB Download. They are used to act on behalf of a Google Account and hence we can use them to prevent hitting Google Drive limits by creating them in a bulk amount. After creating Service Accounts, we have to add them in Google Group so that we can directly add Google Group's Email ID in Shared Drive at place of adding each Service Accounts manually.
1.First go to Google Cloud Console and select "Create or select a project" then click "CREATE PROJECT" as shown in the image.
3.Once your project is created! then click "SELECT PROJECT". Now click on hamburger menu and hover the cursor on "APIs and services" after which a small drop-down menu list is visible, select "Enabled APIs and services"
4.After it, Click "ENABLE APIS AND SERVICES" button and search for "Google Drive API" in the Search bar as shown in the image.
6.Once Enabled, Click on "OAuth consent screen" then select "External" as "User Type" and click "CREATE" button.
7.It will now open "Edit app registration" screen, provide App Name, Support Email and Developer Email ID (Same as Support Email ID) and then click "SAVE AND CONTINUE" button.
8.Now it will ask you to "ADD OR REMOVE SCOPES", just ignore this and directly click "SAVE AND CONTINUE" button. Then it will ask you to "ADD USERS" again ignore it and directly press "SAVE AND CONTINUE"
10.After publishing, Select "Credentials" and click "CREATE CREDENTIALS", from drop down list select "OAuth Client ID".
11.Choose Application type as "Desktop app" and press "CREATE" button. Now create a Folder on your computer with name like "My Service Accounts", and then from pop-up click "DOWNLOAD JSON". Download the json file as credentials.json in the folder you just created.
⛔NOTE: Download json file as credentials.json only!
12.Once downloaded, now download some required python scripts from here and extract it. Then move gen_sa_accounts.py rename_script.py as well as requirements.txt files to folder in which you downloaded credentials.json.
13.Before we proceed further, please confirm you have installed Python (with pip) carefully. Not downloaded yet? Download Now!
14.All Ready? Type "cmd" in the address bar of folder which you created in STEP 11 and hit ENTER or as an alternative of this, you can use cd command like cd FOLDER_PATH in CMD.
1. pip3 install -U -r requirements.txt - To install requirements.
2. py gen_sa_accounts.py - To get login URL.
16.Running command 2 will give you a Login URL, just copy & paste it in your URL and login using your Google Account and provide all asked permission.
⛔NOTE: Login only with Google account which you used to create Project on Google Cloud Console.
3. py gen_sa_accounts.py --list-projects - To get the ID of your created Project.
4. py gen_sa_accounts.py --enable-services PROJECT_ID - To Enable Services in given project.
5. py gen_sa_accounts.py --create-sas PROJECT_ID - To create Service Accounts.
6. py gen_sa_accounts.py --download-keys PROJECT_ID - To download Service Accounts file.
7. py rename_script.py - To rename Service Accounts file in 1-100 sequence.
⛔NOTE: Replace PROJECT_ID with Project ID which you will get from command 3 and if commands not working then replace py with python.
18.Till now, We have created 100 Service Accounts but we have to do some more work before we take them in our use. Open folder which you created in STEP 11 and you will see accounts folder in it which have your 100 Service Accounts file (json files), now type "Powershell" in address bar of accounts folder or as an alternative you can use cd commands like cd FOLDER_PATH in Powershell.
8. $emails = Get-ChildItem .\**.json |Get-Content -Raw |ConvertFrom-Json |Select -ExpandProperty client_email >>emails.txt
20.Above command collects the EMAIL-ID of all your Service Accounts available in accounts folder into emails.txt file. Move emails.txt file from accounts folder to prevent confusion or any other problem.
21.Last work! we have to add them in a Google Group and have to add that Google Group in a Shared Drive to give read + write permission to all Service Accounts at once. Go to Google Groups and press "Create group" button to create a group.
22.In pop-up, fill up details of your Google Group like Name and Email ID as shown in the image then press "Next". After it, let privacy settings as it is and again click "Next"
23.Once done, it will ask you to "Add Members" in your Group as shown in the image,just ignore it and directly press "Create Group". Now open your Google Group and select "Members" from sidebar and click "Add Members"
24.In the pop-up shown, enable "Directly add members" and open emails.txt file which you got from STEP 19 then copy & paste 10 Email IDs in the field named "Group Managers". In this way! add all 100 Email IDs in your Google Group but only 10 Email IDs at once.
25.After adding all Email IDs of your Service Accounts, now copy the Email ID of your Google Group which looks like XXXXX@googlegroups.com and add it in your Shared Drives as "Manager".
26.Finally! We have created 100 Service Accounts and also added them in Google Group. Each Service Account have 750 GB Upload/Clone limit and 10 TB Download limit that means now we can upload/clone 75 TB and can download 100 TB a day.
Join our Update Channel at Telegram: Join Now!
Directly Contact the Developer using Telegram @HelpAutomatted_Bot
Dr.Caduceus: For making this Project and Guide.
Levi: For Gclone and upgrading it.
wrenfairbank: For the original python script.
smartass08: To adapt the scrip to heroku.
anymeofu: For making the Direct Heroku deployable Version.
Zero-The-Kamisama: To making MsGsuite discover this amazing bot and the detailed instructions.
zorgof: For the termux script.
Aishik Tokdar: For Adding Guide to Deploy on Railway.app , Qovery , Clever Cloud , Scalingo and some other Code Improvements.Also Added Heroku Workflow Deployment Method.
Katarina: For adding the ability to be deployed to Clever Cloud and Scanlingo.
Miss Emily: For adding Support of Okteto Cloud Deployment as well as improving little layout.