Skip to content

Commit

Permalink
Solve the problem of authorization failure caused by multi thread upload
Browse files Browse the repository at this point in the history
  • Loading branch information
gaowanliang committed Apr 2, 2021
1 parent 622c196 commit 18623f7
Show file tree
Hide file tree
Showing 4 changed files with 64 additions and 26 deletions.
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
## Project significance
This project is mainly to use small hard disk server for offline downloading, for large BitTorrent files to be downloaded in sections according to the size of the hard disk, each time downloading a part, then uploading the network disk, delete and then download the other parts, until all the files are downloaded.

At the same time, communication via the bot protocol facilitates use on machines that cannot intranet penetration, and simplifies the usual use of download programs for added convenience.For links, sending a message directly to the Bot will directly identify and download them. It can actually delete files from the download folder, which is not possible with web panels such as AriaNG, and is very convenient as a tool for managing downloads and notifying timely completion of downloads. You can move files, and for users who mount their hard drives via rclone you can copy and paste directly through this program, without having to open an ssh connection to the VPS for cp operations, which is also very convenient.
At the same time, communication via the bot protocol facilitates use on machines that cannot intranet penetration, and simplifies the usual use of download programs for added convenience.For links, sending a message directly to the Bot will directly identify and download them. It can actually delete files from the download folder, which is not possible with web panels such as AriaNG, and is very convenient as a tool for managing downloads and notifying timely completion of downloads. You can move files, and for users who mount their hard drives via rclone you can copy and paste directly through this program, without having to open an ssh connection to the VPS for `cp` operations, which is also very convenient.


## Functions realized
Expand All @@ -34,6 +34,8 @@ At the same time, communication via the bot protocol facilitates use on machines

#### The Bot protocol supports
- [x] Telegram Bot
- [ ] Support multi-user use
- [ ] Support group use
- [ ] Tencent QQ (Use regular QQ users to interact)
- [ ] DingTalk Bot

Expand Down Expand Up @@ -127,8 +129,15 @@ At the same time, communication via the bot protocol facilitates use on machines
#### Corresponding explanations
* aria2-server : Aria2 server address. Websocket connection is used by default. If you want to use websocket to connect to aria2, be sure to set `enable-rpc=true` in `aria2.conf`. If not necessary, please try to **set the local aria2 address**, in order to maximize the use of this program
* aria2-key : The value of `rpc-secret` in `aria2.conf`
* bot-key : ID of telegram BOT
* user-id : The ID of the administrator
* bot-key : ID of telegram Bot, get it by using [@BotFather](https://telegram.me/botfather)
* user-id : The ID of the administrator. It supports setting multiple users as administrators. Different users are separated by commas `,` . If you want to set the users whose `user-id` are 123465789, 987654321 and 963852741 as administrators, you need to set them as follows:
```json
{
···
"user-id": "123456789,987654321,963852741",
···
}
```
* max-index:Maximum display quantity of download information, 10 pieces are recommended (to be improved in the future)
* sign: Identification of this Bot, If multiple servers are required to connect to the same Bot, the specific server can be determined through this item.
* language: Language of Bot output
Expand Down
15 changes: 12 additions & 3 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

这个项目主要就是利用吃灰小盘vps进行离线下载,对于大bt文件进行根据硬盘大小分段下载,每次都下载一部分,然后上传网盘,删除再下载其他部分,直到下载完所有文件。

同时,通过机器人协议通信,方便在无法进行内网穿透的机器上进行使用,而且简化了平时使用下载程序的操作,提高了便利性。对于链接,直接向Bot发送消息就可以直接识别并下载,可以真正删除下载文件夹里的文件,是AriaNG等web面板无法做到的,作为管理下载的工具,及时通知下载完成都是非常的方便的。可以移动文件,对于通过rclone挂载硬盘的用户可以直接通过本程序进行复制粘贴等操作,无需打开ssh连接VPS进行cp操作,也非常的方便。
同时,通过机器人协议通信,方便在无法进行内网穿透的机器上进行使用,而且简化了平时使用下载程序的操作,提高了便利性。对于链接,直接向Bot发送消息就可以直接识别并下载,可以真正删除下载文件夹里的文件,是AriaNG等web面板无法做到的,作为管理下载的工具,及时通知下载完成都是非常的方便的。可以移动文件,对于通过rclone挂载硬盘的用户可以直接通过本程序进行复制粘贴等操作,无需打开ssh连接VPS进行`cp`操作,也非常的方便。

## 实现

Expand All @@ -34,6 +34,8 @@
#### 机器人协议支持

- [x] Telegram Bot
- [ ] 支持多用户使用
- [ ] 支持群组内使用
- [ ] 腾讯QQ(使用普通QQ用户来进行交互)
- [ ] 钉钉机器人

Expand Down Expand Up @@ -133,8 +135,15 @@
* aria2-server:aria2服务器地址,默认使用websocket连接。如果要使用websocket连接aria2,请务必设置`aria2.conf`内的`enable-rpc=true`
。如果不是必须,请尽量设置本地的aria2地址,以便于最大化的使用本程序
* aria2-key:`aria2.conf``rpc-secret`的值
* bot-key:Telegram Bot的标识
* user-id:管理员的ID
* bot-key:Telegram Bot的标识,通过 [@BotFather](https://telegram.me/botfather)进行获取。
* user-id:管理员的ID,支持设置多用户为管理员,不同的用户之间使用半角逗号`,`分割。如您要设置`user-id`为123465789、987654321和963852741的用户为管理员,您需要这样设置:
```json
{
···
"user-id": "123456789,987654321,963852741",
···
}
```
* max-index:下载信息最大显示数量,建议10条(以后会改进)
* sign:此机器人的标识,如果需要多个服务器连接同一个机器人,通过这一项可以确定具体是哪一台服务器
* language:机器人输出的语言
Expand Down
30 changes: 20 additions & 10 deletions src/onedrive/onedrive.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,24 @@ import (
"sync"
)

var bearerToken = ""
var userID = ""

func ApplyForNewPass(url string) string {
return httpLocal.NewPassCheck(url)
}

func Upload(infoPath string, filePath string, threads int, sendMsg func() func(text string), locText func(text string) string) {
userID, bearerToken = httpLocal.GetMyIDAndBearer(infoPath)
func refreshToken(infoPath string) (string, string, string) {
userID, bearerToken := httpLocal.GetMyIDAndBearer(infoPath)
username := strings.ReplaceAll(filepath.Base(infoPath), ".json", "")
return userID, bearerToken, username
}

func Upload(infoPath string, filePath string, threads int, sendMsg func() func(text string), locText func(text string) string) {
programPath, err := filepath.Abs(filepath.Dir(infoPath))
if err != nil {
log.Panic(err)
}
infoPath = filepath.Base(infoPath)
infoPath = filepath.Join(programPath, infoPath)

// restoreOption := "orig"
oldDir, err := os.Getwd()
if err != nil {
Expand All @@ -49,7 +57,7 @@ func Upload(infoPath string, filePath string, threads int, sendMsg func() func(t
} else {
restore(restoreSrvc, fileInfoToUpload, threads)
}*/
restore(restoreSrvc, fileInfoToUpload, threads, sendMsg, locText, username)
restore(restoreSrvc, fileInfoToUpload, threads, sendMsg, locText, infoPath)
err = os.Chdir(oldDir)
if err != nil {
log.Panic(err)
Expand All @@ -60,7 +68,7 @@ func changeBlockSize(MB int) {
}

//Restore to original location
func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, threads int, sendMsg func() func(text string), locText func(text string) string, username string) {
func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, threads int, sendMsg func() func(text string), locText func(text string) string, infoPath string) {
var wg sync.WaitGroup
pool := make(chan struct{}, threads)
for filePath, fileInfo := range filesToRestore {
Expand All @@ -73,6 +81,7 @@ func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileu
}()
temp := sendMsg()
temp("`" + filePath + "`" + locText("startUploadOneDrive"))
userID, bearerToken, username := refreshToken(infoPath)
_, err := restoreSrvc.SimpleUploadToOriginalLoc(userID, bearerToken, "rename", filePath, fileInfo, temp, locText, username)
if err != nil {
log.Panicf("Failed to Restore :%v", err)
Expand All @@ -99,7 +108,7 @@ func printResp(resp interface{}) {
}

//Restore to Alternate location 还原到备用位置
func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, sendMsg func() func(text string), locText func(text string) string) {
func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, sendMsg func() func(text string), locText func(text string) string, infoPath string) {
rootFolder := fileutil.GetAlternateRootFolder()
var wg sync.WaitGroup
pool := make(chan struct{}, 10)
Expand All @@ -114,8 +123,9 @@ func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[stri
}()
temp := sendMsg()
temp(filePath + "开始上传至OneDrive")
us:=""
_, err := restoreSrvc.SimpleUploadToAlternateLoc(userID, bearerToken, "rename", rootFilePath, fileItem, temp, locText,us)
us := ""
userID, bearerToken, _ := refreshToken(infoPath)
_, err := restoreSrvc.SimpleUploadToAlternateLoc(userID, bearerToken, "rename", rootFilePath, fileItem, temp, locText, us)
if err != nil {
log.Panicf("Failed to Restore :%v", err)
}
Expand Down
30 changes: 20 additions & 10 deletions vendor/onedrive/onedrive.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,24 @@ import (
"sync"
)

var bearerToken = ""
var userID = ""

func ApplyForNewPass(url string) string {
return httpLocal.NewPassCheck(url)
}

func Upload(infoPath string, filePath string, threads int, sendMsg func() func(text string), locText func(text string) string) {
userID, bearerToken = httpLocal.GetMyIDAndBearer(infoPath)
func refreshToken(infoPath string) (string, string, string) {
userID, bearerToken := httpLocal.GetMyIDAndBearer(infoPath)
username := strings.ReplaceAll(filepath.Base(infoPath), ".json", "")
return userID, bearerToken, username
}

func Upload(infoPath string, filePath string, threads int, sendMsg func() func(text string), locText func(text string) string) {
programPath, err := filepath.Abs(filepath.Dir(infoPath))
if err != nil {
log.Panic(err)
}
infoPath = filepath.Base(infoPath)
infoPath = filepath.Join(programPath, infoPath)

// restoreOption := "orig"
oldDir, err := os.Getwd()
if err != nil {
Expand All @@ -49,7 +57,7 @@ func Upload(infoPath string, filePath string, threads int, sendMsg func() func(t
} else {
restore(restoreSrvc, fileInfoToUpload, threads)
}*/
restore(restoreSrvc, fileInfoToUpload, threads, sendMsg, locText, username)
restore(restoreSrvc, fileInfoToUpload, threads, sendMsg, locText, infoPath)
err = os.Chdir(oldDir)
if err != nil {
log.Panic(err)
Expand All @@ -60,7 +68,7 @@ func changeBlockSize(MB int) {
}

//Restore to original location
func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, threads int, sendMsg func() func(text string), locText func(text string) string, username string) {
func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, threads int, sendMsg func() func(text string), locText func(text string) string, infoPath string) {
var wg sync.WaitGroup
pool := make(chan struct{}, threads)
for filePath, fileInfo := range filesToRestore {
Expand All @@ -73,6 +81,7 @@ func restore(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileu
}()
temp := sendMsg()
temp("`" + filePath + "`" + locText("startUploadOneDrive"))
userID, bearerToken, username := refreshToken(infoPath)
_, err := restoreSrvc.SimpleUploadToOriginalLoc(userID, bearerToken, "rename", filePath, fileInfo, temp, locText, username)
if err != nil {
log.Panicf("Failed to Restore :%v", err)
Expand All @@ -99,7 +108,7 @@ func printResp(resp interface{}) {
}

//Restore to Alternate location 还原到备用位置
func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, sendMsg func() func(text string), locText func(text string) string) {
func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[string]fileutil.FileInfo, sendMsg func() func(text string), locText func(text string) string, infoPath string) {
rootFolder := fileutil.GetAlternateRootFolder()
var wg sync.WaitGroup
pool := make(chan struct{}, 10)
Expand All @@ -114,8 +123,9 @@ func restoreToAltLoc(restoreSrvc *upload.RestoreService, filesToRestore map[stri
}()
temp := sendMsg()
temp(filePath + "开始上传至OneDrive")
us:=""
_, err := restoreSrvc.SimpleUploadToAlternateLoc(userID, bearerToken, "rename", rootFilePath, fileItem, temp, locText,us)
us := ""
userID, bearerToken, _ := refreshToken(infoPath)
_, err := restoreSrvc.SimpleUploadToAlternateLoc(userID, bearerToken, "rename", rootFilePath, fileItem, temp, locText, us)
if err != nil {
log.Panicf("Failed to Restore :%v", err)
}
Expand Down

0 comments on commit 18623f7

Please sign in to comment.