Skip to content

Commit

Permalink
add add subcommand and other tweaks for cookiecloud cmd
Browse files Browse the repository at this point in the history
  • Loading branch information
sagan committed Dec 6, 2023
1 parent 305f2b2 commit f2cd50a
Show file tree
Hide file tree
Showing 15 changed files with 352 additions and 105 deletions.
55 changes: 55 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ cookie = "cookie_here" # 浏览器 F12 获取的网站 cookie

配置好站点后,使用 ```ptool status <site> -t``` 测试(```<site>```参数为站点的 name)。如果配置正确且 Cookie 有效,会显示站点当前登录用户的状态信息和网站最新种子列表。

程序支持自动与浏览器同步站点 Cookies 或导入站点信息。详细信息请参考本文档 "cookiecloud" 命令说明部分。

## 程序功能

所有功能通过启动程序时传入的第一个”命令“参数区分:
Expand All @@ -97,6 +99,7 @@ ptool <command> args... [flags]
* parsetorrent : 显示种子(torrent)文件信息。
* verifytorrent : 测试种子(torrent)文件与硬盘上的文件内容一致。
* partialdownload : 拆包下载。
* cookiecloud (v0.1.8+): 使用 [CookieCloud](https://github.com/easychen/CookieCloud) 同步站点的 Cookies 或导入站点。
* sites : 显示本程序内置支持的所有 PT 站点列表。
* shell : 进入交互式终端环境。
* version : 显示本程序版本信息。
Expand Down Expand Up @@ -459,6 +462,58 @@ ptool partialdownload <client> <infoHash> --chunk-size 1TiB --chuck-index 0

该命令的设计目的不是用于刷流。而是用于使用 VPS 等硬盘空间有限的云服务器(分多次)下载体积非常大的单个种子,然后配合 rclone 将下载的文件直接上传到云盘。

### 同步 Cookies & 导入站点 (cookiecloud)

程序支持通过 [CookieCloud](https://github.com/easychen/CookieCloud) 服务器同步站点 Cookies 或导入站点。

要使用此功能,在 ptool.toml 配置文件里添加 CookieCloud 服务器连接信息:

```
[[cookieclouds]]
#name = "" # 名称可选
server = "https://cookiecloud.example.com"
uuid = "uuid"
password = "password"
```

可以添加任意个 CookieCloud 连接信息。如果想要让某个 CookieCloud 连接信息仅用于同步特定站点 cookies,加上 ```sites = ["sitename"]``` 这行配置。

#### 测试 CookieCloud 服务 (status)

```
ptool cookiecloud status
```

使用配置的 CookieCloud 连接信息连接服务器,测试配置正确性和当前服务器状态。

#### 同步站点 Cookies (sync)

```
ptool cookiecloud sync
```

程序会从 CookieCloud 服务器获取最新的 Cookies,并更新 ptool.toml 里已配置的站点的 Cookies。程序会对 ptool.toml 文件里的站点的当前 Cookie 和其从 CookieCloud 服务器获取的新版 Cookie 分别进行测试,只有在当前 Cookie 失效并且新版 Cookie 有效的情形才会更新 ptool.toml 里的站点 Cookie 字段值。

#### 导入站点 (import)

```
ptool cookiecloud import
```

程序会从 CookieCloud 服务器获取最新的 Cookies,筛选出本程序内置支持的站点(```ptool sites```)中当前 ptool.toml 文件里未配置、并且 CookieCloud 服务器数据里存在对应网站有效 Cookie 的站点,然后添加这些站点的配置信息到 ptool.toml 文件里。

import 命令不会检测或更新 ptool.toml 里当前已存在相应配置的站点的 Cookies。

#### 查看 CookieCloud 里的网站 Cookie (get)

```
ptool cookiecloud get <site>...
```

显示 CookieCloud 服务器数据里网站的最新 Cookies。参数可以是站点名、分组名、任意域名或 Url。

默认以 Http 请求 "Cookie" 头格式显示 Cookies。如果指定 ```--format js``` 参数,则会以 JavaScript 的 "document.cookie='';" 代码段格式显示 Cookies,可以直接将输出结果复制到浏览器 F12 开发者工具 Console 里执行以导入 Cookies。

### 交互式终端 (shell)

```ptool shell``` 可以启动一个交互式的 shell 终端环境。终端里可以运行所有 ptool 支持的命令。命令和命令参数输入支持完整的自动补全。
Expand Down
3 changes: 2 additions & 1 deletion cmd/brush/strategy/strategy.go
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,8 @@ func isTorrentStalled(torrent *client.Torrent) bool {
* * Simply limiting downloading speed (to a very low tier) will also drop uploading speed to the same level
* * Consider removing this behavior
* Add new torrents to client when server uploading and downloading bandwidth is somewhat idle AND there is SOME free disk space
*
* Also:
* * Use the current seeders / leechers info of torrent when make decisions
*/
func Decide(clientStatus *client.Status, clientTorrents []client.Torrent, siteTorrents []site.Torrent,
siteOption *BrushSiteOptionStruct, clientOption *BrushClientOptionStruct) (result *AlgorithmResult) {
Expand Down
12 changes: 12 additions & 0 deletions cmd/common/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,17 @@ var OrderFlag = &cmd.EnumFlag{
},
}

func YesNoAutoFlag(desc string) *cmd.EnumFlag {
return &cmd.EnumFlag{
Description: desc,
Options: [][2]string{
{"auto", ""},
{"yes", ""},
{"no", ""},
},
}
}

// pure flag: bool or counter flag. It does not have a value.
// all single-letter name (shorthand) flags are always considered as pure (for now),
// so they are not included in the list.
Expand All @@ -55,6 +66,7 @@ var pureFlags = []string{
"clients",
"delete-added",
"dense",
"do",
"dry-run",
"force",
"force-dangerous",
Expand Down
1 change: 1 addition & 0 deletions cmd/cookiecloud/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package all

import (
_ "github.com/sagan/ptool/cmd/cookiecloud"
_ "github.com/sagan/ptool/cmd/cookiecloud/get"
_ "github.com/sagan/ptool/cmd/cookiecloud/importsites"
_ "github.com/sagan/ptool/cmd/cookiecloud/status"
_ "github.com/sagan/ptool/cmd/cookiecloud/sync"
Expand Down
61 changes: 38 additions & 23 deletions cmd/cookiecloud/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,9 @@ import (
)

type Ccdata_struct struct {
Domain string
Uuid string
Sites []string
Data *CookiecloudData
Label string
Sites []string
Data *CookiecloudData
}

type Cookie struct {
Expand Down Expand Up @@ -73,7 +72,11 @@ func GetCookiecloudData(server string, uuid string, password string, proxy strin
return cookiecloudData, nil
}

func (cookiecloudData *CookiecloudData) GetEffectiveCookie(urlOrDomain string) (string, error) {
// If all is false, only return cookies which is valid for both the hostname and path part of the urlOrDomain,
// in the case of urlOrDomain being a domain, it's path is assumed to be "/".
// If all is true, path check is skipped and all cookies which domain match will be included.
// format: "http" - http request "Cookie" header; "js" - JavaScript document.cookie="" code snippet
func (cookiecloudData *CookiecloudData) GetEffectiveCookie(urlOrDomain string, all bool, format string) (string, error) {
hostname := urlOrDomain
path := "/"
if util.IsUrl(urlOrDomain) {
Expand Down Expand Up @@ -106,7 +109,7 @@ func (cookiecloudData *CookiecloudData) GetEffectiveCookie(urlOrDomain string) (
if cookiePath == "" {
cookiePath = "/"
}
if !strings.HasPrefix(path, cookiePath) {
if !all && !strings.HasPrefix(path, cookiePath) {
continue
}
// cookiecloud 导出的 cookies 里的 expirationDate 为 float 类型。意义不明确,暂不使用。
Expand All @@ -127,26 +130,38 @@ func (cookiecloudData *CookiecloudData) GetEffectiveCookie(urlOrDomain string) (
if len(effectiveCookies) == 0 {
return "", nil
}
sort.SliceStable(effectiveCookies, func(i, j int) bool {
a := effectiveCookies[i]
b := effectiveCookies[j]
if a.Domain != b.Domain {
if !all {
sort.SliceStable(effectiveCookies, func(i, j int) bool {
a := effectiveCookies[i]
b := effectiveCookies[j]
if a.Domain != b.Domain {
return false
}
// longest path first
if len(a.Path) != len(b.Path) {
return len(a.Path) > len(b.Path)
}
return false
})
effectiveCookies = util.UniqueSliceFn(effectiveCookies, func(cookie *Cookie) string {
return cookie.Name
})
}
cookieStr := ""
if format == "http" {
sep := ""
for _, cookie := range effectiveCookies {
cookieStr += sep + cookie.Name + "=" + cookie.Value
sep = "; "
}
// longest path first
if len(a.Path) != len(b.Path) {
return len(a.Path) > len(b.Path)
} else if format == "js" {
for _, cookie := range effectiveCookies {
// max-age (seconds): 100 years. While Chrome will cap it to max 400 days
cookieStr += `document.cookie='` + cookie.Name + "=" + cookie.Value +
"; path=" + cookie.Path + `; max-age=3153600000` + `';`
}
return false
})
effectiveCookies = util.UniqueSliceFn(effectiveCookies, func(cookie *Cookie) string {
return cookie.Name
})
cookieStr := ""
sep := ""
for _, cookie := range effectiveCookies {
cookieStr += sep + cookie.Name + "=" + cookie.Value
sep = "; "
} else {
return "", fmt.Errorf("invalid format %s", format)
}
return cookieStr, nil
}
136 changes: 136 additions & 0 deletions cmd/cookiecloud/get/get.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
package get

import (
"fmt"
"net/url"

log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"

"github.com/sagan/ptool/cmd"
"github.com/sagan/ptool/cmd/cookiecloud"
"github.com/sagan/ptool/config"
"github.com/sagan/ptool/site"
"github.com/sagan/ptool/util"
)

var (
showAll = false
format = ""
profile = ""
)

var command = &cobra.Command{
Use: "get {site | group | domain | url}...",
Annotations: map[string]string{"cobra-prompt-dynamic-suggestions": "cookiecloud.get"},
Short: "Get cookie for sites or domains from data of cookiecloud servers.",
Long: `Get cookie for sites or domains from data of cookiecloud servers.
Each arg can be a site or group name, a domain (or IP), or a full url. It will query cookiecloud servers
and display all found cookies of the corresponding arg in list.
If --all flag is NOT set (the default case), only cookies which path attribute constitutes the prefix of
the url's path will be included (in the case of arg being a domain or IP, it's path is assumed to be "/").
If --all flag is set, all cookies associated with the domain of arg will be included, the result is only
useful / meanful in the case of "js" format.
By default it will show site cookies in http request "Cookie" header format.
If --format flag is set to "js", it will show cookies in a JavaScript "document.cookie=''" code snippet format,
which is suitable for pasting to the browser developer console to set the cookies of corresponding site.`,
Args: cobra.MatchAll(cobra.MinimumNArgs(1), cobra.OnlyValidArgs),
RunE: get,
}

func init() {
cmd.AddEnumFlagP(command, &format, "format", "", &cmd.EnumFlag{
Description: "Cookies output format",
Options: [][2]string{
{"http", `http request "Cookie" header`},
{"js", `JavaScript 'document.cookie=""' code snippet`},
},
})
command.Flags().BoolVarP(&showAll, "all", "a", false, "Show all cookies associated with the domain (no path checking)")
command.Flags().StringVarP(&profile, "profile", "", "", "Comma-separated string, Set the used cookiecloud profile name(s). If not set, All cookiecloud profiles in config will be used")
cookiecloud.Command.AddCommand(command)
}

func get(cmd *cobra.Command, args []string) error {
cntError := int64(0)
cookiecloudProfiles := cookiecloud.ParseProfile(profile)
if len(cookiecloudProfiles) == 0 {
return fmt.Errorf("no cookiecloud profile specified or found")
}
cookiecloudDatas := []cookiecloud.Ccdata_struct{}
for _, profile := range cookiecloudProfiles {
data, err := cookiecloud.GetCookiecloudData(profile.Server, profile.Uuid, profile.Password, profile.Proxy)
if err != nil {
log.Errorf("Cookiecloud server %s (uuid %s) connection failed: %v\n", profile.Server, profile.Uuid, err)
cntError++
} else {
log.Infof("Cookiecloud server %s (uuid %s) connection ok: cookies of %d domains found\n",
profile.Server, profile.Uuid, len(data.Cookie_data))
cookiecloudDatas = append(cookiecloudDatas, cookiecloud.Ccdata_struct{
Label: fmt.Sprintf("%s-%s", util.GetUrlDomain(profile.Server), profile.Uuid),
Sites: profile.Sites,
Data: data,
})
}
}
if len(cookiecloudDatas) == 0 {
return fmt.Errorf("no cookiecloud server can be connected")
}
siteOrDomainOrUrls := config.ParseGroupAndOtherNames(args...)

fmt.Printf("%-20s %-20s %s\n", "Site/Url/Hostname", "CookieCloud", "Cookie")
for _, siteOrDomainOrUrl := range siteOrDomainOrUrls {
domainOrUrl := ""
if siteConfig := config.GetSiteConfig(siteOrDomainOrUrl); siteConfig != nil {
siteInstance, err := site.CreateSiteInternal(siteOrDomainOrUrl, siteConfig, config.Get())
if err != nil {
log.Debugf("Failed to create site %s", siteOrDomainOrUrl)
} else {
domainOrUrl = util.ParseUrlHostname(siteInstance.GetSiteConfig().Url)
}
} else {
domainOrUrl = siteOrDomainOrUrl
}
var cookieScope = domainOrUrl // the valid scope of cookies
if domainOrUrl == "" {
fmt.Printf("%-20s %-20s %s\n",
util.First(util.StringPrefixInWidth(siteOrDomainOrUrl, 20)), "", "// Error: empty hostname")
cntError++
continue
} else if !util.IsUrl(domainOrUrl) && !util.IsHostname(domainOrUrl) {
fmt.Printf("%-20s %-20s %s\n",
util.First(util.StringPrefixInWidth(siteOrDomainOrUrl, 20)),
"", "// Error: invalid site, url or hostname")
cntError++
continue
} else if util.IsUrl(domainOrUrl) {
urlObj, err := url.Parse(domainOrUrl)
if err == nil {
cookieScope = urlObj.Hostname()
if !showAll && urlObj.Path != "/" {
cookieScope += urlObj.Path
}
}
}
if showAll {
cookieScope += " <all>"
}
for _, cookiecloudData := range cookiecloudDatas {
cookie, _ := cookiecloudData.Data.GetEffectiveCookie(domainOrUrl, showAll, format)
if cookie == "" {
log.Debugf("No cookie found for %s in cookiecloud %s", siteOrDomainOrUrl, cookiecloudData.Label)
continue
}
cookieStr := cookie + " // " + cookieScope
fmt.Printf("%-20s %-20s %s\n", util.First(util.StringPrefixInWidth(siteOrDomainOrUrl, 20)),
util.First(util.StringPrefixInWidth(cookiecloudData.Label, 20)), cookieStr)
}
}

if cntError > 0 {
return fmt.Errorf("%d errors", cntError)
}
return nil
}

0 comments on commit f2cd50a

Please sign in to comment.