[download fail]: bilibili download failed: panic: runtime error: index out of range [1] with length 0 #1213

scegg · 2023-03-11T03:14:51Z

Website name: Bilibili

OS:: Windows

Video URL:: https://www.bilibili.com/bangumi/play/ep704485

Stack overflow

panic: runtime error: index out of range [1] with length 0

Screenshots

none

Additional context

lux -c bl.txt -debug https://www.bilibili.com/bangumi/play/ep704485

lux: version v0.16.0, A fast and simple video downloader.


URL:         https://www.bilibili.com/bangumi/play/ep704485
Method:      GET
Headers:     http.Header{
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "Cookie":          {"_uuid=D9B1AADB-DDF4-10BBA-CF58-676253CDDE9A78747infoc; b_lsid=108EAD7AA_186CEA2D804; b_nut=1663404979; bili_jct=303538e45b28a6bbed35cfb62adb1133; bp_video_offset_26749913=766518492787638345; buvid_fp=950e665fa77caae27ca72c832f0cfeec; buvid_fp_plain=undefined; buvid3=1024B78F-A3CB-BFEF-9B57-E56D1B11145779434infoc; buvid4=3EA21217-05C0-B28E-33A3-2402ACF7E85F88375-022020721-Qd0NZsQY6NFvAP9XHGIwcQ%3D%3D; CURRENT_FNVAL=4048; DedeUserID=26749913; DedeUserID__ckMd5=b94757e248393f8e; fingerprint=950e665fa77caae27ca72c832f0cfeec; LIVE_BUVID=AUTO8416697171148738; nostalgia_conf=-1; PVID=1; rpdid=|(k||RRkYmmJ0J'uY~||llukl; SESSDATA=dc233144%2C1694054793%2C88608%2A31; sid=e3u29g0y"},
    "Referer":         {"https://www.bilibili.com"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 Safari/537.36"},
}
Status Code: 200
panic: runtime error: index out of range [1] with length 0

goroutine 1 [running]:
github.com/iawia002/lux/extractors/bilibili.extractBangumi({0xc0000d4030, 0x2e}, {0xc000a4e000, 0x1903cd}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/bilibili/bilibili.go:95 +0x825
github.com/iawia002/lux/extractors/bilibili.(*extractor).Extract(0x0?, {0xc0000d4030, 0x2e}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, {0xc0000b2078, ...}, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/bilibili/bilibili.go:298 +0x19d
github.com/iawia002/lux/extractors.Extract({0xc0000d4030?, 0x7ff8965304f0?}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0xa, {0xc0000b2078, 0x6}, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/extractors.go:52 +0x391
github.com/iawia002/lux/app.download(0xc0002e6000?, {0xc0000d4030, 0x2e})
        /Users/iawia002/go/src/github.com/iawia002/lux/app/app.go:270 +0x2f2
github.com/iawia002/lux/app.New.func1(0xc000265e40)
        /Users/iawia002/go/src/github.com/iawia002/lux/app/app.go:247 +0x69d
github.com/urfave/cli/v2.(*App).RunContext(0xc0002744e0, {0xaffc48?, 0xc0000b2140}, {0xc0000de000, 0x5, 0x8})
        /Users/iawia002/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:322 +0x953
github.com/urfave/cli/v2.(*App).Run(...)
        /Users/iawia002/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:224
main.main()
        /Users/iawia002/go/src/github.com/iawia002/lux/main.go:13 +0x53

The text was updated successfully, but these errors were encountered:

scegg · 2023-03-11T03:15:38Z

in version 17:

lux -c bl.txt -debug https://www.bilibili.com/bangumi/play/ep704485

lux: version v0.17.0, A fast and simple video downloader.


URL:         https://www.bilibili.com/bangumi/play/ep704485
Method:      GET
Headers:     http.Header{
    "Accept":          {"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
    "Accept-Charset":  {"UTF-8,*;q=0.5"},
    "Accept-Encoding": {"gzip,deflate,sdch"},
    "Accept-Language": {"en-US,en;q=0.8"},
    "Cookie":          {"_uuid=D9B1AADB-DDF4-10BBA-CF58-676253CDDE9A78747infoc; b_lsid=108EAD7AA_186CEA2D804; b_nut=1663404979; bili_jct=303538e45b28a6bbed35cfb62adb1133; bp_video_offset_26749913=766518492787638345; buvid_fp=950e665fa77caae27ca72c832f0cfeec; buvid_fp_plain=undefined; buvid3=1024B78F-A3CB-BFEF-9B57-E56D1B11145779434infoc; buvid4=3EA21217-05C0-B28E-33A3-2402ACF7E85F88375-022020721-Qd0NZsQY6NFvAP9XHGIwcQ%3D%3D; CURRENT_FNVAL=4048; DedeUserID=26749913; DedeUserID__ckMd5=b94757e248393f8e; fingerprint=950e665fa77caae27ca72c832f0cfeec; LIVE_BUVID=AUTO8416697171148738; nostalgia_conf=-1; PVID=1; rpdid=|(k||RRkYmmJ0J'uY~||llukl; SESSDATA=dc233144%2C1694054793%2C88608%2A31; sid=e3u29g0y"},
    "Referer":         {"https://www.bilibili.com"},
    "User-Agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.81 Safari/537.36"},
}
Status Code: 200
panic: runtime error: index out of range [1] with length 0

goroutine 1 [running]:
github.com/iawia002/lux/extractors/bilibili.extractBangumi({0xc00013e060, 0x2e}, {0xc00072c000, 0x1903d7}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/bilibili/bilibili.go:95 +0x806
github.com/iawia002/lux/extractors/bilibili.(*extractor).Extract(0x208002e?, {0xc00013e060, 0x2e}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, {0xc000126078, ...}, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/bilibili/bilibili.go:298 +0x19d
github.com/iawia002/lux/extractors.Extract({0xc00013e060?, 0xb9361b?}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0xa, {0xc000126078, 0x6}, ...})
        /Users/iawia002/go/src/github.com/iawia002/lux/extractors/extractors.go:52 +0x391
github.com/iawia002/lux/app.download(0xc0002d8000?, {0xc00013e060, 0x2e})
        /Users/iawia002/go/src/github.com/iawia002/lux/app/app.go:270 +0x2f2
github.com/iawia002/lux/app.New.func1(0xc000255ac0)
        /Users/iawia002/go/src/github.com/iawia002/lux/app/app.go:247 +0x689
github.com/urfave/cli/v2.(*App).RunContext(0xc000105380, {0x15673e8?, 0xc000126130}, {0xc000154000, 0x5, 0x8})
        /Users/iawia002/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:322 +0x97c
github.com/urfave/cli/v2.(*App).Run(0xc00003c0c8?, {0xc000154000?, 0x101c000070088?, 0x1f142ba1cc8?})
        /Users/iawia002/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:224 +0x38
main.main()
        /Users/iawia002/go/src/github.com/iawia002/lux/main.go:13 +0x45

Hypnodance · 2023-03-12T04:39:13Z

Same problem here, it may be possible to download after a few more attempts. Generally, the download can be successful within 10 attempts, but it is really troublesome to encounter this problem repeatedly.

scegg · 2023-03-12T13:14:01Z

Same problem here, it may be possible to download after a few more attempts. Generally, the download can be successful within 10 attempts, but it is really troublesome to encounter this problem repeatedly.

Tried 10 hrs with no luck at all.

cj3617 · 2023-03-14T03:48:39Z

请更改 bilibi页面获取aid方式 python 实例
initial_state_text = match1(html_content,
r'<script id="__NEXT_DATA__" type="application/json">(.*?)</script>') # FIXME
initial_state_data = json.loads(initial_state_text)
initial_state = initial_state_data['props']['pageProps']['dehydratedState']['queries'][0]['state']['data']
initial_map = initial_state['epMap'][ep_id]
aid = initial_map['aid']
cid = initial_map['cid']

Nathaniel-Wu · 2023-03-15T20:29:40Z

I'm not familiar with Golang and surely there are more elegant and complete solutions to the issue, but here's a rudimentary implementation (in diff format) for the discovery posted by @cj3617 above. This fix does not purge all the problems so I'm not opening a pull request. It enables lux to download bangumi video files just fine, but the episode names are not retrieved correctly, and files are named by bangumi names only.
If you need to download things right now like me, edit extractors/bilibili/bilibili.go accordingly and compile main.go yourself.

-       dataString := utils.MatchOneOf(html, `window.__INITIAL_STATE__=(.+?);\(function`)[1]
-       var data bangumiData
-       err := json.Unmarshal([]byte(dataString), &data)
+       dataString := utils.MatchOneOf(html, `<script\s+id="__NEXT_DATA__"\s+type="application/json"\s*>(.*?)</script\s*>`)[1]
+       epMapString := utils.MatchOneOf(dataString, `"epMap"\s*:\s*(.+?)\s*,\s*"initEpList"`)[1]
+       videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ep(\d+)"`)[1]
+       var epMap map[string]json.RawMessage
+       err := json.Unmarshal([]byte(epMapString), &epMap)
        if err != nil {
                return nil, errors.WithStack(err)
        }
+       var data bangumiData
+       for idString, jsonByte := range epMap {
+               var EpInfo bangumiEpData
+               err := json.Unmarshal(jsonByte, &EpInfo)
+               if err != nil {
+                       return nil, errors.WithStack(err)
+               }
+               EpID, err := strconv.ParseInt(idString, 10, 0);
+               if err != nil {
+                       return nil, errors.WithStack(err)
+               }
+               EpInfo.EpID = int(EpID)
+               if idString == videoIdString {
+                       data.EpInfo = EpInfo
+               }
+               data.EpList = append(data.EpList, EpInfo)
+       }

scegg · 2023-03-18T05:54:32Z

After changing source code, the error is still with new edited lines:

panic: runtime error: index out of range [1] with length 0

goroutine 1 [running]:
github.com/iawia002/lux/extractors/bilibili.extractBangumi({0xc000038190, 0x45}, {0xc000580000, 0x1c2590}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, ...})
        d:/Downloaded/DownTool/lux/extractors/bilibili/bilibili.go:97 +0xc86
github.com/iawia002/lux/extractors/bilibili.(*extractor).Extract(0x208002e?, {0xc000038190, 0x45}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, {0xc00002e0d8, ...}, ...})
        d:/Downloaded/DownTool/lux/extractors/bilibili/bilibili.go:317 +0x19d
github.com/iawia002/lux/extractors.Extract({0xc000038190?, 0x14361b?}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0xa, {0xc00002e0d8, 0x6}, ...})
        d:/Downloaded/DownTool/lux/extractors/extractors.go:52 +0x391
github.com/iawia002/lux/app.download(0xc0002ba500?, {0xc000038190, 0x45})
        d:/Downloaded/DownTool/lux/app/app.go:270 +0x2f2
github.com/iawia002/lux/app.New.func1(0xc000239b00)
        d:/Downloaded/DownTool/lux/app/app.go:247 +0x689
github.com/urfave/cli/v2.(*App).RunContext(0xc0002621a0, {0xb175c8?, 0xc00002e180}, {0xc00008a040, 0x4, 0x4})
        C:/Users/Allen/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:322 +0x97c
github.com/urfave/cli/v2.(*App).Run(0xc00003e0c8?, {0xc00008a040?, 0x100c000072088?, 0x1c0c8759a38?})
        C:/Users/Allen/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:224 +0x38
main.main()
        d:/Downloaded/DownTool/lux/main.go:13 +0x45

jjit · 2023-03-18T08:58:36Z

I can download it based on Nathaniel-Wu's change
just download the Source code(zip) from here
https://github.com/iawia002/lux/releases/tag/v0.17.0
unzip it and edit extractors/bilibili/bilibili.go
you need to delete the first 3 lines code
then use golang docker image to compile
docker run --rm -v /path/to/lux:/usr/src/myapp -w /usr/src/myapp -e GOOS=windows -e GOARCH=amd64 golang go build -v

scegg · 2023-03-18T09:04:44Z

It failed on my Windows. I dont have docker installed on my Windows.

the function edited is:

func extractBangumi(url, html string, extractOption extractors.Options) ([]*extractors.Data, error) {
      dataString := utils.MatchOneOf(html, `<script\s+id="__NEXT_DATA__"\s+type="application/json"\s*>(.*?)</script\s*>`)[1]
      epMapString := utils.MatchOneOf(dataString, `"epMap"\s*:\s*(.+?)\s*,\s*"initEpList"`)[1]
      videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ep(\d+)"`)[1]
      var epMap map[string]json.RawMessage
      err := json.Unmarshal([]byte(epMapString), &epMap)
	if err != nil {
		return nil, errors.WithStack(err)
	}
      var data bangumiData
      for idString, jsonByte := range epMap {
              var EpInfo bangumiEpData
              err := json.Unmarshal(jsonByte, &EpInfo)
              if err != nil {
                      return nil, errors.WithStack(err)
              }
              EpID, err := strconv.ParseInt(idString, 10, 0);
              if err != nil {
                      return nil, errors.WithStack(err)
              }
              EpInfo.EpID = int(EpID)
              if idString == videoIdString {
                      data.EpInfo = EpInfo
              }
              data.EpList = append(data.EpList, EpInfo)
      }
	if !extractOption.Playlist {
		aid := data.EpInfo.Aid
		cid := data.EpInfo.Cid
		bvid := data.EpInfo.BVid
...

Error:

panic: runtime error: index out of range [1] with length 0

goroutine 1 [running]:
github.com/iawia002/lux/extractors/bilibili.extractBangumi({0xc000038190, 0x45}, {0xc000680000, 0x1c25a0}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, ...})
        d:/Downloaded/DownTool/lux/extractors/bilibili/bilibili.go:97 +0xc86
github.com/iawia002/lux/extractors/bilibili.(*extractor).Extract(0x208002e?, {0xc000038190, 0x45}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0x1, {0xc00002e0d8, ...}, ...})
        d:/Downloaded/DownTool/lux/extractors/bilibili/bilibili.go:317 +0x19d
github.com/iawia002/lux/extractors.Extract({0xc000038190?, 0xa3361b?}, {0x0, {0x0, 0x0}, 0x1, 0x0, 0xa, {0xc00002e0d8, 0x6}, ...})
        d:/Downloaded/DownTool/lux/extractors/extractors.go:52 +0x391
github.com/iawia002/lux/app.download(0xc0002ba500?, {0xc000038190, 0x45})
        d:/Downloaded/DownTool/lux/app/app.go:270 +0x2f2
github.com/iawia002/lux/app.New.func1(0xc00023bb00)
        d:/Downloaded/DownTool/lux/app/app.go:247 +0x689
github.com/urfave/cli/v2.(*App).RunContext(0xc0002641a0, {0x14075c8?, 0xc00002e180}, {0xc00008a040, 0x4, 0x4})
        C:/Users/Allen/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:322 +0x97c
github.com/urfave/cli/v2.(*App).Run(0xc00003e0c8?, {0xc00008a040?, 0x100c000072088?, 0x1dfa8d49a38?})
        C:/Users/Allen/go/pkg/mod/github.com/urfave/cli/v2@v2.6.0/app.go:224 +0x38
main.main()
        d:/Downloaded/DownTool/lux/main.go:13 +0x45

the bilibili.go:97 is

videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ep(\d+)"`)[1]

Is there anyone who want to fork this project for a quick fix?

scegg · 2023-03-18T09:25:12Z

After I dig the text dataString, I found that there is no such a text can match the condition of the line 97.

The only one "videoId" (with quotes) is near the end of dataString. The context is:

...,"page":"/bangumi/play/[videoId]","query":{"spm_id_from":"333.337.0.0","videoId":"ss26257"},"buildId":"UjsRaNzhtNURzCjvPY8PC","assetPrefix":...

No such an ep at all.

jjit · 2023-03-18T10:18:10Z

compiled exe based on Nathaniel-Wu's change
win x64: lux.zip

scegg · 2023-03-18T10:20:50Z

compiled exe based on Nathaniel-Wu's change win x64: lux.zip

Thanks a lot.

By providing the url including ep other than ss, the exe you compiled as well as mine works.

fgr178707 · 2023-03-30T07:05:22Z

根据Nathaniel-Wu的更改Win x64：lux编译的exe.zip

你这个 Title: 只会显示这个番的名字这一集叫啥啥也没有
正常的话应该叫 Title: 幸运星:第1话狂奔的女人


英文：
Your Title: Only the name of this episode will be displayed
Normally it should be called Title: Lucky Star: Chapter 1 The Running Woman
@jjit

jjit · 2023-03-30T21:10:17Z

@fgr178707 是基于这个帖子里 Nathaniel-Wu 发的修改编译的
他在原帖里有说，可以往上面翻看他发的帖

but the episode names are not retrieved correctly
and files are named by bangumi names only

番集名字没有正确提取，文件只能以番的名字命名

ChrisVicky · 2023-04-08T11:51:49Z

compiled exe based on Nathaniel-Wu's change win x64: lux.zip

Thanks a lot.

By providing the url including ep other than ss, the exe you compiled as well as mine works.

By Replacing

videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ep(\d+)"`)[1]

with

videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ss(\d+)"`)[1]

you'll be able to download videos with ss in the url.

And you could modify the Regular Expression to satisfy both ss and ep conditions.

baiyufei · 2023-04-19T14:53:48Z

I'm not familiar with Golang and surely there are more elegant and complete solutions to the issue, but here's a rudimentary implementation (in diff format) for the discovery posted by @cj3617 above. This fix does not purge all the problems so I'm not opening a pull request. It enables lux to download bangumi video files just fine, but the episode names are not retrieved correctly, and files are named by bangumi names only. If you need to download things right now like me, edit extractors/bilibili/bilibili.go accordingly and compile main.go yourself.

-       dataString := utils.MatchOneOf(html, `window.__INITIAL_STATE__=(.+?);\(function`)[1]
-       var data bangumiData
-       err := json.Unmarshal([]byte(dataString), &data)
+       dataString := utils.MatchOneOf(html, `<script\s+id="__NEXT_DATA__"\s+type="application/json"\s*>(.*?)</script\s*>`)[1]
+       epMapString := utils.MatchOneOf(dataString, `"epMap"\s*:\s*(.+?)\s*,\s*"initEpList"`)[1]
+       videoIdString := utils.MatchOneOf(dataString, `"videoId"\s*:\s*"ep(\d+)"`)[1]
+       var epMap map[string]json.RawMessage
+       err := json.Unmarshal([]byte(epMapString), &epMap)
        if err != nil {
                return nil, errors.WithStack(err)
        }
+       var data bangumiData
+       for idString, jsonByte := range epMap {
+               var EpInfo bangumiEpData
+               err := json.Unmarshal(jsonByte, &EpInfo)
+               if err != nil {
+                       return nil, errors.WithStack(err)
+               }
+               EpID, err := strconv.ParseInt(idString, 10, 0);
+               if err != nil {
+                       return nil, errors.WithStack(err)
+               }
+               EpInfo.EpID = int(EpID)
+               if idString == videoIdString {
+                       data.EpInfo = EpInfo
+               }
+               data.EpList = append(data.EpList, EpInfo)
+       }

根据这个回答做了些修改，可以正确处理 ep 或者 ss 开头的id，可以识别分段标题，PR在这里 #1227

这是打包好的文件 lux.zip

scegg added the bug Something isn't working label Mar 11, 2023

Nathaniel-Wu mentioned this issue Mar 15, 2023

无法下载哔哩哔哩视频 #1214

Closed

jjit mentioned this issue Mar 19, 2023

[download fail]: bilibili can't dl ep311070 #1216

Closed

baiyufei added a commit to baiyufei/lux that referenced this issue Apr 19, 2023

fix: bilibili extractBangumi(iawia002#1213)

901950b

iawia002 added the bilibili label Apr 20, 2023

iawia002 mentioned this issue Apr 20, 2023

fix: bilibili extractBangumi #1227

Merged

iawia002 closed this as completed in #1227 Apr 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[download fail]: bilibili download failed: panic: runtime error: index out of range [1] with length 0 #1213

[download fail]: bilibili download failed: panic: runtime error: index out of range [1] with length 0 #1213

scegg commented Mar 11, 2023

scegg commented Mar 11, 2023

Hypnodance commented Mar 12, 2023

scegg commented Mar 12, 2023

cj3617 commented Mar 14, 2023

Nathaniel-Wu commented Mar 15, 2023

scegg commented Mar 18, 2023

jjit commented Mar 18, 2023

scegg commented Mar 18, 2023 •

edited

scegg commented Mar 18, 2023 •

edited

jjit commented Mar 18, 2023 •

edited

scegg commented Mar 18, 2023

fgr178707 commented Mar 30, 2023 •

edited

jjit commented Mar 30, 2023

ChrisVicky commented Apr 8, 2023

baiyufei commented Apr 19, 2023

[download fail]: bilibili download failed: panic: runtime error: index out of range [1] with length 0 #1213

[download fail]: bilibili download failed: panic: runtime error: index out of range [1] with length 0 #1213

Comments

scegg commented Mar 11, 2023

scegg commented Mar 11, 2023

Hypnodance commented Mar 12, 2023

scegg commented Mar 12, 2023

cj3617 commented Mar 14, 2023

Nathaniel-Wu commented Mar 15, 2023

scegg commented Mar 18, 2023

jjit commented Mar 18, 2023

scegg commented Mar 18, 2023 • edited

scegg commented Mar 18, 2023 • edited

jjit commented Mar 18, 2023 • edited

scegg commented Mar 18, 2023

fgr178707 commented Mar 30, 2023 • edited

jjit commented Mar 30, 2023

ChrisVicky commented Apr 8, 2023

baiyufei commented Apr 19, 2023

scegg commented Mar 18, 2023 •

edited

scegg commented Mar 18, 2023 •

edited

jjit commented Mar 18, 2023 •

edited

fgr178707 commented Mar 30, 2023 •

edited