A massive difference between network usage and output size

[Williamtung1]

Thanks for your hard work and I find your tool very useful for me. However, I find it interesting that there are a massive difference between network usage and output size of my songs. Also, a song I download with freyr before (probably a few version before) would have 10-20 MB. Now, with the latest version(0.9.0), size of a track would be 500 MB or so. I am just curious why?
For example:

``
Stats
[•] Runtime: [31.6s]
[•] Total queries: [01]
[•] Total tracks: [01]
» Skipped: [00]
✓ Passed: [01]
✕ Failed: [00]
[•] Output directory: [.]
[•] Total Output size: 528.52 MB
[•] Total Network Usage: 10.69 MB
♫ Media: 10.57 MB
➤ Album Art: 118.25 KB
[•] Output bitrate: 320k

``

[miraclx]

Oh wow, that looks like a serious bug. I'll try to reproduce this and investigate why.

Thanks for reporting!

[Williamtung1]

No problem, I will try to reproduce it as well

[Williamtung1]

The problem occurs even I am using 0.8.1

[•] Collation Complete
============ Stats ============
[•] Runtime: [32.2s]
[•] Total queries: [01]
[•] Total tracks: [01]
» Skipped: [00]
✓ Passed: [01]
✕ Failed: [00]
[•] Output directory: [.]
[•] Cover Art: cover.png (640x640)
[•] Total Output size: 510.95 MB
[•] Total Network Usage: 3.01 MB
♫ Media: 3.01 MB
➤ Album Art: 0.00 B
[•] Output bitrate: 320k

[Williamtung1]

By the way, I don't know if it would help, but I can never install freyr directly without an issue:
(logs after I have uninstalled all global npm packages)

william@Williams-Mac-Pro ~ % npm install -g freyr
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARN deprecated superagent@6.1.0: Please upgrade to v7.0.2+ of superagent.  We have fixed numerous issues with streams, form-data, attach(), filesystem errors not bubbling up (ENOENT on attach()), and all tests are now passing.  See the releases tab for more information at <https://github.com/visionmedia/superagent/releases>.
npm ERR! code 127
npm ERR! path /usr/local/lib/node_modules/freyr/node_modules/youtube-dl-exec
npm ERR! command failed
npm ERR! command sh -c npx bin-version-check-cli python3 "> =3.7" || npx bin-version-check-cli python "> =3.7"
npm ERR! sh: bin-version-check: command not found
npm ERR! sh: bin-version-check: command not found

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/william/.npm/_logs/2023-02-21T20_48_19_731Z-debug-0.log

After the installation of bin-version-check-cli , everything seems fine.

william@Williams-Mac-Pro tmp % npm install -g bin-version-check-cli

added 91 packages in 2s

21 packages are looking for funding
  run `npm fund` for details
william@Williams-Mac-Pro tmp % npm install -g freyr
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARN deprecated superagent@6.1.0: Please upgrade to v7.0.2+ of superagent.  We have fixed numerous issues with streams, form-data, attach(), filesystem errors not bubbling up (ENOENT on attach()), and all tests are now passing.  See the releases tab for more information at <https://github.com/visionmedia/superagent/releases>.

added 295 packages in 4s

66 packages are looking for funding
  run `npm fund` for details

But still, the download size is around 500MB

william@Williams-Mac-Pro tmp % freyr spotify:track:3dWjcELpf8YuQzRUl8O2xe
    ____                    
   / __/_______  __  _______
  / /_/ ___/ _ \/ / / / ___/
 / __/ /  /  __/ /_/ / /    
/_/ /_/   \___/\__, /_/     
              /____/ v0.9.0

freyr - (c) Miraculous Owonubi <omiraculous@gmail.com>
------------------------------------------------------
Checking directory permissions...[done] 
[spotify:track:3dWjcELpf8YuQzRUl8O2xe]
 [•] Identifying service...[Spotify]
 [•] Checking authentication...[authenticated]
 Detected [track]
 Obtaining track metadata...[done] 
  ➤ Title: Suite bergamasque, L. 75: III. Clair de lune
  ➤ Album: Claude Debussy
  ➤ Artist: Claude Debussy
  ➤ Year: 2018
  ➤ Playtime: 04:19
 [•] Collating...
 • [04 Suite bergamasque, L. 75: III. Clair de lune]
    | ➤ Collating sources...
    |  ➤ [•] YouTube Music...[success, found 1 source]
    | ➤ Awaiting audiofeeds...[done] 
    | [✓] Got album art 
    | [✓] Got raw track file 
    | [•] Post Processing...
 [•] Download Complete
 [•] Embedding Metadata...
  • [✓] 04 Suite bergamasque, L. 75: III. Clair de lune
[•] Collation Complete
============ Stats ============
 [•] Runtime: [24.2s]
 [•] Total queries: [01]
 [•] Total tracks: [01]
     » Skipped: [00]
     ✓ Passed:  [01]
     ✕ Failed:  [00]
 [•] Output directory: [.]
 [•] Total Output size: 536.85 MB
 [•] Total Network Usage: 4.81 MB
     ♫ Media: 4.71 MB
     ➤ Album Art: 99.10 KB
 [•] Output bitrate: 320k
===============================

[miraclx]

This was caused by buffer overflows in AtomicParsley, which have now been resolved. The docker images have been updated following #476. You can download binaries from https://github.com/miraclx/atomicparsley/releases/tag/20230114.175602.21bde60.

[Williamtung1]

Thanks a lot for your hard work. It works perfectly after using your version of AtomicParsley. Thanks to you again!

…

On Thu, 6 Apr 2023 at 04:34, Miraculous Owonubi ***@***.***> wrote: This was caused by buffer overflows in AtomicParsley, which have now been resolved. The docker images have been updated following #476 <#476>. You can download binaries from https://github.com/miraclx/atomicparsley/releases/tag/20230114.175602.21bde60 . — Reply to this email directly, view it on GitHub <#437 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AW56NBYXSYESRTMCOUSCYADW7Y2UNANCNFSM6AAAAAAVA4MNG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[aarnaegg]

Thanks a lot for your hard work. It works perfectly after using your version of AtomicParsley. Thanks to you again!
…
On Thu, 6 Apr 2023 at 04:34, Miraculous Owonubi @.> wrote: This was caused by buffer overflows in AtomicParsley, which have now been resolved. The docker images have been updated following #476 <#476>. You can download binaries from https://github.com/miraclx/atomicparsley/releases/tag/20230114.175602.21bde60 . — Reply to this email directly, view it on GitHub <#437 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW56NBYXSYESRTMCOUSCYADW7Y2UNANCNFSM6AAAAAAVA4MNG4 . You are receiving this because you authored the thread.Message ID: @.>

After downloading the binary, how do you install it?

[miraclx]

Answered in #662 (comment)