[true java call completed] https://github.com/siddht1/ydl_java help needeed , porting youtube-dl to java #10975

siddht4 · 2016-10-20T04:55:50Z

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.10.19. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

I've verified and I assure that I'm running youtube-dl 2016.10.19

Before submitting an issue make sure you have:

At least skimmed through README and most notably FAQ and BUGS sections
Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

Bug report (encountered problems with youtube-dl)
Site support request (request for adding support for a new site)
Feature request (request for a new functionality)
Question
Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2016.10.19
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Description of your issue, suggested solution and other information

Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.

i m currently developing a fix to run youtube-dl under java

method 1 :

java bytecode ----> jvm ----> exec python ----> python -----> youtube-dl

method 2 :
java bytecode ----> jvm ----> exec jython ----> jython -----> youtube-dl

these are the output if youtube-dl is run through jython directly via terminal

code : java -jar jython.jar main.py -v

output:
Traceback (most recent call last):
File "/home/gh/Documents/jython.jar/Lib/runpy.py", line 161, in _run_module_as_main
File "/home/gh/Documents/jython.jar/Lib/runpy.py", line 72, in _run_code
File "youtube-dl.py/main.py", line 16, in
File "youtube-dl.py/youtube_dl/init.py", line 15, in
File "youtube-dl.py/youtube_dl/options.py", line 8, in
File "youtube-dl.py/youtube_dl/downloader/init.py", line 3, in
File "youtube-dl.py/youtube_dl/downloader/common.py", line 9, in
File "youtube-dl.py/youtube_dl/compat.py", line 2599, in
File "youtube-dl.py/youtube_dl/compat.py", line 2599, in
File "/home/gh/Documents/jython.jar/Lib/shlex.py", line 279, in split
File "/home/gh/Documents/jython.jar/Lib/shlex.py", line 269, in next
File "/home/gh/Documents/jython.jar/Lib/shlex.py", line 96, in get_token
File "/home/gh/Documents/jython.jar/Lib/shlex.py", line 124, in read_token
at org.python.core.PyString.(PyString.java:64)
at org.python.core.PyString.(PyString.java:70)
at org.python.modules.cStringIO$StringIO.read(cStringIO.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)

java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Cannot create PyString with non-byte value

siddht4 · 2016-10-20T05:02:14Z

well these is the current code i have made to run it through exec command in java

import java.io.*;
class test1
{
public static void main(String a[])
{
try{

Process p2 = Runtime.getRuntime().exec("python main.py -v");
BufferedReader in = new BufferedReader(new InputStreamReader(p2.getInputStream()));
String ret= new String(in.readLine());
System.out.println("returned "+ret);
}
catch(Exception e){}
}
}

yan12125 · 2016-10-20T07:21:46Z

java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Cannot create PyString with non-byte value

It's a Jython bug, which can be work-arounded by modifying line 2604 of youtube_dl/compat.py:

$ git diff youtube_dl/compat.py 
diff --git a/youtube_dl/compat.py b/youtube_dl/compat.py
index b8aaf5a..b7a57d6 100644
--- a/youtube_dl/compat.py
+++ b/youtube_dl/compat.py
@@ -2601,7 +2601,7 @@ try:
             isinstance(args[0], compat_str) and
             args[0] == '中文')
     compat_shlex_split = shlex.split
-except (AssertionError, UnicodeEncodeError):
+except:
     # Working around shlex issue with unicode strings on some python 2
     # versions (see http://bugs.python.org/issue1548891)
     def compat_shlex_split(s, comments=False, posix=True):

I suggest not to use Jython as there are too many bugs in it.

Closing as it's not a youtube-dl bug.

siddht4 · 2016-10-20T08:08:50Z

hav3 you a worki g code , if yes share pls as this project has now become a headache for me

yan12125 · 2016-10-20T11:14:18Z

After changing line 2604, jython works for me:

$ youtube-dl jython -v https://vimeo.com/187161170
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://vimeo.com/187161170']
[debug] Encodings: locale UTF-8, fs None, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.10.19
[debug] Git HEAD: 8e2915d
[debug] Python version 2.7.1 - Java-1.7.0_111-OpenJDK_64-Bit_Server_VM,_24.111-b01,_Oracle_Corporation-on-Linux-4.8.2-1-ARCH-amd64
[debug] exe versions: ffmpeg 3.1.4, ffprobe 3.1.4, rtmpdump 2.4
[debug] Proxy map: {}
[vimeo] 187161170: Downloading webpage
[vimeo] 187161170: Extracting information
[vimeo] 187161170: Downloading webpage
[vimeo] 187161170: Downloading JSON metadata
[vimeo] 187161170: Checking Original video URL
[vimeo] 187161170: Downloading m3u8 information
[debug] Invoking downloader on u'https://vimeo.com/187161170/download?t=1476961831&v=618009219&s=0ad552f70c5eb80a478b0decf37d7049151ef74705ba2d1c152af2a4657a1abd'
[download] Destination: We Buy Houses Milwaukee-187161170.asx
[download] 100% of 7.08MiB in 00:00

Jython version:

$ jython -c 'import sys; print(sys._mercurial)'
('Jython', 'default', 'a07c595b410f')

I compile Jython from sources. IIRC the latest release (2.7.0) has quite a few bugs.

yan12125 · 2016-10-20T13:14:14Z

The Jython bug is reported to http://bugs.jython.org/issue2527

siddht4 · 2016-10-20T15:00:55Z

thank you , the makeover took time to register in my mind .

siddht4 · 2016-10-20T15:17:07Z

can you actually verify this indeed is the code , sorry to take your time

compat.py.txt

rename compat.py.txt to compat.py as github doesnt allow to upload python source files yet.

siddht4 · 2016-10-20T15:19:26Z

my jython

java -jar jython.jar

Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0_91

yan12125 · 2016-10-20T15:22:22Z

Seems you've deleted some innocent lines. This is my version: compat.py.txt

siddht4 · 2016-10-21T05:25:55Z

added a working makeover

https://github.com/siddht1/youtube_dl_java

siddht4 · 2016-10-21T05:45:50Z

@yan12125 does youtube-dl externally or internally calls ffmpeg

check my code here

https://github.com/siddht1/youtube_dl_java

feedback from you is appreciated

siddht4 · 2016-10-21T06:38:19Z

https://github.com/siddht1/youtube_dl_java

usage 👎
java -jar youtube-dl

then provide your url or command

siddht4 · 2016-10-21T06:45:36Z

@yan12125

how -v is executed in youtube-dl

my java caller doesnt seem to output it

yan12125 · 2016-10-21T10:05:52Z

Good job!

does youtube-dl externally or internally calls ffmpeg

It indeed calls ffmpeg. I'm not sure what's the difference between "externally" and "internally".

how -v is executed in youtube-dl

Logs are printed to STDERR

siddht4 · 2016-10-21T10:33:03Z

well internally means does youtube-dl handles it i.e there is code to run ffmpeg directly inside youtube-dl

eg
{ youtube-dl ( youtube-dl) <---------> ( ffmpeg) }
{} inside a application

and externally means it has to call ffmpeg , then youtube-dl proceeds

eg 👍
{ youtube-dl ( youtube-dl)} <--------->{ ffmpeg ( ffmpeg) }}

yan12125 · 2016-10-21T10:36:43Z

I guess what youtube-dl does is more likely "externally". youtube-dl and ffmpeg run in different processes.

siddht4 · 2016-10-21T10:38:47Z

@yan12125 can you mail me your mail id , my email id : dhar.75.ap@gmail.com

yan12125 · 2016-10-21T10:41:16Z

You can find my mail on Github profile page, but is there anything you don't want to discuss publicly?

siddht4 · 2016-10-21T10:49:28Z

@yan12125

well you are youtube-dl maintainer , so I thought there are some youtube-dl parts(codes) completely out of my mind

i don`t want to put issues for them , like this is an external bug itself

yan12125 · 2016-10-21T10:53:23Z

OK feel free to drop me mails. I'll try my best to answer you.

siddht4 · 2016-10-21T10:57:35Z

thank you

siddht4 · 2016-10-21T11:01:08Z

well
https://github.com/siddht1/youtube_dl_java

is where i would be working

@yan12125
how to print verbose in java

yan12125 · 2016-10-21T11:06:37Z

Anything printed after adding -v after /usr/local/bin/youtube-dl?

siddht4 · 2016-10-21T11:10:48Z

output , you can try yourself
java -jar youtube-dl.jar
Enter your url or command
-v

yan12125 · 2016-10-21T11:23:15Z

Indeed I got nothing. By the way, Google says getErrorStream is necessary to get things from STDERR.

siddht4 · 2016-10-21T11:32:38Z

yes , fixing it

siddht4 · 2016-10-21T11:48:09Z

yes got what the problem is

siddht4 · 2016-11-15T18:40:27Z

@yan12125 how does youtube_dl suppose to call in mac, in linux its youtube-dl , in windows its youtube-dl.exe , python enviroment python _main.py , i need as i am about to release the next version by detecting os ,test module at https://github.com/siddht1/youtube_dl_java/blob/master/new_comp/OpertingSystemInfo.java

yan12125 · 2016-11-15T18:48:19Z

On Mac is quite similar to on Linux

siddht4 · 2016-11-15T18:50:54Z

@yan12125 what`s it , should i use linux,unix,mac as same

/usr/local/bin/youtube-dl

yan12125 · 2016-11-15T18:56:26Z

You can (nobody can say you should :)

siddht4 · 2016-11-15T18:59:53Z

@yan12125 can you suggest any new feature , not present in original youtube_dl to be added in my project ,

i actually am going to use the sub fork of my projects one this time

yan12125 · 2016-11-15T19:05:47Z

There are quite a few missing features in youtube-dl. What do you want?

siddht4 · 2016-11-16T04:41:24Z

what can you provide , if possible. which version youtube-dl used to support instagram completly

yan12125 · 2016-11-16T05:12:33Z

What do you mean by "completely"? Anyway this may help: https://github.com/rg3/youtube-dl/search?q=instagram&state=open&type=Issues&utf8=%E2%9C%93

siddht4 · 2016-11-16T06:41:05Z

completely meant for both handling image and video. can youtubr-dl download images and other format too. If yes which python script has to be modified

yan12125 · 2016-11-16T06:53:07Z

You need to modify instagram.py. Here's a tip: #9337 (comment)

siddht4 · 2016-11-16T13:46:42Z

@yan12125 thank you for the tip , i m a noob in programming languages so will update it ,if it works . would also add jython comptability this time

siddht4 · 2016-11-20T09:58:46Z

The program is ready and I m now upgrading it. I have also updated the wikI , it lists all p mod of youtube-dl too. I will try to merge it here as its a wikI and will surely help a lot of devs too. I will soon implement code in java too run exactly as youtube-dl. Thats the complete rework/rewrite in java. Once commited you and others can try too.

Stewori · 2017-02-01T18:51:03Z

FWIW the Jython-bug mentioned here (http://bugs.jython.org/issue2527) was just fixed as of jython/frozen-mirror@0b38e39.

siddht4 · 2017-02-12T14:23:20Z

great , the UI should now work with java , python , jython . would update jython and build the old code once more .

siddht4 · 2017-07-09T16:54:56Z

new code should be compatible with c,c++,java,jython and python as it extends https://github.com/siddht1/youtube_dl_embed/blob/master/1.py

Php is also under way

yan12125 closed this Oct 20, 2016

yan12125 added the external-bugs label Oct 20, 2016

siddht4 changed the title ~~help needeed , porting youtube-dl to java~~ [true java call completed] help needeed , porting youtube-dl to java Oct 21, 2016

siddht4 changed the title ~~[true java call completed] help needeed , porting youtube-dl to java~~ [true java call completed] https://github.com/siddht1/ydl_java help needeed , porting youtube-dl to java Oct 21, 2016

siddht4 mentioned this issue Nov 13, 2016

[downloader/hlsnative] Resuming encrypted HLS streams leads to broken files #11015

Open

4 of 8 tasks complete

siddht4 mentioned this issue Nov 18, 2016

add youtube_dl wiki #11234

Open

0 of 5 tasks complete

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

[true java call completed] https://github.com/siddht1/ydl_java help needeed , porting youtube-dl to java #10975

[true java call completed] https://github.com/siddht1/ydl_java help needeed , porting youtube-dl to java #10975

Comments

siddht4 commented Oct 20, 2016

Please follow the guide below

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.10.19. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Description of your issue, suggested solution and other information

siddht4 commented Oct 20, 2016

yan12125 commented Oct 20, 2016

siddht4 commented Oct 20, 2016

yan12125 commented Oct 20, 2016

yan12125 commented Oct 20, 2016

siddht4 commented Oct 20, 2016

siddht4 commented Oct 20, 2016

siddht4 commented Oct 20, 2016

yan12125 commented Oct 20, 2016

siddht4 commented Oct 21, 2016 • edited

siddht4 commented Oct 21, 2016 • edited

siddht4 commented Oct 21, 2016 • edited

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

yan12125 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

siddht4 commented Oct 21, 2016

siddht4 commented Nov 15, 2016

yan12125 commented Nov 15, 2016

siddht4 commented Nov 15, 2016

yan12125 commented Nov 15, 2016

siddht4 commented Nov 15, 2016

yan12125 commented Nov 15, 2016

siddht4 commented Nov 16, 2016

yan12125 commented Nov 16, 2016

siddht4 commented Nov 16, 2016

yan12125 commented Nov 16, 2016

siddht4 commented Nov 16, 2016

siddht4 commented Nov 20, 2016

Stewori commented Feb 1, 2017

siddht4 commented Feb 12, 2017

siddht4 commented Jul 9, 2017 • edited

Essential cookies

Always active

Analytics cookies

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.10.19. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

siddht4 commented Oct 21, 2016 •

edited

siddht4 commented Oct 21, 2016 •

edited

siddht4 commented Oct 21, 2016 •

edited

siddht4 commented Jul 9, 2017 •

edited