`dom parse` hands for utf-8 streams #21

Closed
gromnitsky opened this Issue Oct 12, 2014 · 9 comments

Comments

Projects
None yet
2 participants
@gromnitsky

Fedora 21 x86_64

$ rpm -q tcl
tcl-8.6.2-1.fc21.x86_64

# wget http://echo.msk.ru/contributors/4769/rss-audio.xml -O shender.xml
...

$ cat tdom-utf8-test.tcl
#!/usr/bin/env tclsh

package require -exact tdom 0.8.3
set fd [open "shender.xml"]

# it just hands
set doc [dom parse -channel $fd]

puts $doc

If I run

$ TCLLIBPATH="/opt/tmp/tdom" ./tdom-utf8-test.tcl

It outputs nothing & CPU goes to the roof, until I hit Ctrl-C.

(/opt/tmp/tdom directory is where tdom was cloned & compiled.)

$ locale
LANG=en_US.UTF-8
LC_CTYPE=uk_UA.UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=uk_UA.UTF-8
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=uk_UA.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=uk_UA.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
@pointsman

This comment has been minimized.

Show comment
Hide comment
@pointsman

pointsman Oct 16, 2014

Member

Just works for me, as expected; can't reproduce. Obviously the content of http://echo.msk.ru/contributors/4769/rss-audio.xml changed, since the point of time you've observed the reported behaviour. I'm afraid, without the original content, that triggers the reported problem I'm not able to do much.

Member

pointsman commented Oct 16, 2014

Just works for me, as expected; can't reproduce. Obviously the content of http://echo.msk.ru/contributors/4769/rss-audio.xml changed, since the point of time you've observed the reported behaviour. I'm afraid, without the original content, that triggers the reported problem I'm not able to do much.

@pointsman

This comment has been minimized.

Show comment
Hide comment
@pointsman

pointsman Oct 16, 2014

Member

Still, even with the original file, your test code puts the doc cmd name and exits, as I would expect. You are still able to reproduce? You wrote, you're on fedora, tcl 8.6.2. With tdom build from trunk? If yes: do you have another box, to try and look if you see the same?

Member

pointsman commented Oct 16, 2014

Still, even with the original file, your test code puts the doc cmd name and exits, as I would expect. You are still able to reproduce? You wrote, you're on fedora, tcl 8.6.2. With tdom build from trunk? If yes: do you have another box, to try and look if you see the same?

@gromnitsky

This comment has been minimized.

Show comment
Hide comment
@gromnitsky

gromnitsky Oct 16, 2014

I've tried on 32bit machine too (w/ tdom from the trunk, of course).

I think it has something to do w/ Tcl 8.5 vs. 8.6, because it indeed does work fine on:

Fedora 20 i686
tcl-8.5.14-1.fc20.i686

but doesn't work in this 2 combinations:

Fedora 20 i686
tcl-8.6.2-1.fc20.i686

Fedora 21 x86_64
tcl-8.6.2-1.fc21.x86_64

I've tried on 32bit machine too (w/ tdom from the trunk, of course).

I think it has something to do w/ Tcl 8.5 vs. 8.6, because it indeed does work fine on:

Fedora 20 i686
tcl-8.5.14-1.fc20.i686

but doesn't work in this 2 combinations:

Fedora 20 i686
tcl-8.6.2-1.fc20.i686

Fedora 21 x86_64
tcl-8.6.2-1.fc21.x86_64

@pointsman

This comment has been minimized.

Show comment
Hide comment
@pointsman

pointsman Oct 23, 2014

Member

I tested on 64-bit SuSE linux, with tcl8.6.2 and tcl8.6.3rc0 and couldn't reproduce the failure; it simply works for me.

Could you please try to compile tcl8.6.2 from core source ball (http://www.tcl.tk/software/tcltk/download.html), to check, if the problem is in any fedora modification or special configure setting?

Member

pointsman commented Oct 23, 2014

I tested on 64-bit SuSE linux, with tcl8.6.2 and tcl8.6.3rc0 and couldn't reproduce the failure; it simply works for me.

Could you please try to compile tcl8.6.2 from core source ball (http://www.tcl.tk/software/tcltk/download.html), to check, if the problem is in any fedora modification or special configure setting?

@gromnitsky

This comment has been minimized.

Show comment
Hide comment
@gromnitsky

gromnitsky Oct 24, 2014

I've installed opensuse 13.1 (in minimal config, w/o x11) & the result is quite bizarre.

Everything works fine but only for tcl <= 8.5.15 & tcl <= 8.6.1. W/ the latest releases 8.5.16 & 8.6.2 it hands like in Fedora.

How I tested it. (I use 8.5.x here just because it builds faster.)

https://dl.dropboxusercontent.com/u/24701177/github/tdom/tdom-test.tar.gz contains:

tdom-test/
|__ ..
|__ test/
|   |__ ..
|   |__ ie.xml
|   |__ shender.xml
|   |__ tdom-utf8-test.tcl
|__ test-run.sh*

If you run

$ ./test-run.sh 8.5.16

it will download tcl-8.5.16, compile it, clone tdom repo, compile it against the tcl, then run tdom-utf8-test.tcl against the tdom.

If everything works file, it prints 'OK', otherwise it hands in the end.

In https://dl.dropboxusercontent.com/u/24701177/github/tdom/tdom-bug-report.tar.gz I've included config.log files from tcl & tdom, where 8.5.15 == ok, 8.5.16 == fail.

I've installed opensuse 13.1 (in minimal config, w/o x11) & the result is quite bizarre.

Everything works fine but only for tcl <= 8.5.15 & tcl <= 8.6.1. W/ the latest releases 8.5.16 & 8.6.2 it hands like in Fedora.

How I tested it. (I use 8.5.x here just because it builds faster.)

https://dl.dropboxusercontent.com/u/24701177/github/tdom/tdom-test.tar.gz contains:

tdom-test/
|__ ..
|__ test/
|   |__ ..
|   |__ ie.xml
|   |__ shender.xml
|   |__ tdom-utf8-test.tcl
|__ test-run.sh*

If you run

$ ./test-run.sh 8.5.16

it will download tcl-8.5.16, compile it, clone tdom repo, compile it against the tcl, then run tdom-utf8-test.tcl against the tdom.

If everything works file, it prints 'OK', otherwise it hands in the end.

In https://dl.dropboxusercontent.com/u/24701177/github/tdom/tdom-bug-report.tar.gz I've included config.log files from tcl & tdom, where 8.5.15 == ok, 8.5.16 == fail.

@pointsman

This comment has been minimized.

Show comment
Hide comment
@pointsman

pointsman Oct 27, 2014

Member

Thanks for providing so much useful additional data.

It's believed, that the core of the problem is in fact a tcl core thing. You probably got bitten by http://core.tcl.tk/tcl/tktview?name=ed29c4da21 and the related http://core.tcl.tk/tcl/tktview?name=5e0a543569 (the second is marked as dub, but names the reason more to the point).

Could you please check and confirm, that (if) the already released 8.5.17 (ftp://ftp.tcl.tk/pub/tcl/tcl8_5/tcl8.5.17-src.tar.gz) in fact fixes the problem, you see? Tcl8.6.3 is in release candidate state and includes the same fix. RCs could be found at ftp://ftp.tcl.tk/pub/tcl/tcl8_6

Member

pointsman commented Oct 27, 2014

Thanks for providing so much useful additional data.

It's believed, that the core of the problem is in fact a tcl core thing. You probably got bitten by http://core.tcl.tk/tcl/tktview?name=ed29c4da21 and the related http://core.tcl.tk/tcl/tktview?name=5e0a543569 (the second is marked as dub, but names the reason more to the point).

Could you please check and confirm, that (if) the already released 8.5.17 (ftp://ftp.tcl.tk/pub/tcl/tcl8_5/tcl8.5.17-src.tar.gz) in fact fixes the problem, you see? Tcl8.6.3 is in release candidate state and includes the same fix. RCs could be found at ftp://ftp.tcl.tk/pub/tcl/tcl8_6

@gromnitsky

This comment has been minimized.

Show comment
Hide comment
@gromnitsky

gromnitsky Oct 28, 2014

Yes, 8.5.17 & 8.6.3rc1 both work w/o issues. Hurray!

Could you add to readme a note that 8.5.16 & 8.6.2 versions are 'defective'? Because it'll probably be >= 6 month when new versions are going to be shipped w/ Fedora.

Thank you.

Yes, 8.5.17 & 8.6.3rc1 both work w/o issues. Hurray!

Could you add to readme a note that 8.5.16 & 8.6.2 versions are 'defective'? Because it'll probably be >= 6 month when new versions are going to be shipped w/ Fedora.

Thank you.

@pointsman

This comment has been minimized.

Show comment
Hide comment
@pointsman

pointsman Oct 30, 2014

Member

After looking at it, again, I realized that my analysis was wrong. You suffered not from http://core.tcl.tk/tcl/tktview?name=ed29c4da21 but probably from http://core.tcl.tk/tcl/tktview?name=bc5b79

The cure is the same: avoid 8.6.2 / 8.5.16. And, btw, if you use a just a bit changed script, you'll it the other 8.6.2 I/0 bug. tDOM gives you the power to explore both buggy paths ... But anyway, what was reported here has his reason more likely in http://core.tcl.tk/tcl/tktview?name=bc5b79

Member

pointsman commented Oct 30, 2014

After looking at it, again, I realized that my analysis was wrong. You suffered not from http://core.tcl.tk/tcl/tktview?name=ed29c4da21 but probably from http://core.tcl.tk/tcl/tktview?name=bc5b79

The cure is the same: avoid 8.6.2 / 8.5.16. And, btw, if you use a just a bit changed script, you'll it the other 8.6.2 I/0 bug. tDOM gives you the power to explore both buggy paths ... But anyway, what was reported here has his reason more likely in http://core.tcl.tk/tcl/tktview?name=bc5b79

@pointsman pointsman closed this Nov 22, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment