Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Test reading larger documents

  • Loading branch information...
commit ddf2303807735ef64bf6ce1ac8f91df2b81c6cb7 1 parent 63518a4
@chansen chansen authored
Showing with 157 additions and 7 deletions.
  1. +138 −0 corpus/quickbrown.txt
  2. +19 −7 t/basics.t
View
138 corpus/quickbrown.txt
@@ -0,0 +1,138 @@
+Sentences that contain all letters commonly used in a language
+--------------------------------------------------------------
+
+Markus Kuhn <http://www.cl.cam.ac.uk/~mgk25/> -- 2010-09-20
+
+This is an example of a plain-text file encoded in UTF-8.
+
+
+Danish (da)
+---------
+
+ Quizdeltagerne spiste jordbær med fløde, mens cirkusklovnen
+ Wolther spillede på xylofon.
+ (= Quiz contestants were eating strawbery with cream while Wolther
+ the circus clown played on xylophone.)
+
+German (de)
+-----------
+
+ Falsches Üben von Xylophonmusik quält jeden größeren Zwerg
+ (= Wrongful practicing of xylophone music tortures every larger dwarf)
+
+ Zwölf Boxkämpfer jagten Eva quer über den Sylter Deich
+ (= Twelve boxing fighters hunted Eva across the dike of Sylt)
+
+ Heizölrückstoßabdämpfung
+ (= fuel oil recoil absorber)
+ (jqvwxy missing, but all non-ASCII letters in one word)
+
+Greek (el)
+----------
+
+ Γαζέες καὶ μυρτιὲς δὲν θὰ βρῶ πιὰ στὸ χρυσαφὶ ξέφωτο
+ (= No more shall I see acacias or myrtles in the golden clearing)
+
+ Ξεσκεπάζω τὴν ψυχοφθόρα βδελυγμία
+ (= I uncover the soul-destroying abhorrence)
+
+English (en)
+------------
+
+ The quick brown fox jumps over the lazy dog
+
+Spanish (es)
+------------
+
+ El pingüino Wenceslao hizo kilómetros bajo exhaustiva lluvia y
+ frío, añoraba a su querido cachorro.
+ (Contains every letter and every accent, but not every combination
+ of vowel + acute.)
+
+French (fr)
+-----------
+
+ Portez ce vieux whisky au juge blond qui fume sur son île intérieure, à
+ côté de l'alcôve ovoïde, où les bûches se consument dans l'âtre, ce
+ qui lui permet de penser à la cænogenèse de l'être dont il est question
+ dans la cause ambiguë entendue à Moÿ, dans un capharnaüm qui,
+ pense-t-il, diminue çà et là la qualité de son œuvre.
+
+ l'île exiguë
+ Où l'obèse jury mûr
+ Fête l'haï volapük,
+ Âne ex aéquo au whist,
+ Ôtez ce vœu déçu.
+
+ Le cœur déçu mais l'âme plutôt naïve, Louÿs rêva de crapaüter en
+ canoë au delà des îles, près du mälström où brûlent les novæ.
+
+Irish Gaelic (ga)
+-----------------
+
+ D'fhuascail Íosa, Úrmhac na hÓighe Beannaithe, pór Éava agus Ádhaimh
+
+Hungarian (hu)
+--------------
+
+ Árvíztűrő tükörfúrógép
+ (= flood-proof mirror-drilling machine, only all non-ASCII letters)
+
+Icelandic (is)
+--------------
+
+ Kæmi ný öxi hér ykist þjófum nú bæði víl og ádrepa
+
+ Sævör grét áðan því úlpan var ónýt
+ (some ASCII letters missing)
+
+Japanese (jp)
+-------------
+
+ Hiragana: (Iroha)
+
+ いろはにほへとちりぬるを
+ わかよたれそつねならむ
+ うゐのおくやまけふこえて
+ あさきゆめみしゑひもせす
+
+ Katakana:
+
+ イロハニホヘト チリヌルヲ ワカヨタレソ ツネナラム
+ ウヰノオクヤマ ケフコエテ アサキユメミシ ヱヒモセスン
+
+Hebrew (iw)
+-----------
+
+ ? דג סקרן שט בים מאוכזב ולפתע מצא לו חברה איך הקליטה
+
+Polish (pl)
+-----------
+
+ Pchnąć w tę łódź jeża lub ośm skrzyń fig
+ (= To push a hedgehog or eight bins of figs in this boat)
+
+Russian (ru)
+------------
+
+ В чащах юга жил бы цитрус? Да, но фальшивый экземпляр!
+ (= Would a citrus live in the bushes of south? Yes, but only a fake one!)
+
+ Съешь же ещё этих мягких французских булок да выпей чаю
+ (= Eat some more of these fresh French loafs and have some tea)
+
+Thai (th)
+---------
+
+ [--------------------------|------------------------]
+ ๏ เป็นมนุษย์สุดประเสริฐเลิศคุณค่า กว่าบรรดาฝูงสัตว์เดรัจฉาน
+ จงฝ่าฟันพัฒนาวิชาการ อย่าล้างผลาญฤๅเข่นฆ่าบีฑาใคร
+ ไม่ถือโทษโกรธแช่งซัดฮึดฮัดด่า หัดอภัยเหมือนกีฬาอัชฌาสัย
+ ปฏิบัติประพฤติกฎกำหนดใจ พูดจาให้จ๊ะๆ จ๋าๆ น่าฟังเอย ฯ
+
+ [The copyright for the Thai example is owned by The Computer
+ Association of Thailand under the Royal Patronage of His Majesty the
+ King.]
+
+Special thanks to the people from all over the world who contributed
+these sentences.
View
26 t/basics.t
@@ -14,16 +14,28 @@ binmode $builder->output, ":utf8";
binmode $builder->failure_output, ":utf8";
binmode $builder->todo_output, ":utf8";
-my $filename = catfile(qw/corpus test1.txt/);
-open my $fh, '<:utf8_strict', $filename or die "Couldn't open file $filename";
+{
+ my $filename = catfile(qw/corpus test1.txt/);
+ open my $fh, '<:utf8_strict', $filename or die "Couldn't open file $filename";
-my $line = <$fh>;
+ my $line = <$fh>;
-is($line, "Foö-Báŗ\n", 'Content is Foö-Báŗ');
+ is($line, "Foö-Báŗ\n", 'Content is Foö-Báŗ');
+}
-my $filename2 = catfile(qw/corpus test1-latin1.txt/);
-open my $fh2, '<:utf8_strict', $filename2 or die "Couldn't open file $filename2";
+{
+ my $filename = catfile(qw/corpus quickbrown.txt/);
+ open my $fh, '<:utf8_strict', $filename or die "Couldn't open file $filename";
-throws_ok { $line = <$fh2> } qr/^Invalid unicode character at /, 'Trying to read invalidly encoded utf8 fails' or diag "Just read '$line'";
+ lives_ok { my $data = do { local $/; <$fh> } } 'successfull reading quickbrown.txt'
+}
+
+{
+ my $filename = catfile(qw/corpus test1-latin1.txt/);
+ open my $fh, '<:utf8_strict', $filename or die "Couldn't open file $filename";
+
+ my $line;
+ throws_ok { $line = <$fh> } qr/^Invalid unicode character at /, 'Trying to read ill-formed encoded UTF-8 fails' or diag "Just read '$line'";
+}
done_testing;
Please sign in to comment.
Something went wrong with that request. Please try again.