Strings generated by default should have more usual characters

[vi]

Strings typically generted look like this:

򒽹򪬛򘌟󥫹񕘼򟶹񍀝󿡥񰫩􂒡𻘒򅇪򍸈򁑙򟪆󴥏􅲴󩇇Ք𡫗󼺆򺩪𾺗𵅟𻎚񦎅񳜿󔈐ᘤ񜾊񺥿򍔛󆬭𙚹񃃣񥊆򇆗򂤇􀑬󨍏󉨫򺌑񽙕񷗮󻐉𿞊񹩵򖬊񣁇󘅛󖛉𼄠󄚷򽕎

񡰎𣒌󹣵

􅘤󀞶񼮳𮕇򀁕񗎰򀇦򹖻񚂫󚾪򃢛ൾ𬫤񘖤􇾷򇍤񮅈弅򥅽񨭴󶈡񫯭𧚼䝠Ᏼ􌷞󻃃񛑓󵎍𯭜􄼧򛷺񚂛񊪧󯱩񩴝𛭑𨀒񡥍󛢘񺐢𵑌眻󖙞񅉊򛻨񆷮󸧖󵔸񎽔򡚥񂼮򆼤񄓭񷬗􂭡򼠀񥉑󓊢򡽍􀟞񮆀񿫇񿌝򒋏񜎀񢝒򋯼񹌈􃔪򈟙󟬠󀧸󀉖𠥤񌡭𧆼񦛼񶰆񂠑𻗛򙄒

򁂺󶼀𰷹𾏉򮘭򈝥󓞤򫚁󙖘󆖙𰻟񇫙𴋅𠈼󹌀凾𖙟𠓕𾭭𗢐񥂡񇫘򫯝󨃖僧󰲺򘳑𛉏񡴶􏹊򔌇ᘼ􎜅󇆯󈤟󷃚􎞎񦖄봄򬩘񎩭򓲜󍥊𰀈􍆔񥝖򅈳𿴛􌆗񍏰񛯚􏡩򵆠񝭆􍘣򅕜򞙄󬍦񜟩󐮐򊽛󘉏񔀊񎇝󞏏񥍇󕎎𑝩񡨭򋡠򪙃󆂵𫟯񩽗𕡫򞠇𤷸򙺛򄎩򍮂򒹣䴒𻉜󸅩􎬙𷆊񲎀񪨌򌀀􄻖񇋒󹜅

񒢩򊟗𥁿򌅞񬕧𞎂񮕄򞿮ҙ󐢉󠢜񭈑󕓴󅸛򕎅𳞇񪉩񞢜򭧘󛦦󎪦󱏚홀񍋧篬󦲫򈧹񍔿񚣪򛍾𡚙󫬴򽉫񈝗񔁯󹕩񦟠󼝹𦼄󹆲񜋑񠇒񇽖􋤋񤔇󐂚񼛍􄼀󮀯𔯀󦃂񵒵򄯎󩎅򓊕򯠎񌬆롇񘺊񵺾𕧨񚾓򍚬󷈮󠢧򦂼𹥞󮾛򼢁􀏜򸬟󮃣򪾂򬁷񯕞񋓝󞭤󰥔񐮃󲏫󻞻򻬒􄑎򉗭򄫛󨶇󔽟򹑹򼹨򨵖񰊅񻣊

񕂍􌥩񵉗񶗖𬉞魊񞦹󙤵𦪀𒮡󄮽𕗠񥽕􂥦􈲹󙁡򞞌𵄐𳎩뚲󄬥򶰂񷷢򝬉񞝟򃹪腎󑍭򒋀򔲙󉺿񲥓𝹯󩬑

񾧍󝫮򎿛窯񕓨􈲶񽶛򩪣򁹡􂏔򛘎򘇀𠷺񎂦񭭸񽎊𐊊𨺿򆊺𗞩􌽂񁸶輬񄃸񅙜񐢳񔧏񈙄𝐳𪉛

𥹠𼞳󾢙𔅿􊭷񾣐򎕊񭳖򕸌󎀛򶃣𜿉􀢛󶄿𲊊񪟛񈘉񲻦񭩂󳄒𾺲𯆊񏸆񌼠񭳒񷞈򁓎􈋖𚜸󿒲򁔀𰤫򫥑򞼲𥐸񥾥󓮿򳖱𗋕󳉋􉠨𖍳󷕝򝘊򏩰󁺱񍰉崳󀁈𒯻򿮥𯙇񥻂󓿭򫙎񄦽񠷎𞸁񵳳󿧻􂯓󹗄򇪾󽘝󉵷򗚫񸐄󘁳񏶌񥵾󧒟򮫰𖪄񞼗􏾢𶕘򊮴𗰼򄌋񺿭򣘸

򵑆򢀍򯪂񏁾𧔥󼖈򬿚󰖓􁯌򁆙񎠤񬴞𝒲󹩝򾜣񰢝󭙯񠵛񹘦󒳼񡫉󺲢񘾥񿔷󾣎埙󊽑򼪐򊰭񠮺禹󾣣𼵝򒹐񓪊󫨬򫄒󦉂񇫄󬆿򻃃񯟶󜚓𷗚𲦂󻔴񮏂򶰎񥊢񰵪񢔀򁻢喒󢷅󟽄񚐚񓼈򭶸􅇆󭏂𜯃𨖉󑶧򪌿𨌽򄔖𱢼򩱼򫮴

𻥻򖿄󎰁𔱷𑒂𼟊􇼅𲂄􈔜󉄘􋫢𖔄񋺆𬎽򛖢򠬔􃸫􀵆𮘜󛥺􂶐򵄤𺐅񀕛񟥠􏽑󹫠򡧄󒸡򰎜򊎰񥑌񇇣𩔒򰅈򵄯񓣏󓢍򑛛򉨸𾓊𷅣𵼤𦌘򓀞𽣟򞴍

񒚞񙶴󫃔񀹮񣍯𵘣󾈣򐜴󄌿𦶞󯛫󄊁񳍮񚶨󡷣𸫂𞳊񏜪󜮶𺾆㮄񆡐󓶭﯈󊎊󿧺􏋔󤍄󁾒򣎦񛦋򯠖􍓙񜠔򇺱𓆦񟣤󝢧򱁕񗽳󶱮󶊼󿓊򾚓򵵆򼫏󄈬򜒕󱦡񘗏򮑏󔼋򂄸󺺘𨛽󚲁񶗜󖿭󣠯򆱩񰊫憎񳥒񼷵𠗆񍻠󽖾𿂆󑓓񎚯򦉠𘊲

𳸈򋸴󂩗񽰋񄨪񏉙򀤵󕳷𚱲򵚑󓶵񀐡󈁡򮖵񜬃񫡵񠬳񬶊𡱭񅼺򝙾񞬁ꈺ𹖉𵌛ロ񒁭􉫒󋟑񔊢򽲓񿨈񑜰񇿴򢣖􍟣򰮍󿘱󄡣󹉚𬥿񋘙𔭦򡎑

󰺿󶄔𬆷󱂌󉬞󾪢񖖖񮊕񝧃򾽼񨤳𸾂𣑌󸕖󹴌𬜄𑼦􌷱򩔤𖌸񥺼񗎱񗱐󲣑𴒨󴹒𹒪󋊌󨴖򙀁򁿵񫘕󐇯򂯲񵕌񁢜󹤨򙢄񨓩򄠑񻨸𻍦󦩒񬪸󻴴񜀛򝇩񼗕򣚉󵏛󣹥򉭞򦚎񀺲񆛜󮟏񟒫𲹇򋼡򷧘񿩬󎭝򪆀𛑛񻉞󝫨𡞱㆒

򼋡󯻭􁤛񎔝󸴭󜈰񱚱픦󎍆𢐻򷟝񞮵蘪󖣢𫭪𲊺󿜚񈅗򿧙󿴞񫌱衯򉕭򊞴қ󴙠𱜯񙐰򍊌񮨬󪾪򮝂𾨆񱘾񡠇팢򗡗򽑶򴸟𙅡򎋎𐠮󶖉򄡏󟛿􏠰򟿢񚨬憲򭫉􉌞񘮝󗕖񡏤𮺀񫐀𔚈񷋭𙪢򇬣򛡡񗚡򓛏򘌶𦷴񶵍򮵔숥𲟪

񘿘𛀦򵊻󠑜󼾐𙈨󹲃𸯕𜆦淅󍵮𰂘񛦑󛫌󝌲󔕇󹙮񧃷񧈵񠶄򙹫󓰆򒈐􃎢邮򣻖񩹖𓋱𥥫񩘆

It fails (or takes long) to find simple case like a when something starts with a space, special character or like this.

I think normal characters should be preferred to deeply Unicode things.

I think by default there should be tiers of characters classes:

1. Bug-prone characters: whitespace, punctuation, control characters, maybe up to 5 selected Unicode queeries like zero-width space;
2. [a-zA-Z0-9_];
3. Unicode characters in the first plane;
4. Everything else.

And character generator may aim for, for example, 10% of buggy chars, 40% of [a-zA-Z0-9_], 40% of basic Unicode characters, 10% of everything else.

[BurntSushi]

This isn't a bad idea. It used to be ASCII only. But clearly, that is also an extreme. I often find myself defining new types that restrict the set of possible values to make debugging easier.

See also: #77

[shepmaster]

Copying my idea from #77:

I could conceive of a string with only ASCII as being "smaller" than one with ASCII + ASCII punctuation. Then it could "grow" to include more common Unicode, then "grow" towards uncommon.

[vi]

@shepmaster, Maybe first uncommon/tricky (zero-width things, BOM, left-to-right), then numerous common?

Each Unicode codepoint has equal weight (time required to test with it), but unequal usefullness (probability that this codepoint catches some bug).

[shepmaster]

It's an interesting thing - what is the most useful order to iterate though test cases? I'd think most people using quickcheck would want it to find things that they haven't thought of (at least it's true for me!). However, once something is found, we want it to reduce it to something that we can wrap our brains around.

I think that "simple ASCII" will often be the easy-to-understand group of characters. The problem is going to be that different usages of Strings will have different "tricky" bits. Perhaps your area of code is more likely to have issues with BOMs, but mine with control characters. I'd doubt there's One True Order.

[vi]

But I expect that a problem will rarely come up with, for example, character U+12345 CUNEIFORM SIGN URU TIMES KI (𒍅) exactly (and not with other high-plane characters). Yet including all high-plane characters significantly increase the testing space and outnumbers more useful characters. So for "lesser" strings you can leave just one high-plane character.

Imagine the table:

Option	Weight	Usefulness notes
Don't include high-plane characters in "easy set"	smallest	Won't catch respective problems
Include just one high-plane character in "easy set"	small	Likely to catch the problem with such characters
Include all high-plane characters in "easy set"	big	Only slightly more chance to catch such problems compared to the previous row

Small character classes (control characters, whitespace) should be included entirely.

[BurntSushi]

I somewhat feel like the obvious behavior for String is the current behavior: any Unicode codepoint is fair game.

With that said, there's no reason why quickcheck couldn't define a few other newtypes around String that correspond to useful subsets of Unicode. (If we go that route, I would prefer to the keep the number of such types in quickcheck proper very small.)

[vi]

What does mean "a fair game"? In my idea any codepoint can appear, but probability should be drastically different.

Useful subset may fail to find a problem even if running long enough.

Option	Speed	Immediate results	Long-term results
All codepoints regularly distributed (current)	slow	few	all
"Useful subset"	fast	moderate	moderate
All codepoints, but not regularly distributed (proposed)	medium	moderate	all

For example, if a function breaks just when being fed a string with three spaces in a row, I expect it to find it fast. If a function only breaks when being fed with tree 𒍅s in a row, that is expected to be found out slower (because of space is must more popular for bugs that some arbitrary character).

[BurntSushi]

@vi Ah, I see, I misunderstood. I think I'm fine with a smarter impl of String.

[vi]

@BurntSushi, Maybe smarter impl of char? Do you feel OK if arbitrary char would not be regularly distributed and would prefer some characters?

[BurntSushi]

I think that might be OK.

[vi]

Can such logic be also applied to u32 and friends (making things like 0,1,2,-1,0x80000000 more popular) ?

Probably [0,0,0,0] can trigger more bugs than [1582149423,1582149423,1582149423,1582149423].

[BurntSushi]

Sounds like a good idea to me!

[BurntSushi]

I wonder if it'd be worth looking at what other ports of quickcheck do. Does the Haskell quickcheck do anything fancy like this? If not, did they consider it?

[vi]

Asked on IRC.

http://haddock.stackage.org/lts-3.17/QuickCheck-2.8.1/src/Test-QuickCheck-Arbitrary.html#line-471

kadoban> _Vi: So it basically only ever picks characters between 0 and 255, and it's biased towards 0 to 128

[FranklinChen]

This is a known crappy Haskell QuickCheck default that has bitten many people. The standard workaround is http://hackage.haskell.org/package/quickcheck-unicode

[vi]

Shall I try submitting a pull request about this, making it generate chars a bit like aforementioned quickcheck-unicode (but with some emphasis on whitespace, special characters and specific tricky Unicode characters).?

[BurntSushi]

@vi That would be lovely!

[BurntSushi]

Done with PR #116 in commit faed60d. Thanks @vi!

[vi]

QuickCheck's string generator's motto should be "I love characters you hate".

[BurntSushi]

@vi Haha, I like it!