-
-
Notifications
You must be signed in to change notification settings - Fork 66
/
stream_data.ex
1825 lines (1417 loc) 路 59.6 KB
/
stream_data.ex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
defmodule StreamData do
@moduledoc """
Functions to create and combine generators.
A generator is a `StreamData` struct. Generators can be created through the
functions exposed in this module, like `constant/1`, and by combining other
generators through functions like `bind/2`.
Generators are used for mainly two purposes: to generate test data, and as the
basis of property testing. When a generator is called internally, it will
generate a value that is not a directly usable value. Instead, it's a
"wrapper" around a normal value that is optimized for property testing and,
more specifically, for *shrinking* (see the section about shrinking below).
Note that values generated by generators are not unique.
## Generation size
Generators have access to a generation parameter called the **generation
size**, which is a non-negative integer. This parameter is meant to bind the
data generated by each generator in a way that is completely up to the
generator. For example, a generator that generates integer can use the `size`
parameter to generate integers inside the `-size..size` range. In a similar
way, a generator that generates lists could use this parameter to generate a
list with `0` to `size` elements. The generation size parameter is "global",
that is, complex generators that wrap other generators usually pass it down to
the inner generators when generating data.
When creating generators, they can access the generation size using the
`sized/1` function. Generators can be resized to a fixed generation size using
`resize/2`.
## Enumeration
Generators implement the `Enumerable` protocol. The implementation yields
simple values (not shrinkable wrappers as discussed above) when enumerating a
generator. The enumeration starts with a small generation size, which
increases when the enumeration continues (up to a fixed maximum size).
For example, to get an infinite stream of integers that starts with small
integers and progressively grows the boundaries, you can just use `integer/0`:
Enum.take(StreamData.integer(), 10)
#=> [-1, 0, -3, 4, -4, 5, -1, -3, 5, 8]
Since generators are proper streams, functions from the `Stream` module can be
used to stream values out of them. For example, to build an infinite stream of
positive even integers, you can do:
StreamData.integer()
|> Stream.filter(&(&1 > 0))
|> Stream.map(&(&1 * 2))
|> Enum.take(10)
#=> [4, 6, 4, 10, 14, 16, 4, 16, 36, 16]
As we mentioned above, you can see that in general a generator does not
generate unique values.
Note that all generators are **infinite** streams that never terminate.
When using generators for test data, usually using them as enumerables is the
easiest choice.
## Shrinking
As we mentioned above, the wrapper values generated by generators internally
are optimized for **shrinking**. These wrapper values contain a generated
value (the one used when enumerating a generator) alongside a way to shrink
such value. For example, a value generated by the `list_of/1` generator will
contain the actual list to use alongside a way to shrink that list, which in
this specific case means taking elements out of that list until it comes to
the empty list (which is the "smallest" shrinkable value for lists). Each
generator shrinks generated values with its own logic, which is documented in
the generator's documentation.
Shrinking is used for property testing, where it's important to find the
minimal value for which a property fails (since generated values are often
bigger and full of garbage). See `PropertyTest` for more information.
Note that the generation size is not related in any way with shrinking: while
intuitively one may think that shrinking just means decreasing the generation
size, in reality the generation size is related to how generators generate
values, while shrinking is bound to each generated value and is a way to
shrink that particular value.
## Special generators
Some Elixir types are implicitly converted to `StreamData` generators when
used in property testing or composed. These types are:
* atoms - they generated themselves. For example, `:foo` is equivalent to
`StreamData.constant(:foo)`.
* tuples of generators - they generated tuples where each value is a value
generated by the corresponding generator, exactly like described in
`tuple/1`. For example, `{StreamData.integer(), StreamData.boolean()}`
generates entries like `{10, false}`.
Note that *these terms are only implicitly converted to generators when
composing them*. This means that these terms are not full-fledged generators:
for example, atoms cannot be enumerated directly as they don't implement the
`Enumerable` protocol. However, `StreamData.map(:foo, &Atom.to_string/1)` can
be enumerated since `:foo` is implicitly converted to a generator when passed
to a `StreamData` function.
"""
alias StreamData.LazyTree
@typep seed :: :rand.state
@typep size :: non_neg_integer
@typep generator_fun(a) :: (seed, size -> LazyTree.t(a))
@typedoc """
An opaque type that represents a `StreamData` generator that generates values
of type `a`.
Note that while this type is opaque, a generator is still guaranteed to be a
`StreamData` struct (in case you want to pattern match on it).
"""
@opaque t(a) :: %__MODULE__{
generator: generator_fun(a),
}
@rand_algorithm :exs1024
defstruct [:generator]
defmodule FilterTooNarrowError do
defexception [:max_consecutive_failures]
def message(%{max_consecutive_failures: max_consecutive_failures}) do
"too many (#{max_consecutive_failures}) consecutive elements were filtered out. " <>
"Make sure to avoid filters that are too strict and filter out too many elements " <>
"and make sure a small generation size doesn't affect the filter too heavily (such as " <>
"when empty lists are filtered out but small generation size forces generation of many " <>
"empty lists)"
end
end
defmodule TooManyDuplicatesError do
defexception [:max_tries, :remaining_to_generate, :generated]
def message(%{max_tries: max_tries, remaining_to_generate: remaining, generated: generated}) do
"too many (#{max_tries}) non-unique elements were generated consecutively. " <>
"Make sure to avoid generating from a small space of data (such as only a " <>
"handful of terms) and make sure a small generation size doesn't affect " <>
"uniqueness too heavily. There were still #{remaining} elements left to " <>
"generate, while the generated elements were:\n\n#{inspect(generated)}"
end
end
### Minimal interface
## Helpers
defp new(generator) when is_function(generator, 2) do
%__MODULE__{generator: generator}
end
# We support multiple types of generators through call/3: this is basically a
# poor implementation of a protocol (which we don't want to add just for
# this).
defp call(%__MODULE__{generator: generator}, seed, size) do
%LazyTree{} = generator.(seed, size)
end
defp call(atom, _seed, _size) when is_atom(atom) do
LazyTree.constant(atom)
end
defp call(tuple, seed, size) when is_tuple(tuple) do
case tuple_size(tuple) do
0 ->
LazyTree.constant({})
size ->
{trees, _seed} =
Enum.map_reduce(0..size - 1, seed, fn index, acc ->
{seed1, seed2} = split_seed(acc)
data = elem(tuple, index)
{call(data, seed1, size), seed2}
end)
trees
|> LazyTree.zip()
|> LazyTree.map(&List.to_tuple/1)
end
end
## Generators
@doc """
A generator that always generates the given term.
## Examples
iex> Enum.take(StreamData.constant(:some_term), 3)
[:some_term, :some_term, :some_term]
## Shrinking
This generator doesn't shrink.
"""
@spec constant(a) :: t(a) when a: var
def constant(term) do
new(fn _seed, _size -> LazyTree.constant(term) end)
end
## Combinators
@doc """
Maps the given function `fun` over the given generator `data`.
Returns a new generator that returns elements from `data` after applying `fun`
to them.
## Examples
iex> data = StreamData.map(StreamData.integer(), &Integer.to_string/1)
iex> Enum.take(data, 3)
["1", "0", "3"]
## Shrinking
This generator shrinks exactly like `data`, but with `fun` mapped over the
shrunk data.
"""
@spec map(t(a), (a -> b)) :: t(b) when a: term, b: term
def map(data, fun) when is_function(fun, 1) do
new(fn seed, size ->
data
|> call(seed, size)
|> LazyTree.map(fun)
end)
end
@doc """
Binds each element generated by `data` and to a new generator returned by
applying `fun` or filters the generated element.
Works similarly to `bind/2` but allows to filter out unwanted values. It takes
a generator `data` and invokes `fun` with each element generated by `data`.
`fun` must return one of:
* `{:cont, generator}` - `generator` is then used to generate the next
element
* `:skip` - the value generated by `data` is filtered out and a new element
is generated
Since this function acts as a filter as well, it behaves similarly to
`filter/3`: when more than `max_consecutive_failures` elements are filtered
out (that is, `fun` returns `:skip`), a `StreamData.FilterTooNarrowError` is
raised.
## Examples
Say we wanted to create a generator that generates two-element tuples where
the first element is a list of integers with an even number of members and the
second element is a member of that list. We can do that by generating a list
and, if it has even length, taking an element out of it, otherwise filtering
it out.
require Integer
list_data = StreamData.nonempty(StreamData.list_of(StreamData.integer()))
data =
StreamData.bind_filter(list_data, fn
list when Integer.is_even(length(list)) ->
inner_data = StreamData.bind(StreamData.member_of(list), fn member ->
StreamData.constant({list, member})
end)
{:cont, inner_data}
_odd_list ->
:skip
end)
Enum.at(data, 0)
#=> {[-6, -7, -4, 5, -9, 8, 7, -9], 5}
## Shrinking
This generator shrinks like `bind/2` but values that are skipped are not used
for shrinking (similarly to how `filter/3` works).
"""
@spec bind_filter(t(a), (a -> {:cont, t(b)} | :skip), non_neg_integer) :: t(b) when a: term, b: term
def bind_filter(data, fun, max_consecutive_failures \\ 10)
when is_function(fun, 1) and is_integer(max_consecutive_failures) and max_consecutive_failures >= 0 do
new(fn seed, size ->
case bind_filter(seed, size, data, fun, max_consecutive_failures) do
{:ok, lazy_tree} ->
lazy_tree
:too_many_failures ->
raise FilterTooNarrowError, max_consecutive_failures: max_consecutive_failures
end
end)
end
defp bind_filter(_seed, _size, _data, _mapper, _tries_left = 0) do
:too_many_failures
end
defp bind_filter(seed, size, data, mapper, tries_left) do
{seed1, seed2} = split_seed(seed)
lazy_tree = call(data, seed1, size)
case LazyTree.filter_map(lazy_tree, mapper) do
{:ok, filter_mapped_tree} ->
tree =
filter_mapped_tree
|> LazyTree.map(&call(&1, seed2, size))
|> LazyTree.flatten()
{:ok, tree}
:error ->
bind_filter(seed2, size, data, mapper, tries_left - 1)
end
end
@doc """
Binds each element generated by `data` to a new generator returned by applying `fun`.
This function is the basic mechanism for composing generators. It takes a
generator `data` and invokes `fun` with each element in `data`. `fun` must
return a new *generator* that is effectively used to generate items from
now on.
## Examples
Say we wanted to create a generator that returns two-element tuples where the
first element is a list, and the second element is a random element from that
list. To do that, we can first generate a list and then bind a function to
that list; this function will return the list and a random element from it.
StreamData.bind(StreamData.list_of(StreamData.integer()), fn list ->
StreamData.bind(StreamData.member_of(list), fn elem ->
StreamData.constant({list, elem})
end)
end)
## Shrinking
The generator returned by `bind/2` shrinks by first shrinking the value
generated by the inner generator and then by shrinking the outer generator
given as `data`. When `data` shrinks, `fun` is once more applied on the
shrunk value value and returns a whole new generator, which will most likely
emit new items.
"""
@spec bind(t(a), (a -> t(b))) :: t(b) when a: term, b: term
def bind(data, fun) when is_function(fun, 1) do
bind_filter(data, fn generated_term -> {:cont, fun.(generated_term)} end)
end
@doc """
Filters the given generator `data` according to the given `predicate` function.
Only elements generated by `data` that pass the filter are kept in the
resulting generator.
If the filter is too strict, it can happen that too few values generated by
`data` satisfy it. In case more than `max_consecutive_failures` consecutive
values don't satisfy the filter, a `StreamData.FilterTooNarrowError` will be
raised. Try to make sure that your filter takes out only a small subset of the
elements generated by `data`. When possible, a good way to go around the
limitations of `filter/3` is to instead transform the generated values in the
shape you want them instead of filtering out the ones you don't want.
For example, a generator of odd numbers could be implemented as:
require Integer
odd_ints = StreamData.filter(StreamData.integer(), &Integer.is_odd/1)
Enum.take(odd_ints, 3)
#=> [1, 1, 3]
However, it will do more work and take more time to generate odd integers
because it will have to filter out all the even ones that it generates. In
this case, a better approach would be to generate integers and make sure they
are odd:
odd_ints = StreamData.map(StreamData.integer(), &(&1 * 2 + 1))
Enum.take(odd_ints, 3)
#=> [1, 1, 3]
## Shrinking
All the values that each generated value shrinks to satisfy `predicate` as
well.
"""
@spec filter(t(a), (a -> as_boolean(term)), non_neg_integer) :: t(a) when a: term
def filter(data, predicate, max_consecutive_failures \\ 10)
when is_function(predicate, 1) and is_integer(max_consecutive_failures) and max_consecutive_failures >= 0 do
bind_filter(data, fn term ->
if predicate.(term) do
{:cont, constant(term)}
else
:skip
end
end, max_consecutive_failures)
end
### Rich API
@doc """
Generates an integer in the given `range`.
The generation size is ignored since the integer always lies inside `range`.
## Examples
Enum.take(StreamData.integer(4..8), 3)
#=> [6, 7, 7]
## Shrinking
Shrinks towards with the smallest absolute value that still lie in `range`.
"""
@spec integer(Range.t) :: t(integer)
def integer(_lower.._upper = range) do
new(fn seed, _size ->
range
|> uniform_in_range(seed)
|> integer_lazy_tree()
|> LazyTree.filter(&(&1 in range))
end)
end
defp integer_lazy_tree(int) do
children =
int
|> Stream.iterate(&div(&1, 2))
|> Stream.take_while(&(&1 != 0))
|> Stream.map(&(int - &1))
|> Stream.map(&integer_lazy_tree/1)
LazyTree.new(int, children)
end
## Generator modifiers
@doc """
Resize the given generated `data` to have fixed generation size `new_size`.
The new generator will ignore the generation size and always use `new_size`.
See the "Generation size" section in the documentation for `StreamData` for
more information about the generation size.
## Examples
data = StreamData.resize(StreamData.integer(), 10)
Enum.take(data, 3)
#=> [4, -5, -9]
"""
@spec resize(t(a), size) :: t(a) when a: term
def resize(data, new_size) when is_integer(new_size) and new_size >= 0 do
new(fn seed, _size ->
call(data, seed, new_size)
end)
end
@doc """
Returns the generator returned by calling `fun` with the generation size.
`fun` takes the generation size and has to return a generator, that can use
that size to its advantage.
See the "Generation size" section in the documentation for `StreamData` for
more information about the generation size.
## Examples
Let's build a generator that generates integers in double the range `integer/0`
does:
data = StreamData.sized(fn size ->
StreamData.resize(StreamData.integer(), size * 2)
end)
Enum.take(data, 3)
#=> [0, -1, 5]
"""
@spec sized((size -> t(a))) :: t(a) when a: term
def sized(fun) when is_function(fun, 1) do
new(fn seed, size ->
new_data = fun.(size)
call(new_data, seed, size)
end)
end
@doc """
Scales the generation size of the given generator `data` according to
`size_changer`.
When generating data from `data`, the generation size will be the result of
calling `size_changer` with the generation size as its argument. This is
useful, for example, when a generator needs to grow faster or slower than
the default.
See the "Generation size" section in the documentation for `StreamData` for
more information about the generation size.
## Examples
Let's create a generator that generates much smaller integers than `integer/0`
when size grows. We can do this by scaling the generation size to the
logarithm of the generation size.
data = StreamData.scale(StreamData.integer(), fn size ->
trunc(:math.log(size))
end)
Enum.take(data, 3)
#=> [0, 0, -1]
Another interesting example is creating a generator with a fixed maximum
generation size. For example, say we want to generate binaries but we never
want them to be larger than 64 bytes:
small_binaries = StreamData.scale(StreamData.binary(), fn size ->
min(size, 64)
end)
"""
@spec scale(t(a), (size -> size)) :: t(a) when a: term
def scale(data, size_changer) when is_function(size_changer, 1) do
sized(fn size ->
resize(data, size_changer.(size))
end)
end
@doc """
Makes the values generated by `data` not shrink.
## Examples
Let's build a generator of bytes (integers in the `0..255`) range. We can
build this on top of `integer/1`, but for our purposes, it doesn't make sense for
a byte to shrink towards `0`:
byte = StreamData.unshrinkable(StreamData.integer(0..255))
Enum.take(byte, 3)
#=> [190, 181, 178]
## Shrinking
The generator returned by `unshrinkable/1` generates the same values as `data`,
but such values will not shrink.
"""
@spec unshrinkable(t(a)) :: t(a) when a: term
def unshrinkable(data) do
new(fn seed, size ->
%LazyTree{root: root} = call(data, seed, size)
LazyTree.constant(root)
end)
end
@doc """
Generates values from different generators with specified probability.
`frequencies` is a list of `{frequency, data}` where `frequency` is an integer
and `data` is a generator. The resulting generator will generate data from one
of the generators in `frequency`, with probability `frequency / vsum_of_frequencies`.
## Examples
Let's build a generator that returns a binary around 25% of times and a
integer around 75% of times. We'll use `integer/0` first so that generated values
will shrink towards integers.
ints_and_some_bins = StreamData.frequency([
{3, StreamData.integer()},
{1, StreamData.binary()},
])
Enum.take(ints_and_some_bins, 3)
#=> ["", -2, -1]
## Shrinking
Each generated value is shrunk, and then this generator shrinks towards
values generated by generators earlier in the list of `frequencies`.
"""
# Right now, it shrinks by first shrinking the generated value, and then
# shrinking towards earlier generators in "frequencies". Clojure shrinks
# towards earlier generators *first*, and then shrinks the generated value.
# An implementation that does this can be:
#
# new(fn seed, size ->
# {seed1, seed2} = split_seed(seed)
# frequency = uniform_in_range(0..sum - 1, seed1)
# index = pick_index(Enum.map(frequencies, &elem(&1, 0)), frequency)
# {_frequency, data} = Enum.fetch!(frequencies, index)
#
# tree = call(data, seed2, size)
#
# earlier_children =
# frequencies
# |> Stream.take(index)
# |> Stream.map(&call(elem(&1, 1), seed2, size))
# LazyTree.new(tree.root, Stream.concat(earlier_children, tree.children))
# end)
#
@spec frequency([{pos_integer, t(a)}]) :: t(a) when a: term
def frequency(frequencies) when is_list(frequencies) do
sum = Enum.reduce(frequencies, 0, fn {frequency, _data}, acc -> acc + frequency end)
bind(integer(0..sum - 1), &pick_frequency(frequencies, &1))
end
defp pick_frequency([{frequency, data} | rest], int) do
if int < frequency do
data
else
pick_frequency(rest, int - frequency)
end
end
@doc """
Generates values out of one of the given `datas`.
`datas` must be a list of generators. The values generated by this generator
are values generated by generators in `datas`, chosen each time at random.
## Examples
data = StreamData.one_of([StreamData.integer(), StreamData.binary()])
Enum.take(data, 3)
#=> [-1, <<28>>, ""]
## Shrinking
The generated value will be shrunk first according to the generator that
generated it, and then this generator will shrink towards earlier generators
in `datas`.
"""
@spec one_of([t(a)]) :: t(a) when a: term
def one_of([_ | _] = datas) do
bind(integer(0..length(datas) - 1), fn index ->
Enum.fetch!(datas, index)
end)
end
@doc """
Generates elements taken randomly out of `enum`.
`enum` must be a non-empty and **finite** enumerable. If given an empty
enumerable, this function raises an error. If given an infinite enumerable,
this function will not terminate.
## Examples
Enum.take(StreamData.member_of([:ok, 4, "hello"]), 3)
#=> [4, 4, "hello"]
## Shrinking
This generator shrinks towards elements that appear earlier in `enum`.
"""
@spec member_of(Enumerable.t) :: t(term)
def member_of(enum) do
enum_length = Enum.count(enum)
if enum_length == 0 do
raise "cannot generate elements from an empty enumerable"
end
bind(integer(0..enum_length - 1), fn index ->
constant(Enum.fetch!(enum, index))
end)
end
## Compound data types
@doc """
Generates lists where each values is generated by the given `data`.
Each generated list can contain duplicate elements. The length of the
generated list is bound by the generation size. If the generation size is `0`,
the empty list will always be generated. Note that the accepted options
provide finer control over the size of the generated list. See the "Options"
section below.
## Options
* `:length` - (integer or range) if an integer, the exact length the
generated lists should be; if a range, the range in which the length of
the generated lists should be. If provided, `:min_length` and
`:max_length` are ignored.
* `:min_length` - (integer) the minimum length of the generated lists.
* `:max_length` - (integer) the maximum length of the generated lists.
## Examples
Enum.take(StreamData.list_of(StreamData.binary()), 3)
#=> [[""], [], ["", "w"]
Enum.take(StreamData.list_of(StreamData.integer(), length: 3), 3)
#=> [[0, 0, -1], [2, -1, 1], [0, 3, -3]]
Enum.take(StreamData.list_of(StreamData.integer(), max_length: 1), 3)
#=> [[1], [], []]
## Shrinking
This generator shrinks by taking elements out of the generated list and also
by shrinking the elements of the generated list. Shrinking still respects any
possible length-related option: for example, if `:min_length` is provided, all
shrinked list will have more than `:min_length` elements.
"""
# We could have an implementation that relies on fixed_list/1 and List.duplicate/2,
# it would look like this:
#
# new(fn seed, size ->
# {seed1, seed2} = split_seed(seed)
# length = uniform_in_range(0..size, seed1)
# data
# |> List.duplicate(length)
# |> fixed_list()
# |> call(seed2, size)
# |> LazyTree.map(&list_lazy_tree/1)
# |> LazyTree.flatten()
# end)
#
@spec list_of(t(a), keyword) :: t([a]) when a: term
def list_of(data, options \\ []) do
list_length_range_fun = list_length_range_fun(options)
new(fn seed, size ->
{seed1, seed2} = split_seed(seed)
min_length.._ = length_range = list_length_range_fun.(size)
length = uniform_in_range(length_range, seed1)
data
|> call_n_times(seed2, size, length, [])
|> LazyTree.zip()
|> LazyTree.map(&list_lazy_tree(&1, min_length))
|> LazyTree.flatten()
end)
end
defp list_length_range_fun(options) do
{min, max} =
case Keyword.fetch(options, :length) do
{:ok, length} when is_integer(length) and length >= 0 ->
{length, length}
{:ok, min..max} when min >= 0 and max >= 0 ->
{min(min, max), max(min, max)}
{:ok, other} ->
raise ArgumentError, ":length must be a positive integer or a range " <>
"of positive integers, got: #{inspect(other)}"
:error ->
min_length = Keyword.get(options, :min_length, 0)
max_length = Keyword.get(options, :max_length, :infinity)
unless is_integer(min_length) and min_length >= 0 do
raise ArgumentError, ":min_length must be a positive integer, got: #{inspect(min_length)}"
end
unless (is_integer(max_length) and max_length >= 0) or max_length == :infinity do
raise ArgumentError, ":max_length must be a positive integer, got: #{inspect(max_length)}"
end
{min_length, max_length}
end
fn size -> min..(max |> min(size) |> max(min)) end
end
defp call_n_times(_data, _seed, _size, 0, acc) do
acc
end
defp call_n_times(data, seed, size, length, acc) do
{seed1, seed2} = split_seed(seed)
call_n_times(data, seed2, size, length - 1, [call(data, seed1, size) | acc])
end
defp list_lazy_tree(list, min_length) do
length = length(list)
if length == min_length do
LazyTree.constant(list)
else
children =
0..length - 1
|> Stream.map(&List.delete_at(list, &1))
|> Stream.map(&list_lazy_tree(&1, min_length))
LazyTree.new(list, children)
end
end
@doc """
Generates a list of elements generated by `data` without duplicates (possibly
according to a given uniqueness function).
This generator will generate lists where each list is unique according to the
value returned by applying the given uniqueness function to each element
(similarly to how `Enum.uniq_by/2` works). If more than the value of the
`:max_tries` option consecutive elements are generated that are considered
duplicates according to the uniqueness function, a
`StreamData.TooManyDuplicatesError` error is raised. For this reason, try to
make sure to not make the uniqueness function return values out of a small
value space. The uniqueness function and the max number of tries can be
customized via options.
## Options
* `:uniq_fun` - (a function of arity one) a function that is called with
each generated element and whose return value is used as the value to
compare with other values for uniqueness (similarly to `Enum.uniq_by/2`).
* `:max_tries` - (non-negative integer) the maximum number of times that
this generator tries to generate the next element of the list before
giving up and raising a `StreamData.TooManyDuplicatesError` in case it
can't find a unique element to generate. Note that the generation size
often affects this: for example, if you have a generator like
`uniq_list_of(integer(), min_length: 4)` and you start generating elements
out of it with a generation size of `1`, it will fail by definition
because `integer/0` generates in `-size..size` so it would only generate
in a set (`[-1, 0, 1]`) with three elements. Use `resize/2` or `scale/2`
to manipulate the size (for example by setting a minimum generation size
of `3`) in such cases.
* `:length` - (non-negative integer) same as in `list_of/3`.
* `:min_length` - (non-negative integer) same as in `list_of/3`.
* `:max_length` - (non-negative integer) same as in `list_of/3`.
## Examples
data = StreamData.uniq_list_of(StreamData.integer())
Enum.take(data, 3)
#=> [[1], [], [2, 3, 1]]
## Shrinking
This generator shrinks like `list_of/1`, but the shrunk values are unique
according to the `:uniq_fun` option as well.
"""
@spec uniq_list_of(t(a), keyword) :: t([a]) when a: term
def uniq_list_of(data, options \\ []) do
uniq_fun = Keyword.get(options, :uniq_fun, &(&1))
max_tries = Keyword.get(options, :max_tries, 10)
list_length_range_fun = list_length_range_fun(options)
new(fn seed, size ->
{seed1, seed2} = split_seed(seed)
min_length.._ = length_range = list_length_range_fun.(size)
length = uniform_in_range(length_range, seed1)
data
|> uniq_list_of(uniq_fun, seed2, size, _seen = MapSet.new(), max_tries, max_tries, length, _acc = [])
|> LazyTree.zip()
|> LazyTree.map(&list_lazy_tree(&1, min_length))
|> LazyTree.flatten()
|> LazyTree.map(&Enum.uniq_by(&1, uniq_fun))
|> LazyTree.filter(&(length(&1) >= min_length))
end)
end
defp uniq_list_of(_data, _uniq_fun, _seed, _size, seen, _tries_left = 0, max_tries, remaining, _acc) do
raise TooManyDuplicatesError, max_tries: max_tries, remaining_to_generate: remaining, generated: seen
end
defp uniq_list_of(_data, _uniq_fun, _seed, _size, _seen, _tries_left, _max_tries, _remaining = 0, acc) do
acc
end
defp uniq_list_of(data, uniq_fun, seed, size, seen, tries_left, max_tries, remaining, acc) do
{seed1, seed2} = split_seed(seed)
tree = call(data, seed1, size)
key = uniq_fun.(tree.root)
if MapSet.member?(seen, key) do
uniq_list_of(data, uniq_fun, seed2, size, seen, tries_left - 1, max_tries, remaining, acc)
else
uniq_list_of(data, uniq_fun, seed2, size, MapSet.put(seen, key), max_tries, max_tries, remaining - 1, [tree | acc])
end
end
@doc ~S"""
Generates non-empty improper lists where elements of the list are generated
out of `first` and the improper ending out of `improper`.
## Examples
data = StreamData.nonempty_improper_list_of(StreamData.byte(), StreamData.binary())
Enum.take(data, 3)
#=> [["\f"], [56 | <<140, 137>>], [226 | "j"]]
## Shrinking
Shrinks towards smaller lists (that are still non-empty, having the improper
ending) and towards shrunk elements of the list and a shrunk improper
ending.
"""
@spec nonempty_improper_list_of(t(a), t(b)) :: t(nonempty_improper_list(a, b)) when a: term, b: term
def nonempty_improper_list_of(first, improper) do
map({list_of(first), improper}, fn
{[], ending} ->
[ending]
{list, ending} ->
List.foldr(list, ending, &[&1 | &2])
end)
end
@doc """
Generates lists of elements out of `first` with a chance of them being
improper with the improper ending taken out of `improper`.
Behaves similarly to `nonempty_improper_list_of/2` but can generate empty
lists and proper lists as well.
## Examples
data = StreamData.maybe_improper_list_of(StreamData.byte(), StreamData.binary())
Enum.take(data, 3)
#=> [[60 | "."], [], [<<212>>]]
## Shrinking
Shrinks towards smaller lists and shrunk elements in those lists, and
ultimately towards proper lists.
"""
@spec maybe_improper_list_of(t(a), t(b)) :: t(maybe_improper_list(a, b)) when a: term, b: term
def maybe_improper_list_of(first, improper) do
frequency([
{2, list_of(first)},
{1, nonempty_improper_list_of(first, improper)},
])
end
@doc """
Generates a list of fixed length where each element is generated from the
corresponding generator in `data`.
## Examples
data = StreamData.fixed_list([StreamData.integer(), StreamData.binary()])
Enum.take(data, 3)
#=> [[1, <<164>>], [2, ".T"], [1, ""]]
## Shrinking
Shrinks by shrinking each element in the generated list according to the
corresponding generator. Shrunk lists never lose elements.
"""
@spec fixed_list([t(a)]) :: t([a]) when a: term
def fixed_list(datas) when is_list(datas) do
new(fn seed, size ->
{trees, _seed} =
Enum.map_reduce(datas, seed, fn data, acc ->
{seed1, seed2} = split_seed(acc)
{call(data, seed1, size), seed2}
end)
LazyTree.zip(trees)
end)
end
@doc """
Generates tuples where each element is taken out of the corresponding
generator in the `tuple_datas` tuple.
## Examples
data = StreamData.tuple({StreamData.integer(), StreamData.binary()})
Enum.take(data, 3)
#=> [{-1, <<170>>}, {1, "<"}, {1, ""}]
## Shrinking
Shrinks by shrinking each element in the generated tuple according to the
corresponding generator.
"""
@spec tuple(tuple) :: t(tuple)
def tuple(tuple_datas) when is_tuple(tuple_datas) do
new(fn seed, size -> call(tuple_datas, seed, size) end)
end
@doc """
Generates maps with keys from `key_data` and values from `value_data`.
Since maps require keys to be unique, this generator behaves similarly to
`uniq_list_of/3`: if more than `max_tries` duplicate keys are generated
consequently, it raises a `StreamData.TooManyDuplicatesError` exception.
## Examples
Enum.take(StreamData.map_of(StreamData.integer(), StreamData.boolean()), 3)
#=> [%{}, %{1 => false}, %{-2 => true, -1 => false}]
## Shrinking
Shrinks towards smallest maps and towards shrinking keys and values according
to the respective generators.
"""
@spec map_of(t(key), t(value)) :: t(%{optional(key) => value}) when key: term, value: term
def map_of(key_data, value_data, max_tries \\ 10) do