Provide Vavr randomizers

[Sir4ur0n]

I recently discovered your library and it's quite useful, thank you!

That being said, we massively use Vavr (https://github.com/vavr-io/vavr) collections instead of JDK ones, because of their immutability and more functional features/APIs.
Of course Random Beans don't work OOTB with Vavr collections, we get this kind of exceptions:

io.github.benas.randombeans.api.ObjectGenerationException: Unable to generate a random instance of type class Foo
Caused by: io.github.benas.randombeans.api.ObjectGenerationException: Unable to create type: io.vavr.collection.List for field: strings of class: Foo
Caused by: io.github.benas.randombeans.api.ObjectGenerationException: Unable to create an instance of type: interface io.vavr.collection.List
Caused by: java.lang.InstantiationError: io.vavr.collection.List

I took a quick look at your code and documentation (in particular https://github.com/benas/random-beans/wiki/Supported-types) and came to the conclusion that it would be possible, at least by manually writing Randomizer classes in the same spirit as SetRandomizer, ListRandomizer, etc.

Would that make sense to add support for it?
Please note that most common Vavr collections (like List, Set, Map) provide factory methods to build from JDK List/Set/Map (e.g. io.vavr.collection.List.ofAll(java.lang.Iterable)).

I'm ok with helping (though I would let you the fun part of creating a new module etc. if you wish/accept to expose it as another Random Beans module) 😄.

Cheers

[Sir4ur0n]

I cloned this repo and fiddled with the code to support at least a Vavr List type. I managed to make it work, but actually I had to touch inner code, I couldn't simply add a custom registry as I hoped to at first.
I modified io.github.benas.randombeans.FieldPopulator#generateRandomValue, created a new VavrPopulator (a copy of CollectionPopulator which builds a Vavr List type) and modified a few classes consequently.

Is this the right approach? If yes, how would you suggest we move on? Would you at least consider to be able to plug on the collection detection type? So that we can provide some kind of registry to handle Vavr types (which technically are not Collections, the only common ancestor they share with Collection is Iterable).

I appreciate any advice.

Cheers

[benas]

Hi @Sir4ur0n

Thank you for this suggestion! I remember having the same suggestion for guava collections but I decided to not add support for it as it was moving very fast (I see they are at v27 now, at that time it was 14 IICR) and I was pretty sure I could not catch up.. Some people also wanted support for eclipse collections and now it's vavr's turn. It would be really awesome to add support for these third party libraries, but this also means a maintenance burden I simply could not support. I hope you understand this situation.

If you want to add support for vavr, please go ahead and do it in a repo of your own, I would be more than happy to help and add a reference to it on the home page of RB!

Now in regards to how I would implement this, the way to go IMO is to use a custom registry for vavr types.

I couldn't simply add a custom registry as I hoped to at first

What was the issue? Something like the following should be enough:

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    private final Map<Class<?>, Randomizer<?>> randomizers = new HashMap<>();

    @Override
    public void init(EnhancedRandomParameters parameters) {
        randomizers.put(io.vavr.collection.List.class, new ListRandomizer<>()); // todo implement a ListRandomizer for vavr's List type
        // register other randomizers for vavr types
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        return getRandomizer(field.getType());
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return randomizers.get(type);
    }
}

Kr,
Mahmoud

[Sir4ur0n]

Hi @benas

Thanks for your answer.
I perfectly understand you don't want to support other libraries like Vavr or Guava yourself (the maintenance burden would be high).

That being said, I don't see how to implement your solution with a registry. I guess in your example, VavrListRandomizer should implement Randomizer<io.vavr.collection.List<T>>, correct?
I don't see how this is possible, since I don't have any mean of creating a T.

public class VavrListRandomizer<T> implements Randomizer<List<T>> {

  @Override
  public List<T> getRandomValue() {
    // Impossible, generic type classes can't be captured
    return List(EnhancedRandom.random(T.class));
  }
}

Even if the above code was valid, it still wouldn't work, because of the use case where a Vavr List contains other Vavr structures:

import io.vavr.collection.List;
import io.vavr.collection.Set;

@Data
@AllArgsConstructor
@NoArgsConstructor
@Wither
private static class Person {

  private int age;
  private List<String> names;
  private List<Set<Integer>> favoriteNumberSets;
}

By the way, the code I use to test my registry, is it correct?

EnhancedRandom er = aNewEnhancedRandomBuilder()
    .registerRandomizerRegistry(new VavrRandomizerRegistry()).build();
Person random = er.nextObject(Person.class);

As a reminder, Vavr collections don't extend java.util.Collection, so I can't extend CollectionRandomizer<T>.

For what it's worth, all Vavr collections extend java.lang.Iterable, so we could imagine Random Beans evolves to provide an IterableRandomizer<T>.
AFAICT collections have special support in Random Beans, they don't rely solely on registry (see CollectionPopulator).

[Sir4ur0n]

Has anyone any idea on how to progress on this topic please? I'm really willing to help/contribute/provide a new library, but I need a pointer in the right direction (I could just fork this project but I think nobody would win in this scenario 😢 ).

[benas]

Writing T.class won't work indeed. You need to either use a delegate randomizer (as with the CollectionRandomizer), something like:

public class VavrListRandomizer<T> implements Randomizer<List<T>> {

    private Randomizer<T> delegate;
    private int numberOfElements;

    public VavrListRandomizer(Randomizer<T> delegate, int numberOfElements) {
        this.delegate = delegate;
        this.numberOfElements = numberOfElements;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        for (int i = 0; i < numberOfElements; i++) {
            list = list.append(delegate.getRandomValue());
        }
        return list;
    }
}

or if you don't want to use a delegate, you can pass the type as parameter:

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;

    public VavrListRandomizer(Class<? extends T> type) {
        this.type = type;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt();
        for (int i = 0; i < numberOfElements; i++) {
            list = list.append(EnhancedRandom.random(type));
        }
        return list;
    }
}

but in both cases, it will not be possible to use a registry as the type is erased at runtime. So probably my mistake to suggest a registry in the first place.

However, you can still use one of the above randomizers and register it for a field of type a vavr collection. For example:

class Person {
    List<String> nicknames; // io.vavr.collection.List
}

EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .randomize(FieldDefinitionBuilder.field().named("nicknames").ofType(List.class).inClass(Person.class).get(), new VavrListRandomizer<String>(String.class))
                .build();

Person person = enhancedRandom.nextObject(Person.class); // this correctly generates a person with list of String nicknames

Does this help?

[Sir4ur0n]

I guess this would work (haven't tried it) but I want a generic solution, because we have tens or hundreds of Java models (beans) which almost all use Vavr structures, so I don't want to write hundreds/thousands of lines with hardcoded field names (which is refactor-unfriendly).

I think random-beans could provide a generic way to plug on io.github.benas.randombeans.FieldPopulator#generateRandomValue (I know, easier said than done 😄 ): I managed to do what I need by creating a custom VavrPopulator, and adding this piece of code:

} else if (isCollectionType(fieldType)) {
  value = collectionPopulator.getRandomCollection(field, context);
} else if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) {
  value = vavrPopulator.getRandomIterable(field, context);
}

inside this method.

However as said before, I'd rather not fork this project as much as possible, so I'm really hoping we can find a solution that fits everyone's needs 😄 @benas do you think this is feasible to add some plugging in the populator? Maybe something similar to the registry mechanism.
Example:
Each registry can provide a method that takes precedence in this method, and returns a Optional<Object> (or if you need further backward-compatibility, either return a non-null Object or null). If it's null then run the rest of your existing generateRandomValue method, otherwise return.

Dumb example:

private Object generateRandomValue(final Field field, final RandomizationContext context) {
  // customStuff would be similar to how registries work: anybody can register such a custom "thing", by accessing the Field and the RandomizationContext
  Object value = customStuff.generateRandomValue(field, context);
  if (value != null) {
    return value;
  }
  // Rest of the existing code
}

Let me know if it's impossible or too much hassle (in which case I'll fallback to forking 😞 ).

Thank you for your help!

[benas]

Thanks for these additional details. I thought you had a couple of fields having vavr types, but if you tell me you have hundreds of them, then I fully understand why you need a generic solution. With that said, I can go back to using a registry 😄 This time by getting the generic field type at runtime and using the above VavrListRandomizer (the one without delegate). Here is how I was able to do it:

1. First the randomizer for a given vavr type (List here)

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.randomizers.AbstractRandomizer;
import io.vavr.collection.List;

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;

    public VavrListRandomizer(Class<? extends T> type) {
        this.type = type;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt();
        for (int i = 0; i < numberOfElements; i++) {
            T random = EnhancedRandom.random(type);
            list = list.append(random);
        }
        return list;
    }
}

2. Then a registry that grabs the field type at runtime and returns the corresponding vavr randomizer configured with that type (similar code to CollectionPopulator)

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.annotation.Priority;
import io.github.benas.randombeans.api.EnhancedRandomParameters;
import io.github.benas.randombeans.api.Randomizer;
import io.github.benas.randombeans.api.RandomizerRegistry;

import java.lang.reflect.Field;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;

import static io.github.benas.randombeans.util.ReflectionUtils.isParameterizedType;

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    @Override
    public void init(EnhancedRandomParameters parameters) {
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        Class<?> fieldType = field.getType();
        if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) { // refactor this as needed and add support for other types
            Type fieldGenericType = field.getGenericType();
            if (isParameterizedType(fieldGenericType)) { // populate only parametrized types, raw types will be empty
                ParameterizedType parameterizedType = (ParameterizedType) fieldGenericType;
                Type type = parameterizedType.getActualTypeArguments()[0]; // todo sanity check. I'm also not sure if we need to go deep for things like List<Set<Integer>>
                return new VavrListRandomizer((Class<?>) type);
            }
        }
        return null;
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return null;
    }
}

3. And finally the example

class Person {
    List<String> nicknames;
    List<Integer> numbers;

    @Override
    public String toString() {
        return "Person{" +
                "nicknames=" + nicknames.asJava() +
                ",numbers=" + numbers.asJava() +
                '}';
    }

    public static void main(String[] args) {
        EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .registerRandomizerRegistry(new VavrRandomizerRegistry())
                .build();

        Person person = enhancedRandom.nextObject(Person.class);
        System.out.println("person = " + person);
    }
}

The sample correctly generates a person instance. So the registry works well with generic vavr types without having to register a randomizer for each field.

@Sir4ur0n What do you think?

[Sir4ur0n]

I'm not on my computer to try, but I think this won't work if you have a Person having a Vavr list of addresses and each address has a Vavr list of tags (strings), because the random used to build each address is a new one, not the one with the vavr randomizer registered, so it should fail when building the vavr lost of addresses. Does that make sense?

[benas]

Sure! But that's not an issue as you can pass the enhancedRandom to the randomizer via the registry. Here is a sample with addresses:

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.randomizers.AbstractRandomizer;
import io.vavr.collection.List;

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;
    private EnhancedRandom enhancedRandom;

    public VavrListRandomizer(Class<? extends T> type, EnhancedRandom enhancedRandom) {
        this.type = type;
        this.enhancedRandom = enhancedRandom;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt(10); // todo could be set to a value in EnhancedRandomParameters#getCollectionSizeRange taken from parameters in io.github.benas.randombeans.vavr.VavrRandomizerRegistry#init and propagated here
        for (int i = 0; i < numberOfElements; i++) {
            T random = enhancedRandom.nextObject(type);
            list = list.append(random);
        }
        return list;
    }
}

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.annotation.Priority;
import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.api.EnhancedRandomParameters;
import io.github.benas.randombeans.api.Randomizer;
import io.github.benas.randombeans.api.RandomizerRegistry;

import java.lang.reflect.Field;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;

import static io.github.benas.randombeans.util.ReflectionUtils.isParameterizedType;

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    private EnhancedRandom enhancedRandom;

    @Override
    public void init(EnhancedRandomParameters parameters) {
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        Class<?> fieldType = field.getType();
        if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) { // refactor this as needed and add support for other types
            Type fieldGenericType = field.getGenericType();
            if (isParameterizedType(fieldGenericType)) { // populate only parametrized types, raw types will be empty
                ParameterizedType parameterizedType = (ParameterizedType) fieldGenericType;
                Type type = parameterizedType.getActualTypeArguments()[0];  // todo sanity check. I'm also not sure if we need to go deep for things like List<Set<Integer>>
                return new VavrListRandomizer((Class<?>) type, enhancedRandom);
            }
        }
        return null;
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return null;
    }

    public void setEnhancedRandom(EnhancedRandom enhancedRandom) {
        this.enhancedRandom = enhancedRandom;
    }
}

and the sample:

class Person {
    List<String> nicknames;
    List<Integer> numbers;
    List<Address> addresses;

    @Override
    public String toString() {
        return "Person{" +
                "nicknames=" + nicknames.asJava() +
                ",numbers=" + numbers.asJava() +
                ",addresses=" + addresses.asJava() +
                '}';
    }

    public static void main(String[] args) {
        VavrRandomizerRegistry vavrRandomizerRegistry = new VavrRandomizerRegistry();
        EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .registerRandomizerRegistry(vavrRandomizerRegistry)
                .build();

        vavrRandomizerRegistry.setEnhancedRandom(enhancedRandom);

        Person person = enhancedRandom.nextObject(Person.class);
        System.out.println("person = " + person);
    }
}

class Address {
    List<String> tags;

    @Override
    public String toString() {
        return "Address{" +
                "tags=" + tags.asJava() +
                '}';
    }
}

generates for example:

person = Person{
   nicknames=[IbuAxb, oswsoUNEtOTaJRCuqmqjHVEigKR, eexxihCZLLBQvhCkkGEBAYKsjcfa, fC],
   numbers=[708490968, -494238257],
   addresses=[
      Address{tags=[HsEC]},
      Address{tags=[vHVRIlx, rFmHroKyFMAtcQskXz, JbDHprEovIWrgHaZkRMDnIMn, ngSRnPYnGHC, LMNVgewhQunqYdcSRguIsOalZUy, fqYHsXUJpyzZcdGnaTFqtEtlFc, OfZgFjlKPCxvzDlOaf, vnGFBhoIABanwFNKfaQJsLcawGfT]},
      Address{tags=[bgktNJkDRCnFMpOiTqqkD, jjcxxmAdKBu, kdoUblxoVnFTUCmNvUWpgXCgc, ZpflKkKZRSjwW, IAMfyHNtqU, ZmHwyZjdaPtVkpJtAHynNMDoEbnQP, GCrnREyiXYndOXXF]}, 
      Address{tags=[]},
      Address{tags=[Vl, mfTZseukew, anXIlqjrwZrU, LmTOGa, cxMaxc]},
      Address{tags=[OWleJQEwMaMTRZCTbzu, nnRsOHOl, DXEulnJctCnRkakaickq, NnZgklVuzqIycaNlMqGmoHs, tkzGaRkzWPDnFCGCxzHzunJL, LfEGCeBNQIsHUZzyOHYBs]},
      Address{tags=[uejtEVPGdEPCVdkGRUASskHnibcyU, RYaHlhOAUEWd, LlAZvpq]},
      Address{tags=[]}
   ]
}

[Sir4ur0n]

Thank you, this seems to work fine! There were some missing parts in your example but I could easily figure them out.

I noticed Random-Beans doesn't support (yet? 😄 ) collections of collections (e.g. a List<Set<String>> or a List<List<String>>) but we almost never have this design in our models (and I actually think I can add support for it in my custom registry).

I will do this in our project code and if this works fine for some time, I propose to provide a tiny library for it. I'll let you know in case you want to link to it from your project.

Thanks again @benas I really appreciate the help!
Cheers

[benas]

Thank you, this seems to work fine!

Great! Just saw that I pasted the wrong main method in the previous sample, I updated the snippet with the version where we need to set the enhancedRandom on the registry afterwards to be able to pass it to the randomizer.

There were some missing parts in your example but I could easily figure them out.

Indeed, that was a quick PoC. Feel free to polish it as needed.

I noticed Random-Beans doesn't support (yet? 😄 ) collections of collections

That was deliberate as it will make things unnecessary complex for some rare use cases. I will add a note about it in the known limitation section of the docs.

I will do this in our project code and if this works fine for some time, I propose to provide a tiny library for it. I'll let you know in case you want to link to it from your project.

Sure! I will be more than happy to add a reference to it in the Extensions section.

Thanks again @benas I really appreciate the help!

Glad to be of help. Thank YOU for this interesting thread!

Best regards
Mahmoud

[benas]

I will add a note about it in the known limitation section of the docs.

done here.

[Sir4ur0n]

We've been using this library with our own randomizer for Vavr structures and so far, this seems to work like a charm, thank you.

All I need now is to find time and motivation to open source our randomizer in case anyone is interested. I'll let you know here when it's done!