Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Vavr randomizers #321

Closed
Sir4ur0n opened this Issue Nov 29, 2018 · 13 comments

Comments

2 participants
@Sir4ur0n
Copy link

commented Nov 29, 2018

I recently discovered your library and it's quite useful, thank you!

That being said, we massively use Vavr (https://github.com/vavr-io/vavr) collections instead of JDK ones, because of their immutability and more functional features/APIs.
Of course Random Beans don't work OOTB with Vavr collections, we get this kind of exceptions:

io.github.benas.randombeans.api.ObjectGenerationException: Unable to generate a random instance of type class Foo
Caused by: io.github.benas.randombeans.api.ObjectGenerationException: Unable to create type: io.vavr.collection.List for field: strings of class: Foo
Caused by: io.github.benas.randombeans.api.ObjectGenerationException: Unable to create an instance of type: interface io.vavr.collection.List
Caused by: java.lang.InstantiationError: io.vavr.collection.List

I took a quick look at your code and documentation (in particular https://github.com/benas/random-beans/wiki/Supported-types) and came to the conclusion that it would be possible, at least by manually writing Randomizer classes in the same spirit as SetRandomizer, ListRandomizer, etc.

Would that make sense to add support for it?
Please note that most common Vavr collections (like List, Set, Map) provide factory methods to build from JDK List/Set/Map (e.g. io.vavr.collection.List.ofAll(java.lang.Iterable)).

I'm ok with helping (though I would let you the fun part of creating a new module etc. if you wish/accept to expose it as another Random Beans module) 😄.

Cheers

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Nov 30, 2018

I cloned this repo and fiddled with the code to support at least a Vavr List type. I managed to make it work, but actually I had to touch inner code, I couldn't simply add a custom registry as I hoped to at first.
I modified io.github.benas.randombeans.FieldPopulator#generateRandomValue, created a new VavrPopulator (a copy of CollectionPopulator which builds a Vavr List type) and modified a few classes consequently.

Is this the right approach? If yes, how would you suggest we move on? Would you at least consider to be able to plug on the collection detection type? So that we can provide some kind of registry to handle Vavr types (which technically are not Collections, the only common ancestor they share with Collection is Iterable).

I appreciate any advice.

Cheers

@benas

This comment has been minimized.

Copy link
Collaborator

commented Nov 30, 2018

Hi @Sir4ur0n

Thank you for this suggestion! I remember having the same suggestion for guava collections but I decided to not add support for it as it was moving very fast (I see they are at v27 now, at that time it was 14 IICR) and I was pretty sure I could not catch up.. Some people also wanted support for eclipse collections and now it's vavr's turn. It would be really awesome to add support for these third party libraries, but this also means a maintenance burden I simply could not support. I hope you understand this situation.

If you want to add support for vavr, please go ahead and do it in a repo of your own, I would be more than happy to help and add a reference to it on the home page of RB!

Now in regards to how I would implement this, the way to go IMO is to use a custom registry for vavr types.

I couldn't simply add a custom registry as I hoped to at first

What was the issue? Something like the following should be enough:

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    private final Map<Class<?>, Randomizer<?>> randomizers = new HashMap<>();

    @Override
    public void init(EnhancedRandomParameters parameters) {
        randomizers.put(io.vavr.collection.List.class, new ListRandomizer<>()); // todo implement a ListRandomizer for vavr's List type
        // register other randomizers for vavr types
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        return getRandomizer(field.getType());
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return randomizers.get(type);
    }
}

Kr,
Mahmoud

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Dec 4, 2018

Hi @benas

Thanks for your answer.
I perfectly understand you don't want to support other libraries like Vavr or Guava yourself (the maintenance burden would be high).

That being said, I don't see how to implement your solution with a registry. I guess in your example, VavrListRandomizer should implement Randomizer<io.vavr.collection.List<T>>, correct?
I don't see how this is possible, since I don't have any mean of creating a T.

public class VavrListRandomizer<T> implements Randomizer<List<T>> {

  @Override
  public List<T> getRandomValue() {
    // Impossible, generic type classes can't be captured
    return List(EnhancedRandom.random(T.class));
  }
}

Even if the above code was valid, it still wouldn't work, because of the use case where a Vavr List contains other Vavr structures:

import io.vavr.collection.List;
import io.vavr.collection.Set;

@Data
@AllArgsConstructor
@NoArgsConstructor
@Wither
private static class Person {

  private int age;
  private List<String> names;
  private List<Set<Integer>> favoriteNumberSets;
}

By the way, the code I use to test my registry, is it correct?

EnhancedRandom er = aNewEnhancedRandomBuilder()
    .registerRandomizerRegistry(new VavrRandomizerRegistry()).build();
Person random = er.nextObject(Person.class);

As a reminder, Vavr collections don't extend java.util.Collection, so I can't extend CollectionRandomizer<T>.

For what it's worth, all Vavr collections extend java.lang.Iterable, so we could imagine Random Beans evolves to provide an IterableRandomizer<T>.
AFAICT collections have special support in Random Beans, they don't rely solely on registry (see CollectionPopulator).

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Dec 11, 2018

Has anyone any idea on how to progress on this topic please? I'm really willing to help/contribute/provide a new library, but I need a pointer in the right direction (I could just fork this project but I think nobody would win in this scenario 😢 ).

@benas

This comment has been minimized.

Copy link
Collaborator

commented Dec 12, 2018

Writing T.class won't work indeed. You need to either use a delegate randomizer (as with the CollectionRandomizer), something like:

public class VavrListRandomizer<T> implements Randomizer<List<T>> {

    private Randomizer<T> delegate;
    private int numberOfElements;

    public VavrListRandomizer(Randomizer<T> delegate, int numberOfElements) {
        this.delegate = delegate;
        this.numberOfElements = numberOfElements;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        for (int i = 0; i < numberOfElements; i++) {
            list = list.append(delegate.getRandomValue());
        }
        return list;
    }
}

or if you don't want to use a delegate, you can pass the type as parameter:

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;

    public VavrListRandomizer(Class<? extends T> type) {
        this.type = type;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt();
        for (int i = 0; i < numberOfElements; i++) {
            list = list.append(EnhancedRandom.random(type));
        }
        return list;
    }
}

but in both cases, it will not be possible to use a registry as the type is erased at runtime. So probably my mistake to suggest a registry in the first place.

However, you can still use one of the above randomizers and register it for a field of type a vavr collection. For example:

class Person {
    List<String> nicknames; // io.vavr.collection.List
}
EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .randomize(FieldDefinitionBuilder.field().named("nicknames").ofType(List.class).inClass(Person.class).get(), new VavrListRandomizer<String>(String.class))
                .build();

Person person = enhancedRandom.nextObject(Person.class); // this correctly generates a person with list of String nicknames

Does this help?

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

I guess this would work (haven't tried it) but I want a generic solution, because we have tens or hundreds of Java models (beans) which almost all use Vavr structures, so I don't want to write hundreds/thousands of lines with hardcoded field names (which is refactor-unfriendly).

I think random-beans could provide a generic way to plug on io.github.benas.randombeans.FieldPopulator#generateRandomValue (I know, easier said than done 😄 ): I managed to do what I need by creating a custom VavrPopulator, and adding this piece of code:

} else if (isCollectionType(fieldType)) {
  value = collectionPopulator.getRandomCollection(field, context);
} else if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) {
  value = vavrPopulator.getRandomIterable(field, context);
}

inside this method.

However as said before, I'd rather not fork this project as much as possible, so I'm really hoping we can find a solution that fits everyone's needs 😄 @benas do you think this is feasible to add some plugging in the populator? Maybe something similar to the registry mechanism.
Example:
Each registry can provide a method that takes precedence in this method, and returns a Optional<Object> (or if you need further backward-compatibility, either return a non-null Object or null). If it's null then run the rest of your existing generateRandomValue method, otherwise return.

Dumb example:

private Object generateRandomValue(final Field field, final RandomizationContext context) {
  // customStuff would be similar to how registries work: anybody can register such a custom "thing", by accessing the Field and the RandomizationContext
  Object value = customStuff.generateRandomValue(field, context);
  if (value != null) {
    return value;
  }
  // Rest of the existing code
}

Let me know if it's impossible or too much hassle (in which case I'll fallback to forking 😞 ).

Thank you for your help!

@benas

This comment has been minimized.

Copy link
Collaborator

commented Dec 12, 2018

Thanks for these additional details. I thought you had a couple of fields having vavr types, but if you tell me you have hundreds of them, then I fully understand why you need a generic solution. With that said, I can go back to using a registry 😄 This time by getting the generic field type at runtime and using the above VavrListRandomizer (the one without delegate). Here is how I was able to do it:

1. First the randomizer for a given vavr type (List here)

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.randomizers.AbstractRandomizer;
import io.vavr.collection.List;

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;

    public VavrListRandomizer(Class<? extends T> type) {
        this.type = type;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt();
        for (int i = 0; i < numberOfElements; i++) {
            T random = EnhancedRandom.random(type);
            list = list.append(random);
        }
        return list;
    }
}

2. Then a registry that grabs the field type at runtime and returns the corresponding vavr randomizer configured with that type (similar code to CollectionPopulator)

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.annotation.Priority;
import io.github.benas.randombeans.api.EnhancedRandomParameters;
import io.github.benas.randombeans.api.Randomizer;
import io.github.benas.randombeans.api.RandomizerRegistry;

import java.lang.reflect.Field;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;

import static io.github.benas.randombeans.util.ReflectionUtils.isParameterizedType;

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    @Override
    public void init(EnhancedRandomParameters parameters) {
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        Class<?> fieldType = field.getType();
        if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) { // refactor this as needed and add support for other types
            Type fieldGenericType = field.getGenericType();
            if (isParameterizedType(fieldGenericType)) { // populate only parametrized types, raw types will be empty
                ParameterizedType parameterizedType = (ParameterizedType) fieldGenericType;
                Type type = parameterizedType.getActualTypeArguments()[0]; // todo sanity check. I'm also not sure if we need to go deep for things like List<Set<Integer>>
                return new VavrListRandomizer((Class<?>) type);
            }
        }
        return null;
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return null;
    }
}

3. And finally the example

class Person {
    List<String> nicknames;
    List<Integer> numbers;

    @Override
    public String toString() {
        return "Person{" +
                "nicknames=" + nicknames.asJava() +
                ",numbers=" + numbers.asJava() +
                '}';
    }

    public static void main(String[] args) {
        EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .registerRandomizerRegistry(new VavrRandomizerRegistry())
                .build();

        Person person = enhancedRandom.nextObject(Person.class);
        System.out.println("person = " + person);
    }
}

The sample correctly generates a person instance. So the registry works well with generic vavr types without having to register a randomizer for each field.

@Sir4ur0n What do you think?

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Dec 12, 2018

@benas

This comment has been minimized.

Copy link
Collaborator

commented Dec 12, 2018

Sure! But that's not an issue as you can pass the enhancedRandom to the randomizer via the registry. Here is a sample with addresses:

package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.randomizers.AbstractRandomizer;
import io.vavr.collection.List;

public class VavrListRandomizer<T> extends AbstractRandomizer<List<T>> {

    private Class<? extends T> type;
    private EnhancedRandom enhancedRandom;

    public VavrListRandomizer(Class<? extends T> type, EnhancedRandom enhancedRandom) {
        this.type = type;
        this.enhancedRandom = enhancedRandom;
    }

    @Override
    public List<T> getRandomValue() {
        List<T> list = List.empty();
        int numberOfElements = random.nextInt(10); // todo could be set to a value in EnhancedRandomParameters#getCollectionSizeRange taken from parameters in io.github.benas.randombeans.vavr.VavrRandomizerRegistry#init and propagated here
        for (int i = 0; i < numberOfElements; i++) {
            T random = enhancedRandom.nextObject(type);
            list = list.append(random);
        }
        return list;
    }
}
package io.github.benas.randombeans.vavr;

import io.github.benas.randombeans.annotation.Priority;
import io.github.benas.randombeans.api.EnhancedRandom;
import io.github.benas.randombeans.api.EnhancedRandomParameters;
import io.github.benas.randombeans.api.Randomizer;
import io.github.benas.randombeans.api.RandomizerRegistry;

import java.lang.reflect.Field;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;

import static io.github.benas.randombeans.util.ReflectionUtils.isParameterizedType;

@Priority(-3) // more details on this here: https://github.com/benas/random-beans/wiki/Grouping-Randomizers
public class VavrRandomizerRegistry implements RandomizerRegistry {

    private EnhancedRandom enhancedRandom;

    @Override
    public void init(EnhancedRandomParameters parameters) {
    }

    @Override
    public Randomizer<?> getRandomizer(final Field field) {
        Class<?> fieldType = field.getType();
        if (io.vavr.collection.List.class.isAssignableFrom(fieldType)) { // refactor this as needed and add support for other types
            Type fieldGenericType = field.getGenericType();
            if (isParameterizedType(fieldGenericType)) { // populate only parametrized types, raw types will be empty
                ParameterizedType parameterizedType = (ParameterizedType) fieldGenericType;
                Type type = parameterizedType.getActualTypeArguments()[0];  // todo sanity check. I'm also not sure if we need to go deep for things like List<Set<Integer>>
                return new VavrListRandomizer((Class<?>) type, enhancedRandom);
            }
        }
        return null;
    }

    @Override
    public Randomizer<?> getRandomizer(Class<?> type) {
        return null;
    }

    public void setEnhancedRandom(EnhancedRandom enhancedRandom) {
        this.enhancedRandom = enhancedRandom;
    }
}

and the sample:

class Person {
    List<String> nicknames;
    List<Integer> numbers;
    List<Address> addresses;

    @Override
    public String toString() {
        return "Person{" +
                "nicknames=" + nicknames.asJava() +
                ",numbers=" + numbers.asJava() +
                ",addresses=" + addresses.asJava() +
                '}';
    }

    public static void main(String[] args) {
        VavrRandomizerRegistry vavrRandomizerRegistry = new VavrRandomizerRegistry();
        EnhancedRandom enhancedRandom = new EnhancedRandomBuilder()
                .registerRandomizerRegistry(vavrRandomizerRegistry)
                .build();

        vavrRandomizerRegistry.setEnhancedRandom(enhancedRandom);

        Person person = enhancedRandom.nextObject(Person.class);
        System.out.println("person = " + person);
    }
}

class Address {
    List<String> tags;

    @Override
    public String toString() {
        return "Address{" +
                "tags=" + tags.asJava() +
                '}';
    }
}

generates for example:

person = Person{
   nicknames=[IbuAxb, oswsoUNEtOTaJRCuqmqjHVEigKR, eexxihCZLLBQvhCkkGEBAYKsjcfa, fC],
   numbers=[708490968, -494238257],
   addresses=[
      Address{tags=[HsEC]},
      Address{tags=[vHVRIlx, rFmHroKyFMAtcQskXz, JbDHprEovIWrgHaZkRMDnIMn, ngSRnPYnGHC, LMNVgewhQunqYdcSRguIsOalZUy, fqYHsXUJpyzZcdGnaTFqtEtlFc, OfZgFjlKPCxvzDlOaf, vnGFBhoIABanwFNKfaQJsLcawGfT]},
      Address{tags=[bgktNJkDRCnFMpOiTqqkD, jjcxxmAdKBu, kdoUblxoVnFTUCmNvUWpgXCgc, ZpflKkKZRSjwW, IAMfyHNtqU, ZmHwyZjdaPtVkpJtAHynNMDoEbnQP, GCrnREyiXYndOXXF]}, 
      Address{tags=[]},
      Address{tags=[Vl, mfTZseukew, anXIlqjrwZrU, LmTOGa, cxMaxc]},
      Address{tags=[OWleJQEwMaMTRZCTbzu, nnRsOHOl, DXEulnJctCnRkakaickq, NnZgklVuzqIycaNlMqGmoHs, tkzGaRkzWPDnFCGCxzHzunJL, LfEGCeBNQIsHUZzyOHYBs]},
      Address{tags=[uejtEVPGdEPCVdkGRUASskHnibcyU, RYaHlhOAUEWd, LlAZvpq]},
      Address{tags=[]}
   ]
}

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Dec 13, 2018

Thank you, this seems to work fine! There were some missing parts in your example but I could easily figure them out.

I noticed Random-Beans doesn't support (yet? 😄 ) collections of collections (e.g. a List<Set<String>> or a List<List<String>>) but we almost never have this design in our models (and I actually think I can add support for it in my custom registry).

I will do this in our project code and if this works fine for some time, I propose to provide a tiny library for it. I'll let you know in case you want to link to it from your project.

Thanks again @benas I really appreciate the help!
Cheers

@Sir4ur0n Sir4ur0n closed this Dec 13, 2018

@benas

This comment has been minimized.

Copy link
Collaborator

commented Dec 13, 2018

Thank you, this seems to work fine!

Great! Just saw that I pasted the wrong main method in the previous sample, I updated the snippet with the version where we need to set the enhancedRandom on the registry afterwards to be able to pass it to the randomizer.

There were some missing parts in your example but I could easily figure them out.

Indeed, that was a quick PoC. Feel free to polish it as needed.

I noticed Random-Beans doesn't support (yet? 😄 ) collections of collections

That was deliberate as it will make things unnecessary complex for some rare use cases. I will add a note about it in the known limitation section of the docs.

I will do this in our project code and if this works fine for some time, I propose to provide a tiny library for it. I'll let you know in case you want to link to it from your project.

Sure! I will be more than happy to add a reference to it in the Extensions section.

Thanks again @benas I really appreciate the help!

Glad to be of help. Thank YOU for this interesting thread!

Best regards
Mahmoud

@benas

This comment has been minimized.

Copy link
Collaborator

commented Jan 25, 2019

I will add a note about it in the known limitation section of the docs.

done here.

@Sir4ur0n

This comment has been minimized.

Copy link
Author

commented Jan 26, 2019

We've been using this library with our own randomizer for Vavr structures and so far, this seems to work like a charm, thank you.

All I need now is to find time and motivation to open source our randomizer in case anyone is interested. I'll let you know here when it's done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.