Remove duplicates from yaml values

[TamasSzigeti]

Fixes #1064

[what-the-diff]

PR Summary

This Pull Request involves a series of updates focused on cleaning up data in our codebase. It has systematically removed specific elements from various lists throughout the code. Essentially, this action is akin to deleting unused or unnecessary items from a spreadsheet to help streamline and improve efficiency.

Here's a breakdown of the key changes:

• Cleanup Operation
  This update has removed several items from different lists across multiple lines of code. This process should help make our data cleaner, and potentially reduce clutter and improve the performance of our systems.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (73ae126) 92.29% compared to head (affa80a) 92.42%.
Report is 5 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1066      +/-   ##
============================================
+ Coverage     92.29%   92.42%   +0.12%     
- Complexity     2817     2822       +5     
============================================
  Files           292      292              
  Lines          5607     5608       +1     
  Branches        599      599              
============================================
+ Hits           5175     5183       +8     
+ Misses          276      272       -4     
+ Partials        156      153       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kingthorin]

@TamasSzigeti would you be up for adding tests to this or is that asking too much? (Not a trick, we can do it separately if that's outside your interest/skillset.)

[TamasSzigeti]

Sure, gladly, will make time for it in the coming days.

[kingthorin]

Even just something like:

Get the List from the yaml: List<String> loremWords = getBaseList("lorem.words");
Create a Set from the List (since Sets in Java are unique values): Set<String> uniqueWords = new HashSet<>(loremWords);

Then assert that the two sizes should be equal, then we know there's no duplicates.
Ummm actually you could even do a parameterized test that passes "lorem.words" and "lorem.supplemental" as key String, use "Terms" instead of "Words" for the variable names, then it's just a single test method like wordListsShouldNotHaveDuplicates or something like that.

[kingthorin]

Your method content will be different but here's an example how to annotate and set things up:

datafaker/src/test/java/net/datafaker/providers/base/DateAndTimeTest.java

Lines 194 to 199 in 8166a0f

	@ParameterizedTest
	@ValueSource(strings = {"null", "", "month", "year", "week"})
	void invalidDuration(String invalid) {
	assertThatThrownBy(() -> faker.date().duration(faker.random().nextLong(), invalid))
	.isInstanceOf(IllegalArgumentException.class);
	}

[TamasSzigeti]

@kingthorin I did a bit more than that, have a look pls

[kingthorin]

LGTM

[kingthorin]

Thanks!

[kingthorin]

@snuyanzin @bodiam one of you wanna make a second call on this. I'm not 100% sure all lists should be unique. (I don't see a reason why not, but don't wanna kill something being unaware.)

[snuyanzin]

thanks for the contribution
i think there are a couple of thing need to be done

1. apply spotless to remove unused imports
2. there are several tests failing after this change, they should be adopted

[kingthorin]

Thanks!

[TamasSzigeti]

Thank you too! Could you tag and publish a release, please?

[bodiam]

@TamasSzigeti a snapshot release should have been published automatically, I can make a new official release soon if you need.

[kingthorin]

https://s01.oss.sonatype.org/content/repositories/snapshots/net/datafaker/datafaker/

[TamasSzigeti]

Thanks, I can use the snapshot for now, no problem