Limits for scenario creation and duplicates

[NXXR]

@Kanakanajm brought up the possibility for users to (inadvertently) create a lot of new scenarios to flood the airflow pipeline with work.

Is this something we need to limit somehow? do we need to do this within the project or is this something for after?

@qacwnfq your input might be interesting here since I have no idea how airflow would handle that or what good rates would be 🤔

my assumption is that nothing really would happen to the infrastructure, it would just pick the scenarios and simulate them, useful or not, but it wouldn't crash.

Ideally, scenario creation might not be rate limited but limited by a total I guess so each user/lha is limited to X amount of scenarios, this would also keep our DB healthy 😬

Also @Kanakanajm mentioned that we could check for duplicates on scenario creation.
This would increase validation time when creating a new scenario but save us some database space.
Additionally, found duplicates could be automatically shared between LHAs as long as the input data only contains public data
@annawendler @HenrZu how is input data currently specified in the simulations? in the scenario parameters?
if an LHA uses their own data for a scenario is this reflected in the current scenario data items? or do we need to add some provenance to the scenario info to track from which data it originates?

[annawendler]

So far it isn't reflected in the current scenario data items for the DB which type of data has been used and would need to be added.