Species to be visualized and could be selected for early warning system

[damianooldoni]

As with @timadriaens ever discussed, we should not only visualize the key species for RIPARIAS, but also the species of EU concern. You can find the list here: https://github.com/trias-project/indicators/blob/master/data/input/eu_concern_species.tsv

It seems this list could be updated soon (~December) with 33 new species. We are going to update the file above as soon as this happens.

[damianooldoni]

@niconoe: one species which should be added to the list and it is not present in the RIPARIAS key species and EU concern lists is Reynoutria japonica (Japanese knotweed).

[timadriaens]

@damianooldoni @niconoe I think we should certainly also add the alert list species identified through action A1 as early warning of first records for those is important, I list them here

Plants	Crayfish
Crassula helmsii	Procambarus acutus
Petasites japonicus	Faxonius virilis
Zizania latifolia	Faxonius rusticus
Egeria densa	Faxonius immunis
Aponogeton distachyos	Cherax destructor
Cotula coronopifolia	Procambarus virginalis
Houttuynia cordata	Creaserinus fodiens
Saururus cernuus	Faxonius juvenilis
Pontederia cordata	Faxonius neglectus
Gunnera tinctoria	Cherax quadricarinatus

[timadriaens]

btw, Gunnera tinctoria and Procambarus virginalis are also on the Union List.

[damianooldoni]

@timadriaens: right, I forgot indeed the list of species your team is preparing for action A.1.4! Thanks for reminding us.

So, the RIPARIAS key species and the alert list species should be for sure included. @timadriaens: would you include the speies of the Union lists and Reynoutria japonica as well? I would say yes, but I would like to hear your opinion about this.

[timadriaens]

Yes, why not, although the management of Reynoutria is currently mostly zero-management. However, if we include Reynoutria, we have to include all 3 species: japonica (2889173), x bohemica (4038485) and sachalinensis (2889088). Many people do not make the disctinction.

[niconoe]

@damianooldoni: I'm revisiting this, I think many different things were said by different people at a different times on this topic :-)

I was therefore wondering if you could make a quick tour of the stakeholders again (if you think it's needed), and provide me a table / CSV file with all the species that should be included in the first version of the Early Alert system ? This table should have:

• 1 column with the species name to be shown
• 1 column with the the GBIF taxon ID
• possibly: additional columns we should use to group species (see comment below)

the current system classify species on two axis:

• taxonomic group: plant / crayfish
• spread: Widespread and abundant species in the pilot river basins / Emerging species with a very restricted range in the pilot river basins / Species still absent from the pilot river basins

Could you also see if we should keep this classification (in that case, I'd need two more columns in the table) or if it should be adjusted. This is not visible in the user interface yet, but classifying them is useful to allow grouped selection in the interface (for example: "show me occurrences of all emergent crayfish species in Senne river basin" rather than manually selecting the species). What I'd like to know:

• should we classify species (or just keep a flat list)
• if so, are taxonomic group and spread the right categories or not

I think we that info, we could load more relevant species in the dev website and therefore make it more realistic before the end of the year.

Thanks a lot!

[niconoe]

Here is the current state of the table, so you can start from that and edit as needed:

id	name	gbif_taxon_key	group	category
1	Elodea nuttallii	5329212	PL	W
2	Heracleum mantegazzianum	3034825	PL	W
3	Hydrocotyle ranunculoides	7978544	PL	W
4	Ludwigia grandiflora	5421039	PL	W
5	Impatiens glandulifera	2891770	PL	W
6	Myriophyllum aquaticum	5361785	PL	W
7	Orconectes limosus	2227000	CR	w
8	Pacifastacus leniusculus	2226990	CR	W
9	Lagarosiphon major	2865565	PL	E
10	Lysichiton americanus	2869311	PL	E
11	Myriophyllum heterophyllum	5361762	PL	E
12	Procambarus clarkii	2227300	CR	E
13	Cabomba caroliniana	2882443	PL	SA
14	Heracleum persicum	8000520	PL	SA
15	Heracleum sosnowskyi	3642949	PL	SA
16	Ludwigia peploides	5420991	PL	SA
17	Orconectes virilis	2227064	CR	SA
18	Procambarus fallax	8879526	CR	SA

Legend:

PL=Plant
CR=Crayfish
W=widespread
E=emergent
SA=still absent

[damianooldoni]

Based on what has been discussed earlier in this issue, I propose the following.

Species

All species from:

• LIFE RIPARIAS key species
• RIPARIAS alert list species
• List of Invasive Alien Species of Union concern

Grouping

Grouping/categories:

• LIFE RIPARIAS key species
• LIFE RIPARIAS alert list species
• List of Invasive Alien Species of Union concern
• Plants
• Animals
• Crayfishes
• Vertebrates (chordata)

First three are project based. The last four are taxonomic based.

By adding crayfishes and vertebrates as two different grouping variables, we allow people to select all animals which are not crayfishes and viceversa.

@niconoe: where did you find info about the spread category? IMHO, the spread category can be not very clear for users within and outside the project.

Data structure

A flat table to make things easy! Each group/category is a variable. We love tidy data, isn't?

id	name	gbif_taxon_key	is_key_species	is_alert_list_species	is_union_list	is_plant	is_animal	is_crayfish	is_vertebrate
1	Elodea nuttallii	5329212	T	F	T	T	F	F	F
2	Heracleum mantegazzianum	3034825	T	F	T	T	F	F	F
3	Hydrocotyle ranunculoides	7978544	T	F	T	T	F	F	F
4	Ludwigia grandiflora	5421039	T	F	T	T	F	F	F
17	Orconectes virilis	2227064	T	F	T	F	T	T	F
18	Procambarus fallax	8879526	T	F	T	F	T	T	F
101	Egeria densa	5329260	F	T	F	T	F	F	F
102	Faxonius immunis	8930656	F	T	F	F	T	T	F
201	Ondatra zibethicus	5219858	F	F	T	F	T	F	T
...	..	...	...	..	..	..	..	..	..

@niconoe: I suppose you will put such a table in the repo somewhere and you will read it at every night before triggering a download? Notice that there are some logical constraints you could add while reading this table: in this way, if there are errrors, the previous version of the table is used or, easier, no download is triggered for that night.

Another possible change, @niconoe is to merge the columns is_plant and is_animal to a character column called kingdom. But it would mix up the type of the grouping variables: in my example they are all booleans.

Constraints:

• no empty values allowed in any column
• all grouping variables (is_ prefix) are booleans
• is_plant and is_animal are mutually exclusive ( is_plant != is_animal)
• is_crayfish and is_vertebrate are mutually exclusive (is_crayfish != is_vertebrate)

Notice these are all basic taxonomic constraints. Some project related constraints could be added too, but I think they could be changed during the project, so I would not add them. Example: is_key_species and is_alert_species are mutually exclusive: but maybe an alert list becomes important and is added as key species? Also, key species are all within the union list I think, but this could change even if it is unlikely, so I would avoid to add constraints on this as well.

@timadriaens, as invasive species expert and RIPARIAS member: do you give us the final GO on the species list and grouping?
@niconoe, as software development of this alert system: do you give us the final GO on the table structure and grouping?

[timadriaens]

after discussion with @damianooldoni and @bramdhondt we feel it is better for a first go to limit the scope to:

• riparias target species
• riparias alert list species

[niconoe]

Thanks a lot @damianooldoni. It's good that we have a species list already, waiting for its confirmation by @timadriaens. A few questions about it:

• is it only 12 species, or should the table be completed?
• there are missing IDS for the last three rows

I'll try here to already answer your inline questions:

@niconoe: where did you find info about the spread category? IMHO, the spread category can be not very clear for users within and outside the project.

I think @SoVDH give that data to me at the time. Anyway, we can remove this classification if it's more confusing than helpful to users.

@niconoe: I suppose you will put such a table in the repo somewhere and you will read it at every night before triggering a download? Notice that there are some logical constraints you could add while reading this table: in this way, if there are errors, the previous version of the table is used or, easier, no download is triggered for that night.

No, the web app has a database table for species that acts like the single source of truth for that. This table can be very easily edited by any Admin user (I can give you access so you have a look), constraints are enforced immediately, ... So unless there's some requirements I didn't get yet, I don't see what would be the benefit to make the app dependent of some tabular data in a GitHub repo. (I can elaborate on the why if you want, just tell me!)

Another possible change, @niconoe is to merge the columns is_plant and is_animal to a character column called kingdom. But it would mix up the type of the grouping variables: in my example they are all booleans.

Thanks, that's indeed the possibility. Don't worry too much about the details of the table structure (type of columns, ...). If I have the full species list (with GBIF taxon key) and some explanations in plain English about what should be possible in terms of categories/grouping, that's already great and I'll figure out a table structure that deals with all the constrains we have (we want the code to stay simple and flexible, there are Django conventions, performance considerations, ...).

Have a great weekend!

[damianooldoni]

Thanks @niconoe for your comment.

is it only 12 species, or should the table be completed?

Yes, that's just an example of the table format. I have just edited the table by adding dots as last row

there are missing IDS for the last three rows

No, just an error: I have just corrected it.

I will provide you the final table in the format above asap. I think on Monday.

Have a nice weekend you too.

[damianooldoni]

Adding on #6 (comment), during short meeting,@timadriaens, @bramdhondt and I decided to not add any category regarding emerging status at the moment to avoid any issue with organisations running their own established alert systems with such categories.

[timadriaens]

a quick check of that table for the crayfish because the taxonomy changed:

Orconectes limosus should be Faxonius limosus, so https://www.gbif.org/species/8909595
O. virilis should be Faxonius virilis, so https://www.gbif.org/species/8971201

[damianooldoni]

From mail of @bramdhondt:

the alert list species in #6 (comment) are quite outdated. The reference document for alert list is still in development. However, we can take these alert list species for early alert system version 1:

[damianooldoni]

I worked on the existing species list gsheet (https://docs.google.com/spreadsheets/d/15f17hXE9ZyUxEe2xA0seKs0CSibgYJdkaWcnmVoI1yY/edit#gid=921617273) and I made three sheets:

1. the existing one has been renamed to key_species where I applied the taxonomic changes mentioned by @timadriaens in Species to be visualized and could be selected for early warning system #6 (comment).
2. a new sheet called alert list species containing the list of species as mentioned by @bramdhondt in Species to be visualized and could be selected for early warning system #6 (comment)
3. an early alert species sheet containing the list of species to be used in the early alert system. @niconoe: this sheet is actually your reference sheet which should solve this issue.

Some notes:

• @timadriaens, @bramdhondt: the crayfish Faxonius virilis is present in both lists (project key species, alert list species). I wonder if this is a mistake or not.
• @timadriaens, @bramdhondt: could you please check that everything is right?
• at the moment the cols is_animal and is_crayfish are identical as we opted to look at the RIPARIAS species only. This could change in 2022 of course, so I left both columns.

[timadriaens]

Looking good, made a few tweeks:

• typo Egeria densa
• Creaserinus fodiens was removed from the alert list after updated risk assessment and replaced by Faxonius juvenilis, I updated that

[damianooldoni]

Thanks @timadriaens for checking this! Very much appreciated. Can you also confirm the presence of crayfish Faxonius virilis in both lists (project key species, alert list species) is correct? I assumed that key species were all quite established. But I am still missing something...

[timadriaens]

ah, yes, F. virilis is not a target species but an alert list species, adjusted in sheet 1 and 3

[damianooldoni]

Thanks @timadriaens. I opened the proposal to check the target species myself:

As you see @timadriaens: there are 6 species which are absent in pilot river basins: one of them is F. virilis (or Orconectes viriilis, synonym of the Faxonium one). So, it's possible to have target species in the alert list species, unless the target species list has been changed meanwhile?

[timadriaens]

if they are absent, they are alert list species, right, even if they are mentioned in the original project proposal. The reason they are there (and not considered anymore for the alert list exercise) is simply because they are Union List species. It's a bit of semantics. Does it really matter how they are coded for the system implementation?

[damianooldoni]

The importance for the early alert system is about the groups in the taxonomic selection dropdown menu: @niconoe will creates "groups" i nthe data filtering (and in the alert making page) so that users can easily select such groups if they are interested in them (selecting species by species is of course still possible).
The groups are:

1. all plants
2. all animals/crayfishses
3. all LIFE RIPARIAS target species
4. all LIFE RIPARIAS alert species

[timadriaens]

OK, I'd say they are alert list species as they do not occur in the riparias project area but when detected will be rapid responsed

[damianooldoni]

Thanks @timadriaens. So, we have 19 instead of 20 target species and we state that a species may not be present in both lists (target species and alert species lists). For me it's totally fine. Do you think should I contact Etienne about this as well to be sure nobody complaints about it while using the the early alert system version 1? Or if you want to contact him, you are welcome.

[timadriaens]

let's just move forward, this is a small detail. If the code needs to be adjusted later we can easily do it.

[damianooldoni]

Happy to hear this! 🥳
@niconoe: I think we have done! You can find the information you need in the tab "species for early alert" of this gsheet: https://docs.google.com/spreadsheets/d/15f17hXE9ZyUxEe2xA0seKs0CSibgYJdkaWcnmVoI1yY/edit#gid=921617273

I propose, @niconoe the following:

1. close this issue once you have imported the list in the db
2. every further development around the list of species for early warning system will be discussed in a new issue and will ALWAYS refer to changes in the gsheet above

[damianooldoni]

It seems that we were not the only ones struggling with the question of having a well defined list of species 😄 Xavier Vermeersch from BE had the same problem and this is a nice clear answer of Etienne:

agree with you we should stick as much as possible to what is described in the projet proposal and the grant agreement, which is a selection of EU-listed species and additional species from the alert list.
Amongst EU-listed species, it is indeed: (13 plants + 5 crayfish species) – (Faxonius limosus, Pacifastacus leniusculus and Elodea nuttalii) = 12 plants + 3 crayfish species
For alert-list species, it is: 10 additional plant species + 5 additional crayfish species (as described in the report that has just been finalised).
I hope this could be agreed by everybody and provide more clarity. If we work like this, this is perfectly in line with what was proposed (no modification needed to NEEMO).

My proposal is therefore to refer to such list. I will update our internal gsheet.

[damianooldoni]

I updated the species list in the spreadsheet based on the official list I got by RIPARIAS coordination (BE).

[niconoe]

@damianooldoni: do we agree the current list of species on the development website is good enough for the first release? In that case, I suggest closing this issue.