Making sure to provide as few NAs as possible for focal functional groups #115

agruss2 · 2017-05-25T22:11:42Z

@jhpoelen
This issue is somehow related to #114
For all of the following parameters - It is not reasonable to specify a default value for all the fish and invertebrates recorded in FishBase and SeaLifeBase:
(1) predation.accessibility.stage.threshold.sp*;
(2) predation.predPrey.stage.threshold.sp*;
(3) species.K.sp*;
(4) species.length2weight.allometric.power.sp*;
(5) species.length2weight.condition.factor.sp*;
(6) species.lifespan.sp*;
(7) species.lInf.sp*;
(8) species.maturity.size.sp*;
(9) species.maturity.age.sp*;
(10) species.relativefecundity.sp*;
(11) species.t0.sp*.

Therefore, the default values specified for the 11 abovementioned parameters in the “fishbase-mapping.csv” file should not be used by the API.

Here is what I would like you to implement when estimates are not available for all the species of a given focal functional group to derive a value for one of the 11 abovementioned parameters:
(1) Implementing a rule of thumb in the API such that the API uses information for the other species belonging to the genuses, families and orders to which the species of the focal functional group belong. (These other species can be found anywhere across the globe).
For the parameter of interest, the API would apply the “data richness” rules we discussed with Skit in #59 to produce an estimate for the focal functional group under consideration.
(2) If the API does not manage to obtain an estimate for the parameter of interest after having considered all the species belonging to the genuses, families and orders to which the species of the focal functional group belong -> Setting the value of the parameter of interest to “NA” (not available).

Sorry, this issue is a little bit hard to formulate, so please let me know if anything here is unclear.
Many thanks for everything.

jhpoelen · 2017-05-26T17:49:59Z

If I understand it correctly, you'd like to (a) extend the trait finding algorithm and (b) introduce a way to explicitly state when values are not available using "NA" (or some other "magic" value to identify a value that is unavailable). Item (b) can be introduced independent of (a). About (a) - currently, ordered speccodes (species ids) are passed from the wizard to the api. The api then retrieves relevant fishbase trait archives and attempts to retrieve desired traits. If I understand it correctly, you'd like to extend the ordered list (by decreasing richness) for a functional group to include all speccode that shares a genus, family or order with the species included in the functional group. This extended list of speccodes can then be used to retrieve a reasonable parameter value for a specific functional group. Let's say that a functional group contains 20 species. Each of these species have about 5 sibling species in genus, 20 in family, and 200 in the order. Assuming that these counts are reasonable, and no taxononic overlap between species exist, the list of species would turn to 20 * (5 + 20 + 200) ~ 5k speccodes per functional group (probably a conservative estimate). With 10 functional groups, this would be 50k speccodes to be scanned and examined in the worst case scenario. Before I continue to thinking about ways to implement this, please confirm that we are on the same page.

@agruss2

…uggested by @agruss2

jhpoelen · 2017-05-26T20:06:47Z

Note that currently, the api attempts to retrieve values associated to speccodes provided by UI in order. So, if the UI appends some more speccodes beyond the species explicitly mentioned in the functional group, the API should work as usual.

agruss2 · 2017-05-28T22:44:54Z

@jhpoelen
Here is what I think we could do.
Let’s say we are dealing with a functional group called “smallpelagics” and the parameter “relative fecundity”.
(1) For that “smallpelagics” functional group, the UI has identified, say, 20 species.
(2) Before passing information to the API, the UI appends to these 20 species: all the species that belong to the genuses, families and orders of the 20 species. Let’s say that this results in a list of 500 species in total (20 species + 480 appended species).
(3) When the API receives the list of 500 species from the UI, it first considers only the 20 species that belong to the “smallpelagics” functional group.
-> (i) Case 1: data are available to derive a value for the parameter “relative fecundity” for some of the 20 species that belong to the “smallpelagics” functional group. In that case, the “data richness” rule described in #59 is applied to derive a parameter estimate for the parameter “relative fecundity” for the “smallpelagics” functional group.
-> (ii) Case 2: data are not available to derive a value for the parameter “relative fecundity” for none of the 20 species that belong to the “smallpelagics” functional group. In that case, the API will consider the 480 appended species to see if it can derive a value for the parameter “relative fecundity”.
(4) If we have faced Case 2 above, then:
-> Case 1: data are available to derive a value for the parameter “relative fecundity” for some of the 480 appended species. In that case, the “data richness” rule described in #59 is applied to the list of 480 appended species to derive a parameter estimate for the parameter “relative fecundity” for the “smallpelagics” functional group.
-> (ii) Case 2: data are not available to derive a value for the parameter “relative fecundity” for none of the 480 appended species. In that case, the API will provide the value NA for the parameter “relative fecundity” for the “smallpelagics” functional group.

Does this sound good to you?
If so, could you please interact with @FIN-casey and @FIN-JBarile to implement what is described above?

jhpoelen · 2017-06-05T16:45:06Z

@agruss2 Sounds good. As far as I am concerned, no changes are needed in the API - the API gets an ordered list of species for the specific groups and will work down the list until a specific parameter has been found.

@FIN-JBarile @FIN-casey suggest to extend the list of species in a functional group to include the species as described above by @agruss2 . Please let me know if you have any thoughts on this.

agruss2 · 2017-06-05T17:57:48Z

@FIN-casey @FIN-JBarile Please implement what is described above; and let me know when you think this issue has been solved so that I can run a new test with the bridge between FishBase/SeaLifeBase and OSMOSE. Many thanks.

FIN-casey · 2017-06-25T13:54:45Z

Hi, @agruss2 , I would like to clarify some things. Shouldn't we have a maximum number of species for the extended list for every functional group? And how do we separate the list of species from the actual list and the extended list? I am planning to add the extended list in the json data before passing it to the API. The nodes of the functional group would be "name","type","taxa", and "othertaxa". The "othertaxa" will be the extended species list listing the name and the url of each of the species just like the format of the "taxa" node.
@jhpoelen, The above implementation requires some modification of the API. Please tell me if you think of a better implementation.

agruss2 · 2017-06-25T18:46:21Z

@FIN-casey
Here are my answers to your comments:
(1) I think that it would be better not to have a maximum number of species for the extended list for every functional group; this way, we would maximize the probability to obtain a value for parameters.
However, I guess that, for computional and practical reasons, you may want me to decide of a maximum number of species for the extended list for every functional group. Is this the case? If so, then please let me know and I'll provide you with a maximum number.
(2) You are right that we should think of a way to distinguish between the original list of species and the list of additional species. I think that distinguishing between "taxa" and "othertaxa" is an excellent idea. This way, the API will be able to work first with "taxa" only; and will then consider "othertaxa", if need be.

@jhpoelen Please see my comments above.
Could you please make sure that the "functional_groups.csv" file that is provided in the "osmose_config.zip" file lists only "taxa" (and, therefore, does not provide any information about "othertaxa").

jhpoelen · 2017-06-26T17:11:52Z

@FIN-casey Rather than introducing othertaxa I'd like to suggest to introduce an attribute (or property) in the taxon object.

Currently, we have:

{
  ...
  "groups": [
    {
      "name": "Amberjacks",
      "type": "focal",
      "taxa": [
        {
          "name": "Seriola dumerili",
          "url": "http://fishbase.org/summary/1005"
        },
        {
          "name": "Seriola fasciata",
          "url": "http://fishbase.org/summary/1006"
        }
      ]
    }
    ...

For taxa related to the explicitly mentioned species in the functional groups, I'd label the taxon object with an optional property like "selectionCriteria" and value "implicit", "inferred" or "same genus". The (omitted) default value for "selectionCriteria" would be "exact".

So, including a related species by genus like Seriola hippos (same genus, not exact match), would produce something like:

{
  ...
  "groups": [
    {
      "name": "Amberjacks",
      "type": "focal",
      "taxa": [
        {
          "name": "Seriola dumerili",
          "url": "http://fishbase.org/summary/1005"
        },
        {
          "name": "Seriola fasciata",
          "url": "http://fishbase.org/summary/1006"
        },
        {
          "name": "Seriola hippos",
          "url": "http://fishbase.org/summary/1960",
          "selectionCriteria": "implicit"
        }
      ]
    }
    ...

This solution would make it easy to filter or otherwise transform/group taxa by their selection criteria and would require minimal changes to existing code.

jhpoelen · 2017-06-26T17:13:34Z

@FIN-casey The order of the taxa in the group definition would determine the order in which they are considered. So, if you'd like to make sure to consider the implicitly selected taxa, then add them after the explicitly selected taxa.

Please confirm that this works for you.

jhpoelen · 2017-08-15T21:53:07Z

On creating an osmose configuration for Iceland Shelf/Sea, I noticed that none of the extended taxa seem to make it into the request from the UI to the API (see below).
I've attached the resulting osmose config here: osmose_config (6).zip

@FIN-casey is there any thing the API needs to do to get access to the extended taxon list? If this is not yet implemented, I'd strongly suggest to use the method I suggested in #115 (comment) .

Please let me know if there's anything I missed.

This is what got sent from UI to API:

{
  "timeStepsPerYear": "12",
  "groups": [
    {
      "name": "fishpelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Mallotus villosus",
          "url": "http://fishbase.org/summary/252"
        },
        {
          "name": "Benthosema glaciale",
          "url": "http://fishbase.org/summary/21"
        },
        {
          "name": "Notoscopelus kroyeri",
          "url": "http://fishbase.org/summary/27753"
        }
      ]
    },
    {
      "name": "fishpelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Scomber scombrus",
          "url": "http://fishbase.org/summary/118"
        },
        {
          "name": "Sardina pilchardus",
          "url": "http://fishbase.org/summary/1350"
        },
        {
          "name": "Alosa fallax",
          "url": "http://fishbase.org/summary/5355"
        },
        {
          "name": "Trachurus trachurus",
          "url": "http://fishbase.org/summary/1365"
        },
        {
          "name": "Scomberesox saurus",
          "url": "http://fishbase.org/summary/1084"
        },
        {
          "name": "Schedophilus medusophagus",
          "url": "http://fishbase.org/summary/1721"
        }
      ]
    },
    {
      "name": "fishpelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Salmo trutta",
          "url": "http://fishbase.org/summary/238"
        },
        {
          "name": "Sebastes norvegicus",
          "url": "http://fishbase.org/summary/501"
        },
        {
          "name": "Belone belone",
          "url": "http://fishbase.org/summary/47"
        },
        {
          "name": "Brama brama",
          "url": "http://fishbase.org/summary/391"
        },
        {
          "name": "Mola mola",
          "url": "http://fishbase.org/summary/1732"
        },
        {
          "name": "Regalecus glesne",
          "url": "http://fishbase.org/summary/3267"
        }
      ]
    },
    {
      "name": "fishdemersalsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Agonus cataphractus",
          "url": "http://fishbase.org/summary/36"
        },
        {
          "name": "Ammodytes tobianus",
          "url": "http://fishbase.org/summary/38"
        },
        {
          "name": "Liparis liparis",
          "url": "http://fishbase.org/summary/1378"
        },
        {
          "name": "Callionymus maculatus",
          "url": "http://fishbase.org/summary/1807"
        },
        {
          "name": "Ciliata mustela",
          "url": "http://fishbase.org/summary/1290"
        },
        {
          "name": "Ciliata septentrionalis",
          "url": "http://fishbase.org/summary/1873"
        },
        {
          "name": "Triglops murrayi",
          "url": "http://fishbase.org/summary/4146"
        },
        {
          "name": "Liparis montagui",
          "url": "http://fishbase.org/summary/1379"
        },
        {
          "name": "Micrenophrys lilljeborgii",
          "url": "http://fishbase.org/summary/9198"
        }
      ]
    },
    {
      "name": "fishdemersalmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Eutrigla gurnardus",
          "url": "http://fishbase.org/summary/68"
        },
        {
          "name": "Callionymus lyra",
          "url": "http://fishbase.org/summary/23"
        },
        {
          "name": "Myoxocephalus scorpius",
          "url": "http://fishbase.org/summary/1329"
        },
        {
          "name": "Pholis gunnellus",
          "url": "http://fishbase.org/summary/3804"
        },
        {
          "name": "Lumpenus lampretaeformis",
          "url": "http://fishbase.org/summary/1380"
        },
        {
          "name": "Boreogadus saida",
          "url": "http://fishbase.org/summary/319"
        },
        {
          "name": "Enchelyopus cimbrius",
          "url": "http://fishbase.org/summary/1874"
        },
        {
          "name": "Hyperoplus lanceolatus",
          "url": "http://fishbase.org/summary/1374"
        },
        {
          "name": "Sebastes viviparus",
          "url": "http://fishbase.org/summary/1352"
        },
        {
          "name": "Entelurus aequoreus",
          "url": "http://fishbase.org/summary/67"
        },
        {
          "name": "Myxine glutinosa",
          "url": "http://fishbase.org/summary/2513"
        }
      ]
    },
    {
      "name": "fishdemersallg",
      "type": "focal",
      "taxa": [
        {
          "name": "Dicentrarchus labrax",
          "url": "http://fishbase.org/summary/63"
        },
        {
          "name": "Melanogrammus aeglefinus",
          "url": "http://fishbase.org/summary/1381"
        },
        {
          "name": "Pollachius virens",
          "url": "http://fishbase.org/summary/1343"
        },
        {
          "name": "Chelon labrosus",
          "url": "http://fishbase.org/summary/2499"
        },
        {
          "name": "Merluccius merluccius",
          "url": "http://fishbase.org/summary/30"
        },
        {
          "name": "Molva molva",
          "url": "http://fishbase.org/summary/33"
        },
        {
          "name": "Brosme brosme",
          "url": "http://fishbase.org/summary/51"
        },
        {
          "name": "Polyprion americanus",
          "url": "http://fishbase.org/summary/349"
        },
        {
          "name": "Petromyzon marinus",
          "url": "http://fishbase.org/summary/2530"
        },
        {
          "name": "Molva dypterygia",
          "url": "http://fishbase.org/summary/1383"
        },
        {
          "name": "Anarhichas lupus",
          "url": "http://fishbase.org/summary/2501"
        },
        {
          "name": "Conger conger",
          "url": "http://fishbase.org/summary/301"
        },
        {
          "name": "Anarhichas minor",
          "url": "http://fishbase.org/summary/3811"
        }
      ]
    },
    {
      "name": "fishbathypelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Myctophum punctatum",
          "url": "http://fishbase.org/summary/1328"
        },
        {
          "name": "Maurolicus muelleri",
          "url": "http://fishbase.org/summary/754"
        },
        {
          "name": "Argyropelecus hemigymnus",
          "url": "http://fishbase.org/summary/6968"
        },
        {
          "name": "Argyropelecus olfersii",
          "url": "http://fishbase.org/summary/5097"
        }
      ]
    },
    {
      "name": "fishbathypelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Hoplostethus atlanticus",
          "url": "http://fishbase.org/summary/334"
        },
        {
          "name": "Micromesistius poutassou",
          "url": "http://fishbase.org/summary/31"
        },
        {
          "name": "Sebastes mentella",
          "url": "http://fishbase.org/summary/505"
        },
        {
          "name": "Coryphaenoides mediterraneus",
          "url": "http://fishbase.org/summary/8175"
        },
        {
          "name": "Coryphaenoides brevibarbis",
          "url": "http://fishbase.org/summary/27726"
        }
      ]
    },
    {
      "name": "fishbathypelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Aphanopus carbo",
          "url": "http://fishbase.org/summary/646"
        },
        {
          "name": "Coryphaenoides rupestris",
          "url": "http://fishbase.org/summary/332"
        },
        {
          "name": "Lampris guttatus",
          "url": "http://fishbase.org/summary/1072"
        },
        {
          "name": "Trachipterus arcticus",
          "url": "http://fishbase.org/summary/3265"
        },
        {
          "name": "Centrolophus niger",
          "url": "http://fishbase.org/summary/249"
        },
        {
          "name": "Nemichthys scolopaceus",
          "url": "http://fishbase.org/summary/2660"
        },
        {
          "name": "Coryphaenoides armatus",
          "url": "http://fishbase.org/summary/5127"
        }
      ]
    },
    {
      "name": "fishbathydemersalmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Argentina silus",
          "url": "http://fishbase.org/summary/2700"
        },
        {
          "name": "Argentina sphyraena",
          "url": "http://fishbase.org/summary/20"
        },
        {
          "name": "Coryphaenoides guentheri",
          "url": "http://fishbase.org/summary/8176"
        },
        {
          "name": "Echiodon drummondii",
          "url": "http://fishbase.org/summary/65"
        },
        {
          "name": "Careproctus reinhardti",
          "url": "http://fishbase.org/summary/15500"
        },
        {
          "name": "Bathysaurus ferox",
          "url": "http://fishbase.org/summary/11841"
        }
      ]
    },
    {
      "name": "fishbathydemersallg",
      "type": "focal",
      "taxa": [
        {
          "name": "Lophius piscatorius",
          "url": "http://fishbase.org/summary/716"
        },
        {
          "name": "Alepocephalus bairdii",
          "url": "http://fishbase.org/summary/230"
        },
        {
          "name": "Beryx decadactylus",
          "url": "http://fishbase.org/summary/1319"
        },
        {
          "name": "Chimaera monstrosa",
          "url": "http://fishbase.org/summary/2503"
        },
        {
          "name": "Synaphobranchus kaupii",
          "url": "http://fishbase.org/summary/9104"
        }
      ]
    },
    {
      "name": "fishbenthopelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Gasterosteus aculeatus",
          "url": "http://fishbase.org/summary/2420"
        },
        {
          "name": "Ammodytes marinus",
          "url": "http://fishbase.org/summary/37"
        }
      ]
    },
    {
      "name": "fishbenthopelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Clupea harengus",
          "url": "http://fishbase.org/summary/24"
        },
        {
          "name": "Merlangius merlangus",
          "url": "http://fishbase.org/summary/29"
        },
        {
          "name": "Pagellus bogaraveo",
          "url": "http://fishbase.org/summary/890"
        },
        {
          "name": "Trisopterus esmarkii",
          "url": "http://fishbase.org/summary/1023"
        },
        {
          "name": "Cyclopterus lumpus",
          "url": "http://fishbase.org/summary/62"
        },
        {
          "name": "Coelorinchus caelorhincus",
          "url": "http://fishbase.org/summary/1726"
        }
      ]
    },
    {
      "name": "fishbenthopelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Gadus morhua",
          "url": "http://fishbase.org/summary/69"
        },
        {
          "name": "Salmo salar",
          "url": "http://fishbase.org/summary/236"
        },
        {
          "name": "Salvelinus alpinus",
          "url": "http://sealifebase.org/summary/alpinus"
        },
        {
          "name": "Macrourus berglax",
          "url": "http://fishbase.org/summary/331"
        },
        {
          "name": "Pollachius pollachius",
          "url": "http://fishbase.org/summary/34"
        },
        {
          "name": "Phycis blennoides",
          "url": "http://fishbase.org/summary/1340"
        },
        {
          "name": "Anarhichas denticulatus",
          "url": "http://fishbase.org/summary/3810"
        },
        {
          "name": "Argyrosomus regius",
          "url": "http://fishbase.org/summary/418"
        },
        {
          "name": "Notacanthus chemnitzii",
          "url": "http://fishbase.org/summary/2661"
        }
      ]
    },
    {
      "name": "fishreefassociatedsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Lebetus scorpioides",
          "url": "http://fishbase.org/summary/456"
        }
      ]
    },
    {
      "name": "fishsharksmmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Etmopterus princeps",
          "url": "http://fishbase.org/summary/683"
        },
        {
          "name": "Etmopterus spinax",
          "url": "http://fishbase.org/summary/687"
        }
      ]
    },
    {
      "name": "fishsharklg",
      "type": "focal",
      "taxa": [
        {
          "name": "Lamna nasus",
          "url": "http://fishbase.org/summary/88"
        },
        {
          "name": "Squalus acanthias",
          "url": "http://fishbase.org/summary/139"
        },
        {
          "name": "Galeorhinus galeus",
          "url": "http://fishbase.org/summary/4642"
        },
        {
          "name": "Centrophorus squamosus",
          "url": "http://fishbase.org/summary/653"
        },
        {
          "name": "Cetorhinus maximus",
          "url": "http://fishbase.org/summary/90"
        },
        {
          "name": "Deania calcea",
          "url": "http://fishbase.org/summary/670"
        },
        {
          "name": "Somniosus microcephalus",
          "url": "http://fishbase.org/summary/138"
        },
        {
          "name": "Centroscymnus coelolepis",
          "url": "http://fishbase.org/summary/662"
        },
        {
          "name": "Centroscymnus crepidater",
          "url": "http://fishbase.org/summary/663"
        },
        {
          "name": "Dalatias licha",
          "url": "http://fishbase.org/summary/669"
        },
        {
          "name": "Hexanchus griseus",
          "url": "http://fishbase.org/summary/637"
        },
        {
          "name": "Centroscyllium fabricii",
          "url": "http://fishbase.org/summary/656"
        }
      ]
    },
    {
      "name": "fishraylg",
      "type": "focal",
      "taxa": [
        {
          "name": "Amblyraja radiata",
          "url": "http://fishbase.org/summary/2565"
        },
        {
          "name": "Dipturus batis",
          "url": "http://fishbase.org/summary/2058"
        },
        {
          "name": "Raja clavata",
          "url": "http://fishbase.org/summary/2059"
        },
        {
          "name": "Rajella lintea",
          "url": "http://fishbase.org/summary/6422"
        }
      ]
    },
    {
      "name": "fishflatfishsmmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Hippoglossoides platessoides",
          "url": "http://fishbase.org/summary/4239"
        },
        {
          "name": "Glyptocephalus cynoglossus",
          "url": "http://fishbase.org/summary/26"
        },
        {
          "name": "Lepidorhombus whiffiagonis",
          "url": "http://fishbase.org/summary/28"
        },
        {
          "name": "Phrynorhombus norvegicus",
          "url": "http://fishbase.org/summary/2360"
        }
      ]
    },
    {
      "name": "fishflatfishlg",
      "type": "focal",
      "taxa": [
        {
          "name": "Pleuronectes platessa",
          "url": "http://fishbase.org/summary/1342"
        },
        {
          "name": "Scophthalmus maximus",
          "url": "http://fishbase.org/summary/1348"
        },
        {
          "name": "Hippoglossus hippoglossus",
          "url": "http://fishbase.org/summary/1371"
        },
        {
          "name": "Reinhardtius hippoglossoides",
          "url": "http://fishbase.org/summary/516"
        }
      ]
    },
    {
      "name": "cephalopods",
      "type": "focal",
      "taxa": [
        {
          "name": "Todarodes sagittatus",
          "url": "http://fishbase.org/summary/57559"
        },
        {
          "name": "Rossia macrosoma",
          "url": "http://fishbase.org/summary/57462"
        },
        {
          "name": "Sepiola atlantica",
          "url": "http://fishbase.org/summary/57471"
        },
        {
          "name": "Neorossia caroli",
          "url": "http://fishbase.org/summary/57324"
        }
      ]
    },
    {
      "name": "zooplankton",
      "type": "background",
      "taxa": []
    },
    {
      "name": "phytoplankton",
      "type": "background",
      "taxa": []
    }
  ]
}

FIN-casey · 2017-08-16T10:53:39Z

@jhpoelen The inclusion of the extended list is not yet implemented. I would follow your suggestion in implementing this. Hopefully we won't encounter any limitation issues due to the large amount of data.

…

On Aug 16, 2017 5:53 AM, "Jorrit Poelen" ***@***.***> wrote: On creating an osmose configuration for Iceland Shelf/Sea, I noticed that none of the extended taxa seem to make it into the request from the UI to the API (see below). I've attached the resulting osmose config here: osmose_config (6).zip <https://github.com/jhpoelen/fb-osmose-bridge/files/1226555/osmose_config.6.zip> @FIN-casey <https://github.com/fin-casey> is there any thing the API needs to do to get access to the extended taxon list? If this is not yet implemented, I'd strongly suggest to use the method I suggested in #115 (comment) <#115 (comment)> . Please let me know if there's anything I missed. This is what got sent from UI to API: { "timeStepsPerYear": "12", "groups": [ { "name": "fishpelagicsm", "type": "focal", "taxa": [ { "name": "Mallotus villosus", "url": "http://fishbase.org/summary/252" }, { "name": "Benthosema glaciale", "url": "http://fishbase.org/summary/21" }, { "name": "Notoscopelus kroyeri", "url": "http://fishbase.org/summary/27753" } ] }, { "name": "fishpelagicmd", "type": "focal", "taxa": [ { "name": "Scomber scombrus", "url": "http://fishbase.org/summary/118" }, { "name": "Sardina pilchardus", "url": "http://fishbase.org/summary/1350" }, { "name": "Alosa fallax", "url": "http://fishbase.org/summary/5355" }, { "name": "Trachurus trachurus", "url": "http://fishbase.org/summary/1365" }, { "name": "Scomberesox saurus", "url": "http://fishbase.org/summary/1084" }, { "name": "Schedophilus medusophagus", "url": "http://fishbase.org/summary/1721" } ] }, { "name": "fishpelagiclg", "type": "focal", "taxa": [ { "name": "Salmo trutta", "url": "http://fishbase.org/summary/238" }, { "name": "Sebastes norvegicus", "url": "http://fishbase.org/summary/501" }, { "name": "Belone belone", "url": "http://fishbase.org/summary/47" }, { "name": "Brama brama", "url": "http://fishbase.org/summary/391" }, { "name": "Mola mola", "url": "http://fishbase.org/summary/1732" }, { "name": "Regalecus glesne", "url": "http://fishbase.org/summary/3267" } ] }, { "name": "fishdemersalsm", "type": "focal", "taxa": [ { "name": "Agonus cataphractus", "url": "http://fishbase.org/summary/36" }, { "name": "Ammodytes tobianus", "url": "http://fishbase.org/summary/38" }, { "name": "Liparis liparis", "url": "http://fishbase.org/summary/1378" }, { "name": "Callionymus maculatus", "url": "http://fishbase.org/summary/1807" }, { "name": "Ciliata mustela", "url": "http://fishbase.org/summary/1290" }, { "name": "Ciliata septentrionalis", "url": "http://fishbase.org/summary/1873" }, { "name": "Triglops murrayi", "url": "http://fishbase.org/summary/4146" }, { "name": "Liparis montagui", "url": "http://fishbase.org/summary/1379" }, { "name": "Micrenophrys lilljeborgii", "url": "http://fishbase.org/summary/9198" } ] }, { "name": "fishdemersalmd", "type": "focal", "taxa": [ { "name": "Eutrigla gurnardus", "url": "http://fishbase.org/summary/68" }, { "name": "Callionymus lyra", "url": "http://fishbase.org/summary/23" }, { "name": "Myoxocephalus scorpius", "url": "http://fishbase.org/summary/1329" }, { "name": "Pholis gunnellus", "url": "http://fishbase.org/summary/3804" }, { "name": "Lumpenus lampretaeformis", "url": "http://fishbase.org/summary/1380" }, { "name": "Boreogadus saida", "url": "http://fishbase.org/summary/319" }, { "name": "Enchelyopus cimbrius", "url": "http://fishbase.org/summary/1874" }, { "name": "Hyperoplus lanceolatus", "url": "http://fishbase.org/summary/1374" }, { "name": "Sebastes viviparus", "url": "http://fishbase.org/summary/1352" }, { "name": "Entelurus aequoreus", "url": "http://fishbase.org/summary/67" }, { "name": "Myxine glutinosa", "url": "http://fishbase.org/summary/2513" } ] }, { "name": "fishdemersallg", "type": "focal", "taxa": [ { "name": "Dicentrarchus labrax", "url": "http://fishbase.org/summary/63" }, { "name": "Melanogrammus aeglefinus", "url": "http://fishbase.org/summary/1381" }, { "name": "Pollachius virens", "url": "http://fishbase.org/summary/1343" }, { "name": "Chelon labrosus", "url": "http://fishbase.org/summary/2499" }, { "name": "Merluccius merluccius", "url": "http://fishbase.org/summary/30" }, { "name": "Molva molva", "url": "http://fishbase.org/summary/33" }, { "name": "Brosme brosme", "url": "http://fishbase.org/summary/51" }, { "name": "Polyprion americanus", "url": "http://fishbase.org/summary/349" }, { "name": "Petromyzon marinus", "url": "http://fishbase.org/summary/2530" }, { "name": "Molva dypterygia", "url": "http://fishbase.org/summary/1383" }, { "name": "Anarhichas lupus", "url": "http://fishbase.org/summary/2501" }, { "name": "Conger conger", "url": "http://fishbase.org/summary/301" }, { "name": "Anarhichas minor", "url": "http://fishbase.org/summary/3811" } ] }, { "name": "fishbathypelagicsm", "type": "focal", "taxa": [ { "name": "Myctophum punctatum", "url": "http://fishbase.org/summary/1328" }, { "name": "Maurolicus muelleri", "url": "http://fishbase.org/summary/754" }, { "name": "Argyropelecus hemigymnus", "url": "http://fishbase.org/summary/6968" }, { "name": "Argyropelecus olfersii", "url": "http://fishbase.org/summary/5097" } ] }, { "name": "fishbathypelagicmd", "type": "focal", "taxa": [ { "name": "Hoplostethus atlanticus", "url": "http://fishbase.org/summary/334" }, { "name": "Micromesistius poutassou", "url": "http://fishbase.org/summary/31" }, { "name": "Sebastes mentella", "url": "http://fishbase.org/summary/505" }, { "name": "Coryphaenoides mediterraneus", "url": "http://fishbase.org/summary/8175" }, { "name": "Coryphaenoides brevibarbis", "url": "http://fishbase.org/summary/27726" } ] }, { "name": "fishbathypelagiclg", "type": "focal", "taxa": [ { "name": "Aphanopus carbo", "url": "http://fishbase.org/summary/646" }, { "name": "Coryphaenoides rupestris", "url": "http://fishbase.org/summary/332" }, { "name": "Lampris guttatus", "url": "http://fishbase.org/summary/1072" }, { "name": "Trachipterus arcticus", "url": "http://fishbase.org/summary/3265" }, { "name": "Centrolophus niger", "url": "http://fishbase.org/summary/249" }, { "name": "Nemichthys scolopaceus", "url": "http://fishbase.org/summary/2660" }, { "name": "Coryphaenoides armatus", "url": "http://fishbase.org/summary/5127" } ] }, { "name": "fishbathydemersalmd", "type": "focal", "taxa": [ { "name": "Argentina silus", "url": "http://fishbase.org/summary/2700" }, { "name": "Argentina sphyraena", "url": "http://fishbase.org/summary/20" }, { "name": "Coryphaenoides guentheri", "url": "http://fishbase.org/summary/8176" }, { "name": "Echiodon drummondii", "url": "http://fishbase.org/summary/65" }, { "name": "Careproctus reinhardti", "url": "http://fishbase.org/summary/15500" }, { "name": "Bathysaurus ferox", "url": "http://fishbase.org/summary/11841" } ] }, { "name": "fishbathydemersallg", "type": "focal", "taxa": [ { "name": "Lophius piscatorius", "url": "http://fishbase.org/summary/716" }, { "name": "Alepocephalus bairdii", "url": "http://fishbase.org/summary/230" }, { "name": "Beryx decadactylus", "url": "http://fishbase.org/summary/1319" }, { "name": "Chimaera monstrosa", "url": "http://fishbase.org/summary/2503" }, { "name": "Synaphobranchus kaupii", "url": "http://fishbase.org/summary/9104" } ] }, { "name": "fishbenthopelagicsm", "type": "focal", "taxa": [ { "name": "Gasterosteus aculeatus", "url": "http://fishbase.org/summary/2420" }, { "name": "Ammodytes marinus", "url": "http://fishbase.org/summary/37" } ] }, { "name": "fishbenthopelagicmd", "type": "focal", "taxa": [ { "name": "Clupea harengus", "url": "http://fishbase.org/summary/24" }, { "name": "Merlangius merlangus", "url": "http://fishbase.org/summary/29" }, { "name": "Pagellus bogaraveo", "url": "http://fishbase.org/summary/890" }, { "name": "Trisopterus esmarkii", "url": "http://fishbase.org/summary/1023" }, { "name": "Cyclopterus lumpus", "url": "http://fishbase.org/summary/62" }, { "name": "Coelorinchus caelorhincus", "url": "http://fishbase.org/summary/1726" } ] }, { "name": "fishbenthopelagiclg", "type": "focal", "taxa": [ { "name": "Gadus morhua", "url": "http://fishbase.org/summary/69" }, { "name": "Salmo salar", "url": "http://fishbase.org/summary/236" }, { "name": "Salvelinus alpinus", "url": "http://sealifebase.org/summary/alpinus" }, { "name": "Macrourus berglax", "url": "http://fishbase.org/summary/331" }, { "name": "Pollachius pollachius", "url": "http://fishbase.org/summary/34" }, { "name": "Phycis blennoides", "url": "http://fishbase.org/summary/1340" }, { "name": "Anarhichas denticulatus", "url": "http://fishbase.org/summary/3810" }, { "name": "Argyrosomus regius", "url": "http://fishbase.org/summary/418" }, { "name": "Notacanthus chemnitzii", "url": "http://fishbase.org/summary/2661" } ] }, { "name": "fishreefassociatedsm", "type": "focal", "taxa": [ { "name": "Lebetus scorpioides", "url": "http://fishbase.org/summary/456" } ] }, { "name": "fishsharksmmd", "type": "focal", "taxa": [ { "name": "Etmopterus princeps", "url": "http://fishbase.org/summary/683" }, { "name": "Etmopterus spinax", "url": "http://fishbase.org/summary/687" } ] }, { "name": "fishsharklg", "type": "focal", "taxa": [ { "name": "Lamna nasus", "url": "http://fishbase.org/summary/88" }, { "name": "Squalus acanthias", "url": "http://fishbase.org/summary/139" }, { "name": "Galeorhinus galeus", "url": "http://fishbase.org/summary/4642" }, { "name": "Centrophorus squamosus", "url": "http://fishbase.org/summary/653" }, { "name": "Cetorhinus maximus", "url": "http://fishbase.org/summary/90" }, { "name": "Deania calcea", "url": "http://fishbase.org/summary/670" }, { "name": "Somniosus microcephalus", "url": "http://fishbase.org/summary/138" }, { "name": "Centroscymnus coelolepis", "url": "http://fishbase.org/summary/662" }, { "name": "Centroscymnus crepidater", "url": "http://fishbase.org/summary/663" }, { "name": "Dalatias licha", "url": "http://fishbase.org/summary/669" }, { "name": "Hexanchus griseus", "url": "http://fishbase.org/summary/637" }, { "name": "Centroscyllium fabricii", "url": "http://fishbase.org/summary/656" } ] }, { "name": "fishraylg", "type": "focal", "taxa": [ { "name": "Amblyraja radiata", "url": "http://fishbase.org/summary/2565" }, { "name": "Dipturus batis", "url": "http://fishbase.org/summary/2058" }, { "name": "Raja clavata", "url": "http://fishbase.org/summary/2059" }, { "name": "Rajella lintea", "url": "http://fishbase.org/summary/6422" } ] }, { "name": "fishflatfishsmmd", "type": "focal", "taxa": [ { "name": "Hippoglossoides platessoides", "url": "http://fishbase.org/summary/4239" }, { "name": "Glyptocephalus cynoglossus", "url": "http://fishbase.org/summary/26" }, { "name": "Lepidorhombus whiffiagonis", "url": "http://fishbase.org/summary/28" }, { "name": "Phrynorhombus norvegicus", "url": "http://fishbase.org/summary/2360" } ] }, { "name": "fishflatfishlg", "type": "focal", "taxa": [ { "name": "Pleuronectes platessa", "url": "http://fishbase.org/summary/1342" }, { "name": "Scophthalmus maximus", "url": "http://fishbase.org/summary/1348" }, { "name": "Hippoglossus hippoglossus", "url": "http://fishbase.org/summary/1371" }, { "name": "Reinhardtius hippoglossoides", "url": "http://fishbase.org/summary/516" } ] }, { "name": "cephalopods", "type": "focal", "taxa": [ { "name": "Todarodes sagittatus", "url": "http://fishbase.org/summary/57559" }, { "name": "Rossia macrosoma", "url": "http://fishbase.org/summary/57462" }, { "name": "Sepiola atlantica", "url": "http://fishbase.org/summary/57471" }, { "name": "Neorossia caroli", "url": "http://fishbase.org/summary/57324" } ] }, { "name": "zooplankton", "type": "background", "taxa": [] }, { "name": "phytoplankton", "type": "background", "taxa": [] } ] } — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#115 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AReVf3fQBSpslBUkaUwbJ1knzFOxnoLXks5sYhNEgaJpZM4Nm9A2> .

jhpoelen · 2017-08-16T17:35:17Z

@FIN-casey Thanks for your comment that helps explain why @agruss2 does not see the expected behavior. Assigning issue to you.

FIN-casey · 2017-08-27T15:55:56Z

@agruss2 @jhpoelen , I implemented the inclusion of "implicit" species in the data sent to the API. The implementation is now in the wizard. Can you please verify and test the changes?

Thanks.
Please take note that there is a delay in showing the message "Preparing configuration..." after clicking Submit.

agruss2 · 2017-08-27T17:06:17Z

@FIN-casey @jhpoelen I just ran a test and I can confirm that the implementation is successful; many thanks Casey! Therefore, I am going to close the present issue.
However, please note that I see some potential areas of improvement. Therefore, I will soon create new GitHub issues to address some minor problems.

jhpoelen · 2017-08-27T21:33:49Z

@FIN-casey @agruss2 good the hear that the parameter estimation using the implicit groups is now working as expected. Thanks for fixing and testing!

agruss2 · 2017-09-01T20:19:58Z

@jhpoelen @FIN-casey
I am reopening this issue, because I am still having a few NAs here and there for focal functional groups.
Therefore, I think that, for each functional group, we should increase the number of species that the bridge between FishBase/SeaLifeBase and OSMOSE considers in addition to the species making up the functional groups.
How many additional species does the bridge currently consider for each focal functional group?
How much could we increase that number without making the bridge crash?
Many thanks.

jhpoelen · 2017-09-04T20:27:25Z

@agruss2 the API does not control the amount of taxa that are associated with focal/background functional groups. This is why I consider this a UI issue.

Same applies to #114 .

agruss2 added the API label May 25, 2017

agruss2 added this to the Before June, 4th 2017 milestone May 25, 2017

agruss2 assigned jhpoelen May 25, 2017

jhpoelen pushed a commit that referenced this issue May 26, 2017

introduce explicit NA values related to issue #115

f5fcbc9

jhpoelen pushed a commit that referenced this issue May 26, 2017

replaced default values for properties mentioned in #115 with NA as s…

15ef3fb

…uggested by @agruss2

jhpoelen mentioned this issue May 26, 2017

Making sure to provide as few NAs as possible for background functional groups #114

Closed

agruss2 modified the milestones: Before June, 11th 2017, Before June, 4th 2017 Jun 3, 2017

agruss2 added UI High priority and removed API labels Jun 10, 2017

agruss2 assigned FIN-JBarile and FIN-casey and unassigned jhpoelen Jun 10, 2017

agruss2 modified the milestones: Before June, 18th 2017, Before June, 11th 2017, Before June, 25th 2017 Jun 10, 2017

agruss2 modified the milestones: Before July, 2nd 2017, Before June, 25th 2017 Jun 24, 2017

agruss2 added the API label Jun 25, 2017

agruss2 assigned jhpoelen Jun 25, 2017

agruss2 modified the milestones: Before July, 30th 2017, Before August, 13th 2017, Before August, 6th 2017 Jul 29, 2017

agruss2 modified the milestones: Before August, 20th 2017, Before August, 13th 2017 Aug 12, 2017

jhpoelen mentioned this issue Aug 15, 2017

Parameter values for the zooplankton group #121

Closed

jhpoelen removed their assignment Aug 16, 2017

jhpoelen removed the API label Aug 16, 2017

agruss2 modified the milestones: Before August, 27th 2017, Before August, 20th 2017 Aug 19, 2017

agruss2 modified the milestones: Before September, 3rd 2017, Before August, 27th 2017 Aug 26, 2017

agruss2 closed this as completed Aug 27, 2017

agruss2 reopened this Sep 1, 2017

agruss2 changed the title ~~Parameters of the focal functional groups for which it is not reasonable to define a default value~~ Making sure to provide as few NAs as possible for focal functional groups Sep 1, 2017

agruss2 assigned jhpoelen and unassigned FIN-JBarile Sep 1, 2017

agruss2 added the API label Sep 1, 2017

agruss2 modified the milestones: Before September, 10th 2017, Before September, 3rd 2017 Sep 1, 2017

agruss2 closed this as completed Sep 8, 2017

agruss2 mentioned this issue Nov 10, 2017

Making sure that we obtain as few NAs as possible for the focal functional groups #141

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Making sure to provide as few NAs as possible for focal functional groups #115

Making sure to provide as few NAs as possible for focal functional groups #115

agruss2 commented May 25, 2017

jhpoelen commented May 26, 2017 via email

jhpoelen commented May 26, 2017

agruss2 commented May 28, 2017 •

edited

Loading

jhpoelen commented Jun 5, 2017

agruss2 commented Jun 5, 2017

FIN-casey commented Jun 25, 2017

agruss2 commented Jun 25, 2017

jhpoelen commented Jun 26, 2017 •

edited

Loading

jhpoelen commented Jun 26, 2017 •

edited

Loading

jhpoelen commented Aug 15, 2017

FIN-casey commented Aug 16, 2017 via email

jhpoelen commented Aug 16, 2017

FIN-casey commented Aug 27, 2017

agruss2 commented Aug 27, 2017

jhpoelen commented Aug 27, 2017

agruss2 commented Sep 1, 2017

jhpoelen commented Sep 4, 2017

Making sure to provide as few NAs as possible for focal functional groups #115

Making sure to provide as few NAs as possible for focal functional groups #115

Comments

agruss2 commented May 25, 2017

jhpoelen commented May 26, 2017 via email

jhpoelen commented May 26, 2017

agruss2 commented May 28, 2017 • edited Loading

jhpoelen commented Jun 5, 2017

agruss2 commented Jun 5, 2017

FIN-casey commented Jun 25, 2017

agruss2 commented Jun 25, 2017

jhpoelen commented Jun 26, 2017 • edited Loading

jhpoelen commented Jun 26, 2017 • edited Loading

jhpoelen commented Aug 15, 2017

FIN-casey commented Aug 16, 2017 via email

jhpoelen commented Aug 16, 2017

FIN-casey commented Aug 27, 2017

agruss2 commented Aug 27, 2017

jhpoelen commented Aug 27, 2017

agruss2 commented Sep 1, 2017

jhpoelen commented Sep 4, 2017

agruss2 commented May 28, 2017 •

edited

Loading

jhpoelen commented Jun 26, 2017 •

edited

Loading

jhpoelen commented Jun 26, 2017 •

edited

Loading