Get config always returns the current config generated irrespective of the trial number

[chandrams]

Get config always returns the last config generated irrespective of the trial number in case of gRPC service. With REST service, get config returns -1 for all trial numbers except the current trial for which it shows the config.

To recreate follow the below steps:

• Deploy HPO service - ./deploy_hpo.sh -c native
• Post a new experiment - python grpc_client.py new --file=../tests/resources/searchspace_jsons/newExperiment.json
• Get the config using - python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 0
• Post the result - python grpc_client.py result --name petclinic-sample-2-75884c5549-npvgd --trial 0 --result SUCCESS --value_type double --value 5745.33
• Post the next experiment - python grpc_client.py next --name petclinic-sample-2-75884c5549-npvgd
• Get the config using both trial numbers or any random trial number that has not been generated so far
  python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 10

I have posted the result for trial 0 & trial 1, trial 2 is the current config

(base) [csubrama@csubrama src]$ python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 2
{
  "config": [
    {
      "name": "memoryRequest",
      "value": 209.0
    },
    {
      "name": "cpuRequest",
      "value": 1.89
    }
  ]
}
(base) [csubrama@csubrama src]$ python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 0
{
  "config": [
    {
      "name": "memoryRequest",
      "value": 209.0
    },
    {
      "name": "cpuRequest",
      "value": 1.89
    }
  ]
}
(base) [csubrama@csubrama src]$ python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 1
{
  "config": [
    {
      "name": "memoryRequest",
      "value": 209.0
    },
    {
      "name": "cpuRequest",
      "value": 1.89
    }
  ]
}
(base) [csubrama@csubrama src]$ python grpc_client.py config --name petclinic-sample-2-75884c5549-npvgd --trial 10
{
  "config": [
    {
      "name": "memoryRequest",
      "value": 209.0
    },
    {
      "name": "cpuRequest",
      "value": 1.89
    }
  ]
}

[johnaohara]

@chandrams do you want me to fix this?

[chandrams]

Thanks for fixing this @johnaohara

In optuna we aren't using persistent storage at present, does this mean all previous trial configs for which results are posted aren't stored anywhere in memory? Going through optuna docs. @khansaad - Do you know if the previous trial configs are stored in the study & can be retrieved?

[johnaohara]

@chandrams This opens an interesting question about experiment/trial persistence. I'll open an issue to discuss

[johnaohara]

I opened: #43