#ValueError: to get argmax of an empty sequence

[IgorK7]

Hi,

I get an error:
ValueError: attempt to get argmax of an empty sequence
when I include a "const" token to the library.

I would really appreciate your help with this.

Here the full set up for replication (csv test file attached):

config =  { 
...     "task" : {  "task_type" : "regression",
...     "dataset" : "/Project_dir/data/DSR_real_test.csv",
...                         "metric": "inv_nmse", # "r2_shift" "inv_nmse"
...                         "metric_params": (0,),  
...     "function_set" : ["add", "sub", "mul", "div",  "exp", "log", "sqrt","const"],
...     "reward_noise_type" : "y_hat"
...     # "reward_noise": 0.7 # , "const"
... } ,
...     "training" : {     "n_cores_batch" : -1,
...     "save_pareto_front": False
...         },
...         "prior": {
...         "length": {
...         "min_": 3,
...         "max_": 10,
...         "on": True
...     }}}

with open('/Project_dir/config.json', 'w') as f:
...     json.dump(config, f)
...
os.system('python -m dso.run /Project_dir/config.json') 
Traceback (most recent call last):
  File "", line 1, in
NameError: name 'os' is not defined
import time
import json
import copy
import os
import pandas as pd
pd.set_option("display.max_rows", 1000)
import sys
sys.setrecursionlimit(2500)
import numpy as np
import pickle
config =  { 
...     "task" : {  "task_type" : "regression",
...     "dataset" : "/Project_dir/data/DSR_real_test.csv",
...                         "metric": "inv_nmse", # "r2_shift" "inv_nmse"
...                         "metric_params": (0,),  
...     "function_set" : ["add", "sub", "mul", "div",  "exp", "log", "sqrt","const"],
...     "reward_noise_type" : "y_hat"
...     # "reward_noise": 0.7 # , "const"
... } ,
...     "training" : {     "n_cores_batch" : -1,
...     "save_pareto_front": False
...         },
...         "prior": {
...         "length": {
...         "min_": 3,
...         "max_": 10,
...         "on": True
...     }}}

with open('/Project_dir/config.json', 'w') as f:
...     json.dump(config, f)
...
os.system('python -m dso.run /Project_dir/config.json') 
 
== EXPERIMENT SETUP START ===========
Task type            : regression
Dataset              : /Project_dir/data/DSR_real_test.csv
Starting seed        : 0
Runs                 : 1
== EXPERIMENT SETUP END =============
 
== TRAINING SEED 0 START ============
-- BUILDING PRIOR START -------------
WARNING: Skipping invalid 'RelationalConstraint' with arguments {'targets': [], 'effectors': [], 'relationship': None}. Reason: Prior disabled.
WARNING: Skipping invalid 'TrigConstraint' with arguments {}. Reason: There are no target Tokens.
WARNING: Skipping invalid 'UniformArityPrior' with arguments {}. Reason: Prior disabled.
WARNING: Skipping invalid 'LanguageModelPrior' with arguments {'weight': None}. Reason: Prior disabled.
LengthConstraint: Sequences have minimum length 3.
                  Sequences have maximum length 10.
RepeatConstraint: [const] cannot occur more than 10 times.
RelationalConstraint: [exp] cannot be a child of [log].
InverseUnaryConstraint: RelationalConstraint: [log] cannot be a child of [exp].
ConstConstraint: [const] cannot be the only unique child of [exp, log, sqrt, add, sub, mul, div].
NoInputsConstraint: Sequences contain at least one input variable Token.
SoftLengthPrior: No description available.
-- BUILDING PRIOR END ---------------
 
WARNING: max_length (256) will be overridden by value from LengthConstraint (10).
-- RUNNING EPOCHS START -------------
Traceback (most recent call last):
  File "/Users/User/.pyenv/versions/3.6.15/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "main", mod_spec)
  File "/Users/User/.pyenv/versions/3.6.15/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Project_dir/dso/dso/run.py", line 156, in
    main()
  File "/Project_dir/lib/python3.6/site-packages/click/core.py", line 1128, in call
    return self.main(*args, **kwargs)
  File "/Project_dir/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/Project_dir/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Project_dir/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/Project_dir/dso/dso/run.py", line 139, in main
    result, summary_path = train_dso(config)
  File "/Project_dir/dso/dso/run.py", line 33, in train_dso
    result = model.train()
  File "/Project_dir/dso/dso/core.py", line 83, in train
    **self.config_training))
  File "/Project_dir/dso/dso/train.py", line 446, in learn
    priority_queue.push_best(sampled_batch, programs)
  File "/Project_dir/dso/dso/memory.py", line 314, in push_best
    i = np.argmax(batch.rewards)
  File "<array_function internals>", line 6, in argmax
  File "/Project_dir/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 1188, in argmax
    return _wrapfunc(a, 'argmax', axis=axis, out=out)
  File "/Project_dir/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence

DSR_real_test.csv

[ed0001tt]

I have the same problem when i set y as a constant array

[IgorK7]

I did a bid digging myself and it seems like the issue comes from train.py. Specifically, the code can pick nan value as a threshold for risk seeking policy gradient. It is on lines 365-370 (approx). If one replaces np.quantile() with np.nanquantile(), then the code runs just fine (See below)

        else: # Empirical quantile   
            # IK added 02.03.2023   np.nanquantile                          
            # quantile = np.quantile(r, 1 - epsilon, interpolation="higher")
            quantile = np.nanquantile(r, 1 - epsilon, interpolation="higher")

[ed0001tt]

Oh, this seems to make the code run. but the R value is now nan.

[brendenpetersen]

Thanks @IgorK7! Yea, we have run into some issues with nan. Good to know about np.nanquantile, though it still seems like an issue to have a nan reward. We have also found nan reward values sneak in from time to time, which can cause weird behaviors, but we haven't yet been able to nail down why; and it's hard to reproduce as it depends on the dataset... Replacing nan rewards with some bad negative reward might patch this, but still not a great fix...