h2o crashes while debugging my python code using intellij/pycharm

[exalate-issue-sync]

Hi,

I'm getting the following exception while debugging my own code in intellij and pycharm.

--------------------------  ------------------------------------------------------------------
H2O cluster uptime:         02 secs
H2O cluster timezone:       America/New_York
H2O data parsing timezone:  UTC
H2O cluster version:        3.28.1.2
H2O cluster version age:    6 days
H2O cluster name:          myname
H2O cluster total nodes:    1
H2O cluster free memory:    13.33 Gb
H2O cluster total cores:    4
H2O cluster allowed cores:  4
H2O cluster status:         accepting new members, healthy
H2O connection url:         http://127.0.0.1:54321
H2O connection proxy:       {'http': None, 'https': None}
H2O internal security:      False
H2O API Extensions:         Amazon S3, XGBoost, Algos, AutoML, Core V3, TargetEncoder, Core V4
Python version:             3.6.9 final
--------------------------  ------------------------------------------------------------------
Parse progress: |█████████████████████████████████████████████████████████| 100%
===========================
Calculating Measurement: by_channel
===========================
Parse progress: |█████████████████████████████████████████████████████████| 100%
H2O session _sid_9835 closed.
Traceback (most recent call last):
  File "/Users/malarcon/Documents/omni_measurement_automation/omni_measurement/catalina_digital/glm.py", line 782, in calculate_measurement
    raise inst
  File "/Users/malarcon/Documents/omni_measurement_automation/omni_measurement/catalina_digital/glm.py", line 775, in calculate_measurement
    measurement_by_dimension(dimension, dimensions_hh, params)
  File "/Users/malarcon/Documents/omni_measurement_automation/omni_measurement/catalina_digital/glm.py", line 461, in measurement_by_dimension
    g=df.group_by("tc_flag").count().get_frame().as_data_frame()
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/group_by.py", line 343, in get_frame
    self._check_string_columns(cind)
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/group_by.py", line 370, in _check_string_columns
    if self._fr[colIndex].isstring()[0]:
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/frame.py", line 1987, in isstring
    return [bool(o) for o in ExprNode("is.character", self)._eager_scalar()]
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/expr.py", line 99, in _eager_scalar
    self._eval_driver(False)
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/expr.py", line 110, in _eval_driver
    res = ExprNode.rapids(exec_str)
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/expr.py", line 249, in rapids
    return h2o.api("POST /99/Rapids", data={"ast": expr, "session_id": h2o.connection().session_id})
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/h2o.py", line 109, in api
    return h2oconn.request(endpoint, data=data, json=json, filename=filename, save_to=save_to)
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/backend/connection.py", line 478, in request
    return self._process_response(resp, save_to)
  File "/anaconda3/envs/omni-py36/lib/python3.6/site-packages/h2o/backend/connection.py", line 824, in _process_response
    raise H2OResponseError(data)
h2o.exceptions.H2OResponseError: Server error java.lang.IllegalArgumentException:
  Error: Expected a Frame but found a class water.rapids.vals.ValNums
  Request: POST /99/Rapids
    data: {'ast': '(tmp= py_27_sid_9835 (is.character (tmp= py_26_sid_9835 (cols_py py_25_sid_9835 0))))', 'session_id': '_sid_9835'}

/anaconda3/envs/omni-py36/lib/python3.6/subprocess.py:786: ResourceWarning: subprocess 3929 is still running
  ResourceWarning, source=self)
sys:1: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 49828), raddr=('127.0.0.1', 49826)>

Process finished with exit code 1

The same code actually runs fine with the debugger off.

The same code also runs on 3.28.0.3, regardless of the debugger.

The code is rather trivial, I'm just generating a group_by summary after loading the data.

Thanks!

[exalate-issue-sync]

Nidhi Mehta commented: import h2o

h2o.init()

test_df = h2o.create_frame()
first_enum_colname = list(filter(lambda elem: elem[1] == 'enum', test_df.types.items()))[0][0]
print(test_df.group_by(first_enum_colname).count().get_frame().as_data_frame())

[exalate-issue-sync]

Neema Mashayekhi commented: Was able to repro on 3.28.1.2 and 3.30.0.1, but did not get error on 3.28.0.2 when using script below with PyCharm (debug run fails, regular run has no issue).

{code:python}import h2o

h2o.init()

test_df = h2o.create_frame()
first_enum_colname = list(filter(lambda elem: elem[1] == 'enum', test_df.types.items()))[0][0]
print(test_df.group_by(first_enum_colname).count().get_frame().as_data_frame()){code}

[exalate-issue-sync]

Daniele Cellai commented: I’m experiencing the same problem but with VSCode (python ver. 3.6.8). Affected H2O ver. 3.28.1.2, 3.29.1.3, and 3.30.0.1, not affected ver. 3.28.0.2, 3.28.0.3, and lower.

[exalate-issue-sync]

Pavel Pscheidl commented: Caused by [https://0xdata.atlassian.net/browse/PUBDEV-7227|https://0xdata.atlassian.net/browse/PUBDEV-7227|smart-link].

[exalate-issue-sync]

Pavel Pscheidl commented: Fixed by [~accountid:5b153fb1b0d76456f36daced] - can we close this one please ?

[h2o-ops]

JIRA Issue Migration Info

Jira Issue: PUBDEV-7400
Assignee: Pavel Pscheidl
Reporter: Mauricio Alarcon
State: Closed
Fix Version: 3.30.0.2
Attachments: N/A
Development PRs: Available

Linked PRs from JIRA

#4523