Skip to content
This repository has been archived by the owner on Feb 3, 2021. It is now read-only.

Confusing error message when master is not yet selected #476

Closed
shtratos opened this issue Apr 4, 2018 · 0 comments
Closed

Confusing error message when master is not yet selected #476

shtratos opened this issue Apr 4, 2018 · 0 comments
Assignees
Labels
Milestone

Comments

@shtratos
Copy link
Contributor

shtratos commented Apr 4, 2018

When I try to submit a job to a fresh cluster that still has no master selected I get this error:

$ aztk spark cluster submit --id testpy --name pipy2 ~/aztk/examples/src/main/python/pi.py 100
-------------------------------------------
Spark cluster id:        testpy
Spark app name:          pipy2
Wait for app completion: True
Application:             /Users/uuu/aztk/examples/src/main/python/pi.py
Application arguments:   ['100']
-------------------------------------------
Traceback (most recent call last):
  File "/Users/uuu/.virtualenvs/aztk/bin/aztk", line 11, in <module>
    load_entry_point('aztk', 'console_scripts', 'aztk')()
  File "/Users/uuu/aztk/aztk_cli/entrypoint.py", line 35, in main
    run_software(args)
  File "/Users/uuu/aztk/aztk_cli/entrypoint.py", line 63, in run_software
    func(args)
  File "/Users/uuu/aztk/aztk_cli/spark/endpoints/spark.py", line 31, in execute
    func(args)
  File "/Users/uuu/aztk/aztk_cli/spark/endpoints/cluster/cluster.py", line 81, in execute
    func(args)
  File "/Users/uuu/aztk/aztk_cli/spark/endpoints/cluster/cluster_submit.py", line 139, in execute
    wait=False
  File "/Users/uuu/aztk/aztk/spark/client.py", line 92, in submit
    cluster_submit_helper.submit_application(self, cluster_id, application, wait)
  File "/Users/uuu/aztk/aztk/spark/helpers/submit.py", line 117, in submit_application
    task = affinitize_task_to_master(spark_client, cluster_id, task)
  File "/Users/uuu/aztk/aztk/spark/helpers/submit.py", line 107, in affinitize_task_to_master
    master_node = spark_client.batch_client.compute_node.get(pool_id=cluster_id, node_id=cluster.master_node_id)
  File "/Users/uuu/.virtualenvs/aztk/lib/python3.6/site-packages/azure/batch/operations/compute_node_operations.py", line 368, in get
    'nodeId': self._serialize.url("node_id", node_id, 'str')
  File "/Users/uuu/.virtualenvs/aztk/lib/python3.6/site-packages/msrest/serialization.py", line 470, in url
    data = self.validate(data, name, required=True, **kwargs)
  File "/Users/uuu/.virtualenvs/aztk/lib/python3.6/site-packages/msrest/serialization.py", line 540, in validate
    raise ValidationError("required", name, True)
msrest.exceptions.ValidationError: Parameter 'node_id' can not be None.

It was quite hard to understand at first that this means "master is not selected yet"

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants