Notebook "Multiple_Watersheds_Simulation" fails with a "broken pipe" error #361
Comments
|
Related? bird-house/finch#98 |
tlvu
added a commit
to bird-house/birdhouse-deploy
that referenced
this issue
Apr 16, 2021
…nable in prod Value 100mb is reasonable according to @huard. Fixes Ouranosinc/raven#361. Also fixes issue describe here for Finch bird-house/finch#98 (comment). Fix this kind of error found by Jenkins: ``` _ raven-master/docs/source/notebooks/Multiple_watersheds_simulation.ipynb::Cell 1 _ Notebook cell execution failed Cell 1: Cell execution caused an exception Input: # The model parameters for gr4jcn for both catchments. Can either be a string of comma separated values, a list, an array or a named tuple. gr4jcn1 = "0.529, -3.396, 407.29, 1.072, 16.9, 0.947" gr4jcn2 = "0.28, -3.6, 380.9, 1.092, 14.6, 0.831" params = [gr4jcn1, gr4jcn2] # Forcing files. Raven uses the same forcing files for all and extracts the information it requires for each model. ts = get_file("input2d/input2d.nc") # Model configuration parameters. In a real case, we'd set nc_index to two different values for two different watersheds. config = dict( start_date=[dt.datetime(2000, 1, 1), dt.datetime(2000, 1, 1)], end_date=[dt.datetime(2002, 1, 1), dt.datetime(2002, 1, 1)], area=[4250.6, 5000], elevation=[843.0, 780], latitude=[54.4848, 48.0], longitude=[-123.3659, -122.99], nc_index=[0, 0], ) # Launch the WPS to get the multi-model results. Note the "gr4jcn" and "hmets" keys. resp = wps.raven_gr4j_cemaneige(ts=str(ts), params=params, **config) # And get the response # With `asobj` set to False, only the reference to the output is returned in the response. # Setting `asobj` to True will retrieve the actual files and copy the locally. [hydrograph, storage, solution, diagnostics, rv] = resp.get(asobj=True) Traceback: --------------------------------------------------------------------------- ServiceException Traceback (most recent call last) <ipython-input-2-37168570fca3> in <module> 20 21 # Launch the WPS to get the multi-model results. Note the "gr4jcn" and "hmets" keys. ---> 22 resp = wps.raven_gr4j_cemaneige(ts=str(ts), params=params, **config) 23 24 # And get the response </opt/conda/envs/birdy/lib/python3.7/site-packages/birdy/client/base.py-240> in raven_gr4j_cemaneige(self, ts, nc_spec, params, start_date, end_date, nc_index, duration, run_name, name, hrus, area, latitude, longitude, elevation, evaporation, rain_snow_fraction, rvc, output_formats) /opt/conda/envs/birdy/lib/python3.7/site-packages/birdy/client/base.py in _execute(self, pid, **kwargs) 345 try: 346 wps_response = self._wps.execute( --> 347 pid, inputs=wps_inputs, output=wps_outputs, mode=mode 348 ) 349 /opt/conda/envs/birdy/lib/python3.7/site-packages/owslib/wps.py in execute(self, identifier, inputs, output, mode, lineage, request, response) 357 # submit the request to the live server 358 if response is None: --> 359 response = execution.submitRequest(request) 360 else: 361 response = etree.fromstring(response) /opt/conda/envs/birdy/lib/python3.7/site-packages/owslib/wps.py in submitRequest(self, request) 910 reader = WPSExecuteReader(verbose=self.verbose, timeout=self.timeout, auth=self.auth) 911 response = reader.readFromUrl( --> 912 self.url, request, method='Post', headers=self.headers) 913 self.response = response 914 return response /opt/conda/envs/birdy/lib/python3.7/site-packages/owslib/wps.py in readFromUrl(self, url, data, method, username, password, headers, verify, cert) 601 602 return self._readFromUrl(url, data, self.timeout, method, username=username, password=password, --> 603 headers=headers, verify=verify, cert=cert) 604 605 /opt/conda/envs/birdy/lib/python3.7/site-packages/owslib/wps.py in _readFromUrl(self, url, data, timeout, method, username, password, headers, verify, cert) 513 u = openURL(url, data, method='Post', 514 username=self.auth.username, password=self.auth.password, --> 515 headers=headers, verify=self.auth.verify, cert=self.auth.cert, timeout=timeout) 516 return etree.fromstring(u.read()) 517 /opt/conda/envs/birdy/lib/python3.7/site-packages/owslib/util.py in openURL(url_base, data, method, cookies, username, password, timeout, headers, verify, cert, auth) 209 210 if req.status_code in [400, 401]: --> 211 raise ServiceException(req.text) 212 213 if req.status_code in [404, 500, 502, 503, 504]: # add more if needed ServiceException: <?xml version="1.0" encoding="utf-8"?> <ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd"> <Exception exceptionCode="NoApplicableCode" locator="NotAcceptable"> <ExceptionText>Request failed: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))</ExceptionText> </Exception> </ExceptionReport> ```
tlvu
added a commit
to bird-house/birdhouse-deploy
that referenced
this issue
Apr 16, 2021
…for-raven-demo Update Raven and Jupyter env for Raven demo Raven release notes PR Ouranosinc/raven#374 + Ouranosinc/raven#382 Jupyter env update PR Ouranosinc/PAVICS-e2e-workflow-tests#71 Other fixes: * Fix intermittent Jupyter spawning error by doubling various timeouts config (it's intermittent so hard to test so we are not sure which ones of timeout fixed it) * Fix Finch and Raven "Broken pipe" error when the request size is larger than default 3mb (bumped to 100mb) (fixes Ouranosinc/raven#361 and Finch related comment bird-house/finch#98 (comment)) * Lower chance to have "Max connection" error for Finch and Raven (bump parallelprocesses from 2 to 10). In prod, the server has the CPU needed to run 10 concurrent requests if needed so this prevent users having to "wait" after each other.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When running the second cell, I get this error:
The text was updated successfully, but these errors were encountered: