[python] Avoid creating unused ThreadPools (OpenAPITools#1387)

* Avoid creating unused ThreadPools Instead, create ApiClient.pool on first request for .pool property. avoids spawning n-cpus threads (the default for ThreadPool) at instantiation of every ApiClient * update doc * set pool_thread to None
ihsmarkitoss · Nov 8, 2018 · 3fe303b · 3fe303b
1 parent b163af5
commit 3fe303b
Show file tree

Hide file tree

Showing 16 changed files with 522 additions and 19 deletions.
diff --git a/modules/openapi-generator/src/main/resources/python/api_client.mustache b/modules/openapi-generator/src/main/resources/python/api_client.mustache
@@ -40,6 +40,8 @@ class ApiClient(object):
         the API.
     :param cookie: a cookie to include in the header when making calls
         to the API
+    :param pool_threads: The number of threads to use for async requests
+        to the API. More threads means more concurrent API requests.
     """
 
     PRIMITIVE_TYPES = (float, bool, bytes, six.text_type) + six.integer_types
@@ -53,14 +55,15 @@ class ApiClient(object):
         'datetime': datetime.datetime,
         'object': object,
     }
+    _pool = None
 
     def __init__(self, configuration=None, header_name=None, header_value=None,
-                 cookie=None):
+                 cookie=None, pool_threads=None):
         if configuration is None:
             configuration = Configuration()
         self.configuration = configuration
+        self.pool_threads = pool_threads
 
-        self.pool = ThreadPool()
         self.rest_client = rest.RESTClientObject(configuration)
         self.default_headers = {}
         if header_name is not None:
@@ -70,8 +73,19 @@ class ApiClient(object):
         self.user_agent = '{{#httpUserAgent}}{{{.}}}{{/httpUserAgent}}{{^httpUserAgent}}OpenAPI-Generator/{{{packageVersion}}}/python{{/httpUserAgent}}'
 
     def __del__(self):
-        self.pool.close()
-        self.pool.join()
+        if self._pool:
+            self._pool.close()
+            self._pool.join()
+            self._pool = None
+
+    @property
+    def pool(self):
+        """Create thread pool on first request
+         avoids instantiating unused threadpool for blocking clients.
+        """
+        if self._pool is None:
+            self._pool = ThreadPool(self.pool_threads)
+        return self._pool
 
     @property
     def user_agent(self):

diff --git a/samples/client/petstore/python-asyncio/.openapi-generator/VERSION b/samples/client/petstore/python-asyncio/.openapi-generator/VERSION
@@ -1 +1 @@
-3.3.2-SNAPSHOT
+3.3.3-SNAPSHOT
diff --git a/samples/client/petstore/python-asyncio/README.md b/samples/client/petstore/python-asyncio/README.md
@@ -80,6 +80,7 @@ Class | Method | HTTP request | Description
 *FakeApi* | [**test_client_model**](docs/FakeApi.md#test_client_model) | **PATCH** /fake | To test \&quot;client\&quot; model
 *FakeApi* | [**test_endpoint_parameters**](docs/FakeApi.md#test_endpoint_parameters) | **POST** /fake | Fake endpoint for testing various parameters 假端點 偽のエンドポイント 가짜 엔드 포인트 
 *FakeApi* | [**test_enum_parameters**](docs/FakeApi.md#test_enum_parameters) | **GET** /fake | To test enum parameters
+*FakeApi* | [**test_group_parameters**](docs/FakeApi.md#test_group_parameters) | **DELETE** /fake | Fake endpoint to test group parameters (optional)
 *FakeApi* | [**test_inline_additional_properties**](docs/FakeApi.md#test_inline_additional_properties) | **POST** /fake/inline-additionalProperties | test inline additionalProperties
 *FakeApi* | [**test_json_form_data**](docs/FakeApi.md#test_json_form_data) | **GET** /fake/jsonFormData | test json serialization of form data
 *FakeClassnameTags123Api* | [**test_classname**](docs/FakeClassnameTags123Api.md#test_classname) | **PATCH** /fake_classname_test | To test class name in snake case

diff --git a/samples/client/petstore/python-asyncio/docs/FakeApi.md b/samples/client/petstore/python-asyncio/docs/FakeApi.md
@@ -13,6 +13,7 @@ Method | HTTP request | Description
 [**test_client_model**](FakeApi.md#test_client_model) | **PATCH** /fake | To test \&quot;client\&quot; model
 [**test_endpoint_parameters**](FakeApi.md#test_endpoint_parameters) | **POST** /fake | Fake endpoint for testing various parameters 假端點 偽のエンドポイント 가짜 엔드 포인트 
 [**test_enum_parameters**](FakeApi.md#test_enum_parameters) | **GET** /fake | To test enum parameters
+[**test_group_parameters**](FakeApi.md#test_group_parameters) | **DELETE** /fake | Fake endpoint to test group parameters (optional)
 [**test_inline_additional_properties**](FakeApi.md#test_inline_additional_properties) | **POST** /fake/inline-additionalProperties | test inline additionalProperties
 [**test_json_form_data**](FakeApi.md#test_json_form_data) | **GET** /fake/jsonFormData | test json serialization of form data
 
@@ -486,6 +487,57 @@ No authorization required
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
+# **test_group_parameters**
+> test_group_parameters(string_group=string_group, boolean_group=boolean_group, int64_group=int64_group)
+
+Fake endpoint to test group parameters (optional)
+
+Fake endpoint to test group parameters (optional)
+
+### Example
+```python
+from __future__ import print_function
+import time
+import petstore_api
+from petstore_api.rest import ApiException
+from pprint import pprint
+
+# create an instance of the API class
+api_instance = petstore_api.FakeApi()
+string_group = 56 # int | String in group parameters (optional)
+boolean_group = True # bool | Boolean in group parameters (optional)
+int64_group = 56 # int | Integer in group parameters (optional)
+
+try:
+    # Fake endpoint to test group parameters (optional)
+    api_instance.test_group_parameters(string_group=string_group, boolean_group=boolean_group, int64_group=int64_group)
+except ApiException as e:
+    print("Exception when calling FakeApi->test_group_parameters: %s\n" % e)
+```
+
+### Parameters
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **string_group** | **int**| String in group parameters | [optional] 
+ **boolean_group** | **bool**| Boolean in group parameters | [optional] 
+ **int64_group** | **int**| Integer in group parameters | [optional] 
+
+### Return type
+
+void (empty response body)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: Not defined
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+
 # **test_inline_additional_properties**
 > test_inline_additional_properties(request_body)
 

diff --git a/samples/client/petstore/python-asyncio/petstore_api/api/fake_api.py b/samples/client/petstore/python-asyncio/petstore_api/api/fake_api.py
@@ -1009,6 +1009,102 @@ def test_enum_parameters_with_http_info(self, **kwargs):  # noqa: E501
             _request_timeout=local_var_params.get('_request_timeout'),
             collection_formats=collection_formats)
 
+    def test_group_parameters(self, **kwargs):  # noqa: E501
+        """Fake endpoint to test group parameters (optional)  # noqa: E501
+
+        Fake endpoint to test group parameters (optional)  # noqa: E501
+        This method makes a synchronous HTTP request by default. To make an
+        asynchronous HTTP request, please pass async_req=True
+        >>> thread = api.test_group_parameters(async_req=True)
+        >>> result = thread.get()
+
+        :param async_req bool
+        :param int string_group: String in group parameters
+        :param bool boolean_group: Boolean in group parameters
+        :param int int64_group: Integer in group parameters
+        :return: None
+                 If the method is called asynchronously,
+                 returns the request thread.
+        """
+        kwargs['_return_http_data_only'] = True
+        if kwargs.get('async_req'):
+            return self.test_group_parameters_with_http_info(**kwargs)  # noqa: E501
+        else:
+            (data) = self.test_group_parameters_with_http_info(**kwargs)  # noqa: E501
+            return data
+
+    def test_group_parameters_with_http_info(self, **kwargs):  # noqa: E501
+        """Fake endpoint to test group parameters (optional)  # noqa: E501
+
+        Fake endpoint to test group parameters (optional)  # noqa: E501
+        This method makes a synchronous HTTP request by default. To make an
+        asynchronous HTTP request, please pass async_req=True
+        >>> thread = api.test_group_parameters_with_http_info(async_req=True)
+        >>> result = thread.get()
+
+        :param async_req bool
+        :param int string_group: String in group parameters
+        :param bool boolean_group: Boolean in group parameters
+        :param int int64_group: Integer in group parameters
+        :return: None
+                 If the method is called asynchronously,
+                 returns the request thread.
+        """
+
+        local_var_params = locals()
+
+        all_params = ['string_group', 'boolean_group', 'int64_group']  # noqa: E501
+        all_params.append('async_req')
+        all_params.append('_return_http_data_only')
+        all_params.append('_preload_content')
+        all_params.append('_request_timeout')
+
+        for key, val in six.iteritems(local_var_params['kwargs']):
+            if key not in all_params:
+                raise TypeError(
+                    "Got an unexpected keyword argument '%s'"
+                    " to method test_group_parameters" % key
+                )
+            local_var_params[key] = val
+        del local_var_params['kwargs']
+
+        collection_formats = {}
+
+        path_params = {}
+
+        query_params = []
+        if 'string_group' in local_var_params:
+            query_params.append(('string_group', local_var_params['string_group']))  # noqa: E501
+        if 'int64_group' in local_var_params:
+            query_params.append(('int64_group', local_var_params['int64_group']))  # noqa: E501
+
+        header_params = {}
+        if 'boolean_group' in local_var_params:
+            header_params['boolean_group'] = local_var_params['boolean_group']  # noqa: E501
+
+        form_params = []
+        local_var_files = {}
+
+        body_params = None
+        # Authentication setting
+        auth_settings = []  # noqa: E501
+
+        return self.api_client.call_api(
+            '/fake', 'DELETE',
+            path_params,
+            query_params,
+            header_params,
+            body=body_params,
+            post_params=form_params,
+            files=local_var_files,
+            response_type=None,  # noqa: E501
+            auth_settings=auth_settings,
+            async_req=local_var_params.get('async_req'),
+            _return_http_data_only=local_var_params.get('_return_http_data_only'),  # noqa: E501
+            _preload_content=local_var_params.get('_preload_content', True),
+            _request_timeout=local_var_params.get('_request_timeout'),
+            collection_formats=collection_formats)
+
     def test_inline_additional_properties(self, request_body, **kwargs):  # noqa: E501
         """test inline additionalProperties  # noqa: E501
 

diff --git a/samples/client/petstore/python-asyncio/petstore_api/api_client.py b/samples/client/petstore/python-asyncio/petstore_api/api_client.py
@@ -45,6 +45,8 @@ class ApiClient(object):
         the API.
     :param cookie: a cookie to include in the header when making calls
         to the API
+    :param pool_threads: The number of threads to use for async requests
+        to the API. More threads means more concurrent API requests.
     """
 
     PRIMITIVE_TYPES = (float, bool, bytes, six.text_type) + six.integer_types
@@ -58,14 +60,15 @@ class ApiClient(object):
         'datetime': datetime.datetime,
         'object': object,
     }
+    _pool = None
 
     def __init__(self, configuration=None, header_name=None, header_value=None,
-                 cookie=None):
+                 cookie=None, pool_threads=1):
         if configuration is None:
             configuration = Configuration()
         self.configuration = configuration
+        self.pool_threads = pool_threads
 
-        self.pool = ThreadPool()
         self.rest_client = rest.RESTClientObject(configuration)
         self.default_headers = {}
         if header_name is not None:
@@ -75,8 +78,19 @@ def __init__(self, configuration=None, header_name=None, header_value=None,
         self.user_agent = 'OpenAPI-Generator/1.0.0/python'
 
     def __del__(self):
-        self.pool.close()
-        self.pool.join()
+        if self._pool:
+            self._pool.close()
+            self._pool.join()
+            self._pool = None
+
+    @property
+    def pool(self):
+        """Create thread pool on first request
+         avoids instantiating unused threadpool for blocking clients.
+        """
+        if self._pool is None:
+            self._pool = ThreadPool(self.pool_threads)
+        return self._pool
 
     @property
     def user_agent(self):

diff --git a/samples/client/petstore/python-tornado/.openapi-generator/VERSION b/samples/client/petstore/python-tornado/.openapi-generator/VERSION
@@ -1 +1 @@
-3.3.2-SNAPSHOT
+3.3.3-SNAPSHOT
diff --git a/samples/client/petstore/python-tornado/README.md b/samples/client/petstore/python-tornado/README.md
@@ -80,6 +80,7 @@ Class | Method | HTTP request | Description
 *FakeApi* | [**test_client_model**](docs/FakeApi.md#test_client_model) | **PATCH** /fake | To test \&quot;client\&quot; model
 *FakeApi* | [**test_endpoint_parameters**](docs/FakeApi.md#test_endpoint_parameters) | **POST** /fake | Fake endpoint for testing various parameters 假端點 偽のエンドポイント 가짜 엔드 포인트 
 *FakeApi* | [**test_enum_parameters**](docs/FakeApi.md#test_enum_parameters) | **GET** /fake | To test enum parameters
+*FakeApi* | [**test_group_parameters**](docs/FakeApi.md#test_group_parameters) | **DELETE** /fake | Fake endpoint to test group parameters (optional)
 *FakeApi* | [**test_inline_additional_properties**](docs/FakeApi.md#test_inline_additional_properties) | **POST** /fake/inline-additionalProperties | test inline additionalProperties
 *FakeApi* | [**test_json_form_data**](docs/FakeApi.md#test_json_form_data) | **GET** /fake/jsonFormData | test json serialization of form data
 *FakeClassnameTags123Api* | [**test_classname**](docs/FakeClassnameTags123Api.md#test_classname) | **PATCH** /fake_classname_test | To test class name in snake case

diff --git a/samples/client/petstore/python-tornado/docs/FakeApi.md b/samples/client/petstore/python-tornado/docs/FakeApi.md
@@ -13,6 +13,7 @@ Method | HTTP request | Description
 [**test_client_model**](FakeApi.md#test_client_model) | **PATCH** /fake | To test \&quot;client\&quot; model
 [**test_endpoint_parameters**](FakeApi.md#test_endpoint_parameters) | **POST** /fake | Fake endpoint for testing various parameters 假端點 偽のエンドポイント 가짜 엔드 포인트 
 [**test_enum_parameters**](FakeApi.md#test_enum_parameters) | **GET** /fake | To test enum parameters
+[**test_group_parameters**](FakeApi.md#test_group_parameters) | **DELETE** /fake | Fake endpoint to test group parameters (optional)
 [**test_inline_additional_properties**](FakeApi.md#test_inline_additional_properties) | **POST** /fake/inline-additionalProperties | test inline additionalProperties
 [**test_json_form_data**](FakeApi.md#test_json_form_data) | **GET** /fake/jsonFormData | test json serialization of form data
 
@@ -486,6 +487,57 @@ No authorization required
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
+# **test_group_parameters**
+> test_group_parameters(string_group=string_group, boolean_group=boolean_group, int64_group=int64_group)
+
+Fake endpoint to test group parameters (optional)
+
+Fake endpoint to test group parameters (optional)
+
+### Example
+```python
+from __future__ import print_function
+import time
+import petstore_api
+from petstore_api.rest import ApiException
+from pprint import pprint
+
+# create an instance of the API class
+api_instance = petstore_api.FakeApi()
+string_group = 56 # int | String in group parameters (optional)
+boolean_group = True # bool | Boolean in group parameters (optional)
+int64_group = 56 # int | Integer in group parameters (optional)
+
+try:
+    # Fake endpoint to test group parameters (optional)
+    api_instance.test_group_parameters(string_group=string_group, boolean_group=boolean_group, int64_group=int64_group)
+except ApiException as e:
+    print("Exception when calling FakeApi->test_group_parameters: %s\n" % e)
+```
+
+### Parameters
+
+Name | Type | Description  | Notes
+------------- | ------------- | ------------- | -------------
+ **string_group** | **int**| String in group parameters | [optional] 
+ **boolean_group** | **bool**| Boolean in group parameters | [optional] 
+ **int64_group** | **int**| Integer in group parameters | [optional] 
+
+### Return type
+
+void (empty response body)
+
+### Authorization
+
+No authorization required
+
+### HTTP request headers
+
+ - **Content-Type**: Not defined
+ - **Accept**: Not defined
+
+[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
+
 # **test_inline_additional_properties**
 > test_inline_additional_properties(request_body)