Skip to content
Permalink
Browse files
IGNITE-14595 Implement ExpiryPolicy support - Fixes #35.
  • Loading branch information
ivandasch committed Apr 26, 2021
1 parent 8b1e61c commit 9f72781795e6fa1f427e559426ab931a49772716
Show file tree
Hide file tree
Showing 38 changed files with 1,235 additions and 659 deletions.
@@ -3,9 +3,9 @@ Apache Ignite thin (binary protocol) client, written in Python 3.

## Prerequisites

- Python 3.4 or above (3.6, 3.7 and 3.8 are tested),
- Python 3.6 or above (3.6, 3.7, 3.8 and 3.9 are tested),
- Access to Apache Ignite node, local or remote. The current thin client
version was tested on Apache Ignite 2.7.0 (binary client protocol 1.2.0).
version was tested on Apache Ignite 2.10 (binary client protocol 1.7.0).

## Installation

@@ -48,14 +48,39 @@ that yields the resulting rows.

.. literalinclude:: ../examples/async_key_value.py
:language: python
:dedent: 4
:dedent: 8
:lines: 39-50

ExpiryPolicy
============
File: `expiry_policy.py`_.

File: `async_sql.py`_.
You can enable expiry policy (TTL) by two approaches.

Firstly, expiry policy can be set for entire cache by setting :py:attr:`~pyignite.datatypes.prop_codes.PROP_EXPIRY_POLICY`
in cache settings dictionary on creation.

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 72-75

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 81-89

Secondly, expiry policy can be set for all cache operations, which are done under decorator. To create it use
:py:meth:`~pyignite.cache.BaseCache.with_expire_policy`

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 96-105

SQL
---
File: `async_sql.py`_.

First let us establish a connection.

@@ -146,6 +171,6 @@ Finally, delete the tables used in this example with the following queries:




.. _expiry_policy.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/expiry_policy.py
.. _async_key_value.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/async_key_value.py
.. _async_sql.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/async_sql.py

Large diffs are not rendered by default.

@@ -85,6 +85,33 @@ As a rule of thumb:
Refer the :ref:`data_types` section for the full list
of parser/constructor classes you can use as type hints.

ExpiryPolicy
============
File: `expiry_policy.py`_.

You can enable expiry policy (TTL) by two approaches.

Firstly, expiry policy can be set for entire cache by setting :py:attr:`~pyignite.datatypes.prop_codes.PROP_EXPIRY_POLICY`
in cache settings dictionary on creation.

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 31-34

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 40-46

Secondly, expiry policy can be set for all cache operations, which are done under decorator. To create it use
:py:meth:`~pyignite.cache.BaseCache.with_expire_policy`

.. literalinclude:: ../examples/expiry_policy.py
:language: python
:dedent: 12
:lines: 53-60

Scan
====
File: `scans.py`_.
@@ -558,13 +585,13 @@ Gather 3 Ignite nodes on `localhost` into one cluster and run:

.. literalinclude:: ../examples/failover.py
:language: python
:lines: 16-53
:lines: 16-52

Then try shutting down and restarting nodes, and see what happens.

.. literalinclude:: ../examples/failover.py
:language: python
:lines: 55-67
:lines: 54-66

Client reconnection do not require an explicit user action, like calling
a special method or resetting a parameter.
@@ -683,6 +710,7 @@ with the following message:
.. _type_hints.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/type_hints.py
.. _failover.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/failover.py
.. _scans.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/scans.py
.. _expiry_policy.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/expiry_policy.py
.. _sql.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/sql.py
.. _async_sql.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/async_sql.py
.. _binary_basics.py: https://github.com/apache/ignite-python-thin-client/blob/master/examples/binary_basics.py
@@ -0,0 +1,22 @@
.. Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
pyignite.aio_cluster module
===========================

.. automodule:: pyignite.aio_cluster
:members:
:undoc-members:
:show-inheritance:
@@ -0,0 +1,22 @@
.. Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
pyignite.cluster module
=======================

.. automodule:: pyignite.cluster
:members:
:undoc-members:
:show-inheritance:
@@ -0,0 +1,21 @@
.. Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
pyignite.datatypes.cluster_state module
=======================================

.. automodule:: pyignite.datatypes.cluster_state
:members:
:show-inheritance:
@@ -0,0 +1,21 @@
.. Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
pyignite.datatypes.expiry_policy module
=======================================

.. automodule:: pyignite.datatypes.expiry_policy
:members:
:show-inheritance:
@@ -31,6 +31,8 @@ Submodules
pyignite.datatypes.cache_config
pyignite.datatypes.cache_properties
pyignite.datatypes.complex
pyignite.datatypes.cluster_state
pyignite.datatypes.expiry_policy
pyignite.datatypes.internal
pyignite.datatypes.key_value
pyignite.datatypes.null_object
@@ -39,4 +41,3 @@ Submodules
pyignite.datatypes.primitive_objects
pyignite.datatypes.sql
pyignite.datatypes.standard

@@ -39,7 +39,8 @@ Submodules
pyignite.aio_cache
pyignite.client
pyignite.aio_client
pyignite.constants
pyignite.cluster
pyignite.aio_cluster
pyignite.cursors
pyignite.exceptions

@@ -0,0 +1,113 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
import time

from pyignite import Client, AioClient
from pyignite.datatypes import ExpiryPolicy
from pyignite.datatypes.prop_codes import PROP_NAME, PROP_EXPIRY_POLICY
from pyignite.exceptions import NotSupportedByClusterError


def main():
print("Running sync ExpiryPolicy example.")

client = Client()
with client.connect('127.0.0.1', 10800):
print("Create cache with expiry policy.")
try:
ttl_cache = client.create_cache({
PROP_NAME: 'test',
PROP_EXPIRY_POLICY: ExpiryPolicy(create=1.0)
})
except NotSupportedByClusterError:
print("'ExpiryPolicy' API is not supported by cluster. Finishing...")
return

try:
ttl_cache.put(1, 1)
time.sleep(0.5)
print(f"key = {1}, value = {ttl_cache.get(1)}")
# key = 1, value = 1
time.sleep(1.2)
print(f"key = {1}, value = {ttl_cache.get(1)}")
# key = 1, value = None
finally:
ttl_cache.destroy()

print("Create simple Cache and set TTL through `with_expire_policy`")
simple_cache = client.create_cache('test')
try:
ttl_cache = simple_cache.with_expire_policy(access=1.0)
ttl_cache.put(1, 1)
time.sleep(0.5)
print(f"key = {1}, value = {ttl_cache.get(1)}")
# key = 1, value = 1
time.sleep(1.7)
print(f"key = {1}, value = {ttl_cache.get(1)}")
# key = 1, value = None
finally:
simple_cache.destroy()


async def async_main():
print("Running async ExpiryPolicy example.")

client = AioClient()
async with client.connect('127.0.0.1', 10800):
print("Create cache with expiry policy.")
try:
ttl_cache = await client.create_cache({
PROP_NAME: 'test',
PROP_EXPIRY_POLICY: ExpiryPolicy(create=1.0)
})
except NotSupportedByClusterError:
print("'ExpiryPolicy' API is not supported by cluster. Finishing...")
return

try:
await ttl_cache.put(1, 1)
await asyncio.sleep(0.5)
value = await ttl_cache.get(1)
print(f"key = {1}, value = {value}")
# key = 1, value = 1
await asyncio.sleep(1.2)
value = await ttl_cache.get(1)
print(f"key = {1}, value = {value}")
# key = 1, value = None
finally:
await ttl_cache.destroy()

print("Create simple Cache and set TTL through `with_expire_policy`")
simple_cache = await client.create_cache('test')
try:
ttl_cache = simple_cache.with_expire_policy(access=1.0)
await ttl_cache.put(1, 1)
await asyncio.sleep(0.5)
value = await ttl_cache.get(1)
print(f"key = {1}, value = {value}")
# key = 1, value = 1
await asyncio.sleep(1.7)
value = await ttl_cache.get(1)
print(f"key = {1}, value = {value}")
# key = 1, value = None
finally:
await simple_cache.destroy()

if __name__ == '__main__':
main()

loop = asyncio.get_event_loop()
loop.run_until_complete(async_main())

0 comments on commit 9f72781

Please sign in to comment.