New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch timezone singletons and caches with weak references #635

Closed
pganssle opened this Issue Mar 12, 2018 · 1 comment

Comments

Projects
2 participants
@pganssle
Member

pganssle commented Mar 12, 2018

Right now tzoffset, tzstr and tz.gettz each retain strong references to every unique time zone ever constructed by that method. I think in the end this will be a net savings in memory in most use cases since it prevents duplication of time zones, but it could also cause a huge proliferation of memory in applications that make short-lived references to a large number of time zones (since those time zones, when they go out of scope, would normally get garbage collected).

In Python 3 there is the weakref module, which allows the creation of weak references, such that an object will be garbage collected if only weak references to it exist.

Rather than try and use backports.weakref in Python 2, I think we can just write a "pass through" compatibility shim that has the same external form as weakref but simply holds a strong reference to the object in question.

@cs-cordero

This comment has been minimized.

Contributor

cs-cordero commented Apr 12, 2018

I'd be interested in working on this.

@pganssle pganssle added this to Claimed in PyData NYC Sprint Apr 12, 2018

PyData NYC Sprint automation moved this from Claimed to Done Apr 14, 2018

@pganssle pganssle modified the milestones: Bugfix release, 2.7.3, 2.8.0 Apr 23, 2018

gokcennurlu added a commit to gokcennurlu/dateutil that referenced this issue Jun 14, 2018

Add LRU caching to tzoffset, tzstr and gettz
Caching had been switched to use `weakrefs` in order to reuse instances if they
are still alive, by dateutil#635. This introduces a LRU cache to the mentioned functions
in order to prevent the instances created by them getting dealloc'd and alloc'd
unnecessarily, in situations like this:

```python
for i in range(100):
    tz.gettz('America/New_York')
```

`tz.tzoffset` and `tz.tzstr` get a LRU cache with size of 8.
`tz.gettz`'s cache starts with 8 by default and can be modified by the
introduced `tz.set_cache_size(int)`

Closes dateutil#691

pganssle added a commit to gokcennurlu/dateutil that referenced this issue Sep 24, 2018

Add LRU caching to tzoffset, tzstr and gettz
Caching had been switched to use `weakrefs` in order to reuse instances if they
are still alive, by dateutil#635. This introduces a LRU cache to the mentioned functions
in order to prevent the instances created by them getting dealloc'd and alloc'd
unnecessarily, in situations like this:

```python
for i in range(100):
    tz.gettz('America/New_York')
```

`tz.tzoffset` and `tz.tzstr` get a LRU cache with size of 8.
`tz.gettz`'s cache starts with 8 by default and can be modified by the
introduced `tz.set_cache_size(int)`

Closes dateutil#691

pganssle added a commit to gokcennurlu/dateutil that referenced this issue Oct 16, 2018

Add LRU caching to tzoffset, tzstr and gettz
Caching had been switched to use `weakrefs` in order to reuse instances if they
are still alive, by dateutil#635. This introduces a LRU cache to the mentioned functions
in order to prevent the instances created by them getting dealloc'd and alloc'd
unnecessarily, in situations like this:

```python
for i in range(100):
    tz.gettz('America/New_York')
```

`tz.tzoffset` and `tz.tzstr` get a LRU cache with size of 8.
`tz.gettz`'s cache starts with 8 by default and can be modified by the
introduced `tz.set_cache_size(int)`

Closes dateutil#691

pganssle added a commit to gokcennurlu/dateutil that referenced this issue Oct 16, 2018

Add LRU caching to tzoffset, tzstr and gettz
Caching had been switched to use `weakrefs` in order to reuse instances if they
are still alive, by dateutil#635. This introduces a LRU cache to the mentioned functions
in order to prevent the instances created by them getting dealloc'd and alloc'd
unnecessarily, in situations like this:

```python
for i in range(100):
    tz.gettz('America/New_York')
```

`tz.tzoffset` and `tz.tzstr` get a LRU cache with size of 8.
`tz.gettz`'s cache starts with 8 by default and can be modified by the
introduced `tz.set_cache_size(int)`

Closes dateutil#691
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment