DiskCache API Reference ¶

The DiskCache Tutorial provides a helpful walkthrough of most methods.

Cache ¶

Read the Cache tutorial for example usage.

class diskcache.Cache(directory=None, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)¶

Disk and file backed cache.

__contains__(key)¶

Return True if key matching item is found in cache.

Parameters:: key – key matching item
Returns:: True if key matching item

__delitem__(key, retry=True)¶

Delete corresponding item for key from cache.

Raises Timeout error when database timeout occurs and retry is False (default True).

Parameters:

key – key matching item
retry (bool) – retry if database timeout occurs (default True)

Raises:

KeyError – if key is not found
Timeout – if database timeout occurs

__getitem__(key)¶

Return corresponding value for key from cache.

Parameters:: key – key matching item
Returns:: corresponding value
Raises:: KeyError – if key is not found

__getstate__()¶: Helper for pickle.

__init__(directory=None, timeout=60, disk=<class 'diskcache.core.Disk'>, **settings)¶

Initialize cache instance.

Parameters:

directory (str) – cache directory
timeout (float) – SQLite connection timeout
disk – Disk type or subclass for serialization
settings – any of DEFAULT_SETTINGS

__iter__()¶: Iterate keys in cache including expired items.

__len__()¶: Count of items in cache including expired items.

__reversed__()¶: Reverse iterate keys in cache including expired items.

__setitem__(key, value)¶

Set corresponding value for key in cache.

Parameters:

key – key for item
value – value for item

Returns:

corresponding value

Raises:

KeyError – if key is not found

add(key, value, expire=None, read=False, tag=None, retry=False)¶

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

Operation is atomic. Only one concurrent add operation for a given key will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

key – key for item
value – value for item
expire (float) – seconds until the key expires (default None, no expiry)
read (bool) – read value as bytes from file (default False)
tag (str) – text to associate with key (default None)
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if item was added

Raises:

Timeout – if database timeout occurs

check(fix=False, retry=False)¶

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the Cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

fix (bool) – correct inconsistencies
retry (bool) – retry if database timeout occurs (default False)

Returns:

list of warnings

Raises:

Timeout – if database timeout occurs

clear(retry=False)¶

Remove all items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:: retry (bool) – retry if database timeout occurs (default False)
Returns:: count of rows removed
Raises:: Timeout – if database timeout occurs

close()¶: Close database connection.

create_tag_index()¶

Create tag index on cache database.

It is better to initialize cache with tag_index=True than use this.

Raises:: Timeout – if database timeout occurs

cull(retry=False)¶

Cull items from cache until volume is less than size limit.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:: retry (bool) – retry if database timeout occurs (default False)
Returns:: count of items removed
Raises:: Timeout – if database timeout occurs

decr(key, delta=1, default=0, retry=False)¶

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

key – key for item
delta (int) – amount to decrement (default 1)
default (int) – value if key is missing (default 0)
retry (bool) – retry if database timeout occurs (default False)

Returns:

new value for item

Raises:

KeyError – if key is not found and default is None
Timeout – if database timeout occurs

delete(key, retry=False)¶

Delete corresponding item for key from cache.

Missing keys are ignored.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

key – key matching item
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if item was deleted

Raises:

Timeout – if database timeout occurs

property directory¶: Cache directory.

property disk¶: Disk used for serialization.

drop_tag_index()¶

Drop tag index on cache database.

Raises:: Timeout – if database timeout occurs

evict(tag, retry=False)¶

Remove items with matching tag from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

tag (str) – tag identifying items
retry (bool) – retry if database timeout occurs (default False)

Returns:

count of rows removed

Raises:

Timeout – if database timeout occurs

expire(now=None, retry=False)¶

Remove expired items from cache.

Removing items is an iterative process. In each iteration, a subset of items is removed. Concurrent writes may occur between iterations.

If a Timeout occurs, the first element of the exception’s args attribute will be the number of items removed before the exception occurred.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

now (float) – current time (default None, time.time() used)
retry (bool) – retry if database timeout occurs (default False)

Returns:

count of items removed

Raises:

Timeout – if database timeout occurs

get(key, default=None, read=False, expire_time=False, tag=False, retry=False)¶

Retrieve value from cache. If key is missing, return default.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

key – key for item
default – value to return if key is missing (default None)
read (bool) – if True, return file handle to value (default False)
expire_time (bool) – if True, return expire_time in tuple (default False)
tag (bool) – if True, return tag in tuple (default False)
retry (bool) – retry if database timeout occurs (default False)

Returns:

value for item or default if key not found

Raises:

Timeout – if database timeout occurs

incr(key, delta=1, default=0, retry=False)¶

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Raises Timeout error when database timeout occurs and retry is False (default).

Parameters:

key – key for item
delta (int) – amount to increment (default 1)
default (int) – value if key is missing (default 0)
retry (bool) – retry if database timeout occurs (default False)

Returns:

new value for item

Raises:

KeyError – if key is not found and default is None
Timeout – if database timeout occurs

iterkeys(reverse=False)¶

Iterate Cache keys in database sort order.

>>> cache = Cache()
>>> for key in [4, 1, 3, 0, 2]:
...     cache[key] = key
>>> list(cache.iterkeys())
[0, 1, 2, 3, 4]
>>> list(cache.iterkeys(reverse=True))
[4, 3, 2, 1, 0]

Parameters:: reverse (bool) – reverse sort order (default False)
Returns:: iterator of Cache keys

memoize(name=None, typed=False, expire=None, tag=None, ignore=())¶

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When expire is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @cache.memoize(expire=1, tag='fib')
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(cache[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @cache.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable

Parameters:

cache – cache to store callable arguments and return values
name (str) – name given for callable (default None, automatic)
typed (bool) – cache different types separately (default False)
expire (float) – seconds until arguments expire (default None, no expiry)
tag (str) – text to associate with arguments (default None)
ignore (set) – positional or keyword args to ignore (default ())

Returns:

callable decorator

peek(prefix=None, default=(None, None), side='front', expire_time=False, tag=False, retry=False)¶

Peek at key and value item pair from side of queue in cache.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to peeking at key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

Expired items are deleted from cache. Operation is atomic. Concurrent operations will be serialized.

Raises Timeout error when database timeout occurs and retry is False (default).

FanoutCache ¶

Read the FanoutCache tutorial for example usage.

class diskcache.FanoutCache(directory=None, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)¶

Cache that shards keys and values.

__contains__(key)¶

Return True if key matching item is found in cache.

Parameters:: key – key for item
Returns:: True if key is found

__delitem__(key)¶

Delete corresponding item for key from cache.

Calls FanoutCache.delete() internally with retry set to True.

Parameters:: key – key for item
Raises:: KeyError – if key is not found

__getitem__(key)¶

Return corresponding value for key from cache.

Calls FanoutCache.get() internally with retry set to True.

Parameters:: key – key for item
Returns:: value for item
Raises:: KeyError – if key is not found

__getstate__()¶: Helper for pickle.

__init__(directory=None, shards=8, timeout=0.01, disk=<class 'diskcache.core.Disk'>, **settings)¶

Initialize cache instance.

Parameters:

directory (str) – cache directory
shards (int) – number of shards to distribute writes
timeout (float) – SQLite connection timeout
disk – Disk instance for serialization
settings – any of DEFAULT_SETTINGS

__iter__()¶: Iterate keys in cache including expired items.

__len__()¶: Count of items in cache including expired items.

__reversed__()¶: Reverse iterate keys in cache including expired items.

__setitem__(key, value)¶

Set key and value item in cache.

Calls FanoutCache.set() internally with retry set to True.

Parameters:

key – key for item
value – value for item

add(key, value, expire=None, read=False, tag=None, retry=False)¶

Add key and value item to cache.

Similar to set, but only add to cache if key not present.

This operation is atomic. Only one concurrent add operation for given key from separate threads or processes will succeed.

When read is True, value should be a file-like object opened for reading in binary mode.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
value – value for item
expire (float) – seconds until the key expires (default None, no expiry)
read (bool) – read value as bytes from file (default False)
tag (str) – text to associate with key (default None)
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if item was added

cache(name, timeout=60, disk=None, **settings)¶

Return Cache with given name in subdirectory.

If disk is none (default), uses the fanout cache disk.

>>> fanout_cache = FanoutCache()
>>> cache = fanout_cache.cache('test')
>>> cache.set('abc', 123)
True
>>> cache.get('abc')
123
>>> len(cache)
1
>>> cache.delete('abc')
True

Parameters:

name (str) – subdirectory name for Cache
timeout (float) – SQLite connection timeout
disk – Disk type or subclass for serialization
settings – any of DEFAULT_SETTINGS

Returns:

Cache with given name

check(fix=False, retry=False)¶

Check database and file system consistency.

Intended for use in testing and post-mortem error analysis.

While checking the cache table for consistency, a writer lock is held on the database. The lock blocks other cache clients from writing to the database. For caches with many file references, the lock may be held for a long time. For example, local benchmarking shows that a cache with 1,000 file references takes ~60ms to check.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

fix (bool) – correct inconsistencies
retry (bool) – retry if database timeout occurs (default False)

Returns:

list of warnings

Raises:

Timeout – if database timeout occurs

clear(retry=False)¶

Remove all items from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:: retry (bool) – retry if database timeout occurs (default False)
Returns:: count of items removed

close()¶: Close database connection.

create_tag_index()¶

Create tag index on cache database.

Better to initialize cache with tag_index=True than use this.

Raises:: Timeout – if database timeout occurs

cull(retry=False)¶

Cull items from cache until volume is less than size limit.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:: retry (bool) – retry if database timeout occurs (default False)
Returns:: count of items removed

decr(key, delta=1, default=0, retry=False)¶

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
delta (int) – amount to decrement (default 1)
default (int) – value if key is missing (default 0)
retry (bool) – retry if database timeout occurs (default False)

Returns:

new value for item on success else None

Raises:

KeyError – if key is not found and default is None

delete(key, retry=False)¶

Delete corresponding item for key from cache.

Missing keys are ignored.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if item was deleted

deque(name, maxlen=None)¶

Return Deque with given name in subdirectory.

>>> cache = FanoutCache()
>>> deque = cache.deque('test')
>>> deque.extend('abc')
>>> deque.popleft()
'a'
>>> deque.pop()
'c'
>>> len(deque)
1

Parameters:

name (str) – subdirectory name for Deque
maxlen – max length (default None, no max)

Returns:

Deque with given name

property directory¶: Cache directory.

drop_tag_index()¶

Drop tag index on cache database.

Raises:: Timeout – if database timeout occurs

evict(tag, retry=False)¶

Remove items with matching tag from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

tag (str) – tag identifying items
retry (bool) – retry if database timeout occurs (default False)

Returns:

count of items removed

expire(retry=False)¶

Remove expired items from cache.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:: retry (bool) – retry if database timeout occurs (default False)
Returns:: count of items removed

get(key, default=None, read=False, expire_time=False, tag=False, retry=False)¶

Retrieve value from cache. If key is missing, return default.

If database timeout occurs then returns default unless retry is set to True (default False).

Parameters:

key – key for item
default – return value if key is missing (default None)
read (bool) – if True, return file handle to value (default False)
expire_time (float) – if True, return expire_time in tuple (default False)
tag – if True, return tag in tuple (default False)
retry (bool) – retry if database timeout occurs (default False)

Returns:

value for item if key is found else default

incr(key, delta=1, default=0, retry=False)¶

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
delta (int) – amount to increment (default 1)
default (int) – value if key is missing (default 0)
retry (bool) – retry if database timeout occurs (default False)

Returns:

new value for item on success else None

Raises:

KeyError – if key is not found and default is None

index(name)¶

Return Index with given name in subdirectory.

>>> cache = FanoutCache()
>>> index = cache.index('test')
>>> index['abc'] = 123
>>> index['def'] = 456
>>> index['ghi'] = 789
>>> index.popitem()
('ghi', 789)
>>> del index['abc']
>>> len(index)
1
>>> index['def']
456

Parameters:: name (str) – subdirectory name for Index
Returns:: Index with given name

memoize(name=None, typed=False, expire=None, tag=None, ignore=())¶

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When expire is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @cache.memoize(expire=1, tag='fib')
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(cache[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @cache.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable

Parameters:

cache – cache to store callable arguments and return values
name (str) – name given for callable (default None, automatic)
typed (bool) – cache different types separately (default False)
expire (float) – seconds until arguments expire (default None, no expiry)
tag (str) – text to associate with arguments (default None)
ignore (set) – positional or keyword args to ignore (default ())

Returns:

callable decorator

pop(key, default=None, expire_time=False, tag=False, retry=False)¶

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
default – return value if key is missing (default None)
expire_time (float) – if True, return expire_time in tuple (default False)
tag – if True, return tag in tuple (default False)
retry (bool) – retry if database timeout occurs (default False)

Returns:

value for item if key is found else default

read(key)¶

Return file handle corresponding to key from cache.

Parameters:: key – key for item
Returns:: file open for reading in binary mode
Raises:: KeyError – if key is not found

reset(key, value=ENOVAL)¶

Reset key and value item from Settings table.

If value is not given, it is reloaded from the Settings table. Otherwise, the Settings table is updated.

Settings attributes on cache objects are lazy-loaded and read-only. Use reset to update the value.

Settings with the sqlite_ prefix correspond to SQLite pragmas. Updating the value will execute the corresponding PRAGMA statement.

Parameters:

key (str) – Settings key for item
value – value for item (optional)

Returns:

updated value for item

set(key, value, expire=None, read=False, tag=None, retry=False)¶

Set key and value item in cache.

When read is True, value should be a file-like object opened for reading in binary mode.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
value – value for item
expire (float) – seconds until the key expires (default None, no expiry)
read (bool) – read value as raw bytes from file (default False)
tag (str) – text to associate with key (default None)
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if item was set

stats(enable=True, reset=False)¶

Return cache statistics hits and misses.

Parameters:

enable (bool) – enable collecting statistics (default True)
reset (bool) – reset hits and misses to 0 (default False)

Returns:

(hits, misses)

touch(key, expire=None, retry=False)¶

Touch key in cache and update expire time.

If database timeout occurs then fails silently unless retry is set to True (default False).

Parameters:

key – key for item
expire (float) – seconds until the key expires (default None, no expiry)
retry (bool) – retry if database timeout occurs (default False)

Returns:

True if key was touched

transact(retry=True)¶

Context manager to perform a transaction by locking the cache.

While the cache is locked, no other write operation is permitted. Transactions should therefore be as short as possible. Read and write operations performed in a transaction are atomic. Read operations may occur concurrent to a transaction.

Transactions may be nested and may not be shared between threads.

Blocks until transactions are held on all cache shards by retrying as necessary.

>>> cache = FanoutCache()
>>> with cache.transact():  # Atomically increment two keys.
...     _ = cache.incr('total', 123.4)
...     _ = cache.incr('count', 1)
>>> with cache.transact():  # Atomically calculate average.
...     average = cache['total'] / cache['count']
>>> average
123.4

Returns:: context manager for use in with statement

volume()¶

Return estimated total size of cache on disk.

Returns:: size in bytes

DjangoCache ¶

Read the DjangoCache tutorial for example usage.

class diskcache.DjangoCache(directory, params)¶

Django-compatible disk and file backed cache.

__init__(directory, params)¶

Initialize DjangoCache instance.

Parameters:

directory (str) – cache directory
params (dict) – cache parameters

add(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)¶

Set a value in the cache if the key does not already exist. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Return True if the value was stored, False otherwise.

Parameters:

key – key for item
value – value for item
timeout (float) – seconds until the item expires (default 300 seconds)
version (int) – key version number (default None, cache parameter)
read (bool) – read value as bytes from file (default False)
tag (str) – text to associate with key (default None)
retry (bool) – retry if database timeout occurs (default True)

Returns:

True if item was added

cache(name)¶

Return Cache with given name in subdirectory.

Parameters:: name (str) – subdirectory name for Cache
Returns:: Cache with given name

clear()¶: Remove all values from the cache at once.

close(**kwargs)¶: Close the cache connection.

create_tag_index()¶

Create tag index on cache database.

Better to initialize cache with tag_index=True than use this.

Raises:: Timeout – if database timeout occurs

cull()¶

Cull items from cache until volume is less than size limit.

Returns:: count of items removed

decr(key, delta=1, version=None, default=None, retry=True)¶

Decrement value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent decrement operations will be counted individually.

Unlike Memcached, negative values are supported. Value may be decremented below zero.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:

key – key for item
delta (int) – amount to decrement (default 1)
version (int) – key version number (default None, cache parameter)
default (int) – value if key is missing (default None)
retry (bool) – retry if database timeout occurs (default True)

Returns:

new value for item on success else None

Raises:

ValueError – if key is not found and default is None

delete(key, version=None, retry=True)¶

Delete a key from the cache, failing silently.

Parameters:

key – key for item
version (int) – key version number (default None, cache parameter)
retry (bool) – retry if database timeout occurs (default True)

Returns:

True if item was deleted

deque(name, maxlen=None)¶

Return Deque with given name in subdirectory.

Parameters:

name (str) – subdirectory name for Deque
maxlen – max length (default None, no max)

Returns:

Deque with given name

property directory¶: Cache directory.

drop_tag_index()¶

Drop tag index on cache database.

Raises:: Timeout – if database timeout occurs

evict(tag)¶

Remove items with matching tag from cache.

Parameters:: tag (str) – tag identifying items
Returns:: count of items removed

expire()¶

Remove expired items from cache.

Returns:: count of items removed

get(key, default=None, version=None, read=False, expire_time=False, tag=False, retry=False)¶

Fetch a given key from the cache. If the key does not exist, return default, which itself defaults to None.

Parameters:

key – key for item
default – return value if key is missing (default None)
version (int) – key version number (default None, cache parameter)
read (bool) – if True, return file handle to value (default False)
expire_time (float) – if True, return expire_time in tuple (default False)
tag – if True, return tag in tuple (default False)
retry (bool) – retry if database timeout occurs (default False)

Returns:

value for item if key is found else default

get_backend_timeout(timeout=<object object>)¶

Return seconds to expiration.

Parameters:: timeout (float) – seconds until the item expires (default 300 seconds)

has_key(key, version=None)¶

Returns True if the key is in the cache and has not expired.

Parameters:

key – key for item
version (int) – key version number (default None, cache parameter)

Returns:

True if key is found

incr(key, delta=1, version=None, default=None, retry=True)¶

Increment value by delta for item with key.

If key is missing and default is None then raise KeyError. Else if key is missing and default is not None then use default for value.

Operation is atomic. All concurrent increment operations will be counted individually.

Assumes value may be stored in a SQLite column. Most builds that target machines with 64-bit pointer widths will support 64-bit signed integers.

Parameters:

key – key for item
delta (int) – amount to increment (default 1)
version (int) – key version number (default None, cache parameter)
default (int) – value if key is missing (default None)
retry (bool) – retry if database timeout occurs (default True)

Returns:

new value for item on success else None

Raises:

ValueError – if key is not found and default is None

index(name)¶

Return Index with given name in subdirectory.

Parameters:: name (str) – subdirectory name for Index
Returns:: Index with given name

memoize(name=None, timeout=<object object>, version=None, typed=False, tag=None, ignore=())¶

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

When timeout is set to zero, function results will not be set in the cache. Cache lookups still occur, however. Read Case Study: Landing Page Caching for example usage.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur.

Parameters:

name (str) – name given for callable (default None, automatic)
timeout (float) – seconds until the item expires (default 300 seconds)
version (int) – key version number (default None, cache parameter)
typed (bool) – cache different types separately (default False)
tag (str) – text to associate with arguments (default None)
ignore (set) – positional or keyword args to ignore (default ())

Returns:

callable decorator

pop(key, default=None, version=None, expire_time=False, tag=False, retry=True)¶

Remove corresponding item for key from cache and return value.

If key is missing, return default.

Operation is atomic. Concurrent operations will be serialized.

Parameters:

key – key for item
default – return value if key is missing (default None)
version (int) – key version number (default None, cache parameter)
expire_time (float) – if True, return expire_time in tuple (default False)
tag – if True, return tag in tuple (default False)
retry (bool) – retry if database timeout occurs (default True)

Returns:

value for item if key is found else default

read(key, version=None)¶

Return file handle corresponding to key from Cache.

Parameters:

key – Python key to retrieve
version (int) – key version number (default None, cache parameter)

Returns:

file open for reading in binary mode

Raises:

KeyError – if key is not found

set(key, value, timeout=<object object>, version=None, read=False, tag=None, retry=True)¶

Set a value in the cache. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Parameters:

key – key for item
value – value for item
timeout (float) – seconds until the item expires (default 300 seconds)
version (int) – key version number (default None, cache parameter)
read (bool) – read value as bytes from file (default False)
tag (str) – text to associate with key (default None)
retry (bool) – retry if database timeout occurs (default True)

Returns:

True if item was set

stats(enable=True, reset=False)¶

Return cache statistics hits and misses.

Parameters:

enable (bool) – enable collecting statistics (default True)
reset (bool) – reset hits and misses to 0 (default False)

Returns:

(hits, misses)

touch(key, timeout=<object object>, version=None, retry=True)¶

Touch a key in the cache. If timeout is given, that timeout will be used for the key; otherwise the default cache timeout will be used.

Parameters:

key – key for item
timeout (float) – seconds until the item expires (default 300 seconds)
version (int) – key version number (default None, cache parameter)
retry (bool) – retry if database timeout occurs (default True)

Returns:

True if key was touched

Deque ¶

class diskcache.Deque(iterable=(), directory=None, maxlen=None)¶

Persistent sequence with double-ended queue semantics.

Double-ended queue is an ordered collection with optimized access at its endpoints.

Items are serialized to disk. Deque may be initialized from directory path where items are stored.

>>> deque = Deque()
>>> deque += range(5)
>>> list(deque)
[0, 1, 2, 3, 4]
>>> for value in range(5):
...     deque.appendleft(-value)
>>> len(deque)
10
>>> list(deque)
[-4, -3, -2, -1, 0, 0, 1, 2, 3, 4]
>>> deque.pop()
4
>>> deque.popleft()
-4
>>> deque.reverse()
>>> list(deque)
[3, 2, 1, 0, 0, -1, -2, -3]

__delitem__(index)¶

deque.__delitem__(index) <==> del deque[index]

Delete item in deque at index.

>>> deque = Deque()
>>> deque.extend([None] * 3)
>>> del deque[0]
>>> del deque[1]
>>> del deque[-1]
>>> len(deque)
0

Parameters:: index (int) – index of item
Raises:: IndexError – if index out of range

__eq__(that)¶: Return True if and only if deque is equal to that.

__ge__(that)¶: Return True if and only if deque is greater than or equal to that.

__getitem__(index)¶

deque.__getitem__(index) <==> deque[index]

Return corresponding item for index in deque.

See also Deque.peekleft and Deque.peek for indexing deque at index 0 or -1.

>>> deque = Deque()
>>> deque.extend('abcde')
>>> deque[1]
'b'
>>> deque[-2]
'd'

Parameters:: index (int) – index of item
Returns:: corresponding item
Raises:: IndexError – if index out of range

__getstate__()¶: Helper for pickle.

__gt__(that)¶: Return True if and only if deque is greater than that.

__hash__ = None¶

__iadd__(iterable)¶

deque.__iadd__(iterable) <==> deque += iterable

Extend back side of deque with items from iterable.

Parameters:: iterable – iterable of items to append to deque
Returns:: deque with added items

__init__(iterable=(), directory=None, maxlen=None)¶

Initialize deque instance.

If directory is None then temporary directory created. The directory will not be automatically removed.

Parameters:

iterable – iterable of items to append to deque
directory – deque directory (default None)

__iter__() <==> iter(deque)¶: Return iterator of deque from front to back.

__le__(that)¶: Return True if and only if deque is less than or equal to that.

__len__() <==> len(deque)¶: Return length of deque.

__lt__(that)¶: Return True if and only if deque is less than that.

__ne__(that)¶: Return True if and only if deque is not equal to that.

__repr__() <==> repr(deque)¶: Return string with printable representation of deque.

__reversed__() <==> reversed(deque)¶

Return iterator of deque from back to front.

>>> deque = Deque()
>>> deque.extend('abcd')
>>> iterator = reversed(deque)
>>> next(iterator)
'd'
>>> list(iterator)
['c', 'b', 'a']

__setitem__(index, value)¶

deque.__setitem__(index, value) <==> deque[index] = value

Store value in deque at index.

>>> deque = Deque()
>>> deque.extend([None] * 3)
>>> deque[0] = 'a'
>>> deque[1] = 'b'
>>> deque[-1] = 'c'
>>> ''.join(deque)
'abc'

Parameters:

index (int) – index of value
value – value to store

Raises:

IndexError – if index out of range

append(value)¶

Add value to back of deque.

>>> deque = Deque()
>>> deque.append('a')
>>> deque.append('b')
>>> deque.append('c')
>>> list(deque)
['a', 'b', 'c']

Parameters:: value – value to add to back of deque

appendleft(value)¶

Add value to front of deque.

>>> deque = Deque()
>>> deque.appendleft('a')
>>> deque.appendleft('b')
>>> deque.appendleft('c')
>>> list(deque)
['c', 'b', 'a']

Parameters:: value – value to add to front of deque

property cache¶: Cache used by deque.

clear()¶

Remove all elements from deque.

>>> deque = Deque('abc')
>>> len(deque)
3
>>> deque.clear()
>>> list(deque)
[]

copy()¶: Copy deque with same directory and max length.

count(value)¶

Return number of occurrences of value in deque.

>>> deque = Deque()
>>> deque += [num for num in range(1, 5) for _ in range(num)]
>>> deque.count(0)
0
>>> deque.count(1)
1
>>> deque.count(4)
4

Parameters:: value – value to count in deque
Returns:: count of items equal to value in deque

property directory¶: Directory path where deque is stored.

extend(iterable)¶

Extend back side of deque with values from iterable.

Parameters:: iterable – iterable of values

extendleft(iterable)¶

Extend front side of deque with value from iterable.

>>> deque = Deque()
>>> deque.extendleft('abc')
>>> list(deque)
['c', 'b', 'a']

Parameters:: iterable – iterable of values

classmethod fromcache(cache, iterable=(), maxlen=None)¶

Initialize deque using cache.

>>> cache = Cache()
>>> deque = Deque.fromcache(cache, [5, 6, 7, 8])
>>> deque.cache is cache
True
>>> len(deque)
4
>>> 7 in deque
True
>>> deque.popleft()
5

Parameters:

cache (Cache) – cache to use
iterable – iterable of items

Returns:

initialized Deque

property maxlen¶: Max length of the deque.

peek()¶

Peek at value at back of deque.

Faster than indexing deque at -1.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque.peek()
Traceback (most recent call last):
    ...
IndexError: peek from an empty deque
>>> deque += 'abc'
>>> deque.peek()
'c'

Returns:: value at back of deque
Raises:: IndexError – if deque is empty

peekleft()¶

Peek at value at front of deque.

Faster than indexing deque at 0.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque.peekleft()
Traceback (most recent call last):
    ...
IndexError: peek from an empty deque
>>> deque += 'abc'
>>> deque.peekleft()
'a'

Returns:: value at front of deque
Raises:: IndexError – if deque is empty

pop()¶

Remove and return value at back of deque.

If deque is empty then raise IndexError.

>>> deque = Deque()
>>> deque += 'ab'
>>> deque.pop()
'b'
>>> deque.pop()
'a'
>>> deque.pop()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque

Returns:: value at back of deque
Raises:: IndexError – if deque is empty

popleft()¶

Remove and return value at front of deque.

>>> deque = Deque()
>>> deque += 'ab'
>>> deque.popleft()
'a'
>>> deque.popleft()
'b'
>>> deque.popleft()
Traceback (most recent call last):
    ...
IndexError: pop from an empty deque

Returns:: value at front of deque
Raises:: IndexError – if deque is empty

remove(value)¶

Remove first occurrence of value in deque.

>>> deque = Deque()
>>> deque += 'aab'
>>> deque.remove('a')
>>> list(deque)
['a', 'b']
>>> deque.remove('b')
>>> list(deque)
['a']
>>> deque.remove('c')
Traceback (most recent call last):
    ...
ValueError: deque.remove(value): value not in deque

Parameters:: value – value to remove
Raises:: ValueError – if value not in deque

reverse()¶

Reverse deque in place.

>>> deque = Deque()
>>> deque += 'abc'
>>> deque.reverse()
>>> list(deque)
['c', 'b', 'a']

rotate(steps=1)¶

Rotate deque right by steps.

If steps is negative then rotate left.

>>> deque = Deque()
>>> deque += range(5)
>>> deque.rotate(2)
>>> list(deque)
[3, 4, 0, 1, 2]
>>> deque.rotate(-1)
>>> list(deque)
[4, 0, 1, 2, 3]

Parameters:: steps (int) – number of steps to rotate (default 1)

transact()¶

Context manager to perform a transaction by locking the deque.

While the deque is locked, no other write operation is permitted. Transactions should therefore be as short as possible. Read and write operations performed in a transaction are atomic. Read operations may occur concurrent to a transaction.

Transactions may be nested and may not be shared between threads.

>>> from diskcache import Deque
>>> deque = Deque()
>>> deque += range(5)
>>> with deque.transact():  # Atomically rotate elements.
...     value = deque.pop()
...     deque.appendleft(value)
>>> list(deque)
[4, 0, 1, 2, 3]

Returns:: context manager for use in with statement

Index ¶

class diskcache.Index(*args, **kwargs)¶

Persistent mutable mapping with insertion order iteration.

Items are serialized to disk. Index may be initialized from directory path where items are stored.

Hashing protocol is not used. Keys are looked up by their serialized format. See diskcache.Disk for details.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index['a']
1
>>> list(index)
['a', 'b', 'c']
>>> len(index)
3
>>> del index['b']
>>> index.popitem()
('c', 3)

__delitem__(key)¶

index.__delitem__(key) <==> del index[key]

Delete corresponding item for key from index.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2})
>>> del index['a']
>>> del index['b']
>>> len(index)
0
>>> del index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'

Parameters:: key – key for item
Raises:: KeyError – if key is not found

__eq__(other)¶

index.__eq__(other) <==> index == other

Compare equality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index()
>>> pairs = [('a', 1), ('b', 2), ('c', 3)]
>>> index.update(pairs)
>>> from collections import OrderedDict
>>> od = OrderedDict(pairs)
>>> index == od
True
>>> index == {'c': 3, 'b': 2, 'a': 1}
True

Parameters:: other – other mapping in equality comparison
Returns:: True if index equals other

__getitem__(key)¶

index.__getitem__(key) <==> index[key]

Return corresponding value for key in index.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2})
>>> index['a']
1
>>> index['b']
2
>>> index['c']
Traceback (most recent call last):
    ...
KeyError: 'c'

Parameters:: key – key for item
Returns:: value for item in index with given key
Raises:: KeyError – if key is not found

__getstate__()¶: Helper for pickle.

__hash__ = None¶

__init__(*args, **kwargs)¶

Initialize index in directory and update items.

Optional first argument may be string specifying directory where items are stored. When None or not given, temporary directory is created.

>>> index = Index({'a': 1, 'b': 2, 'c': 3})
>>> len(index)
3
>>> directory = index.directory
>>> inventory = Index(directory, d=4)
>>> inventory['b']
2
>>> len(inventory)
4

__iter__() <==> iter(index)¶: Return iterator of index keys in insertion order.

__len__() <==> len(index)¶: Return length of index.

__ne__(other)¶

index.__ne__(other) <==> index != other

Compare inequality for index and other.

Comparison to another index or ordered dictionary is order-sensitive. Comparison to all other mappings is order-insensitive.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> from collections import OrderedDict
>>> od = OrderedDict([('c', 3), ('b', 2), ('a', 1)])
>>> index != od
True
>>> index != {'a': 1, 'b': 2}
True

Parameters:: other – other mapping in inequality comparison
Returns:: True if index does not equal other

__repr__() <==> repr(index)¶: Return string with printable representation of index.

__reversed__() <==> reversed(index)¶

Return iterator of index keys in reversed insertion order.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> iterator = reversed(index)
>>> next(iterator)
'c'
>>> list(iterator)
['b', 'a']

__setitem__(key, value)¶

index.__setitem__(key, value) <==> index[key] = value

Set key and value item in index.

>>> index = Index()
>>> index['a'] = 1
>>> index[0] = None
>>> len(index)
2

Parameters:

key – key for item
value – value for item

property cache¶: Cache used by index.

clear()¶

Remove all items from index.

>>> index = Index({'a': 0, 'b': 1, 'c': 2})
>>> len(index)
3
>>> index.clear()
>>> dict(index)
{}

property directory¶: Directory path where items are stored.

classmethod fromcache(cache, *args, **kwargs)¶

Initialize index using cache and update items.

>>> cache = Cache()
>>> index = Index.fromcache(cache, {'a': 1, 'b': 2, 'c': 3})
>>> index.cache is cache
True
>>> len(index)
3
>>> 'b' in index
True
>>> index['c']
3

Parameters:

cache (Cache) – cache to use
args – mapping or sequence of items
kwargs – mapping of items

Returns:

initialized Index

items()¶

Set-like object providing a view of index items.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> items_view = index.items()
>>> ('b', 2) in items_view
True

Returns:: items view

keys()¶

Set-like object providing a view of index keys.

>>> index = Index()
>>> index.update({'a': 1, 'b': 2, 'c': 3})
>>> keys_view = index.keys()
>>> 'b' in keys_view
True

Returns:: keys view

memoize(name=None, typed=False, ignore=())¶

Memoizing cache decorator.

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

If name is set to None (default), the callable name will be determined automatically.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Index
>>> mapping = Index()
>>> @mapping.memoize()
... def fibonacci(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fibonacci(number - 1) + fibonacci(number - 2)
>>> print(fibonacci(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fibonacci.__cache_key__(100)
>>> print(mapping[key])
354224848179261915075

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur. Note the lack of parenthenses after memoize below:

>>> @mapping.memoize
... def test():
...     pass
Traceback (most recent call last):
    ...
TypeError: name cannot be callable

Parameters:

name (str) – name given for callable (default None, automatic)
typed (bool) – cache different types separately (default False)
ignore (set) – positional or keyword args to ignore (default ())

Returns:

callable decorator

peekitem(last=True)¶

Peek at key and value item pair in index based on iteration order.

>>> index = Index()
>>> for num, letter in enumerate('xyz'):
...     index[letter] = num
>>> index.peekitem()
('z', 2)
>>> index.peekitem(last=False)
('x', 0)

Parameters:: last (bool) – last item in iteration order (default True)
Returns:: key and value item pair
Raises:: KeyError – if cache is empty

pop(key, default=ENOVAL)¶

Remove corresponding item for key from index and return value.

If key is missing then return default. If default is ENOVAL then raise KeyError.

>>> index = Index({'a': 1, 'b': 2})
>>> index.pop('a')
1
>>> index.pop('b')
2
>>> index.pop('c', default=3)
3
>>> index.pop('d')
Traceback (most recent call last):
    ...
KeyError: 'd'

Parameters:

key – key for item
default – return value if key is missing (default ENOVAL)

Returns:

value for item if key is found else default

Raises:

KeyError – if key is not found and default is ENOVAL

popitem(last=True)¶

Remove and return item pair.

Item pairs are returned in last-in-first-out (LIFO) order if last is True else first-in-first-out (FIFO) order. LIFO order imitates a stack and FIFO order imitates a queue.

>>> index = Index()
>>> index.update([('a', 1), ('b', 2), ('c', 3)])
>>> index.popitem()
('c', 3)
>>> index.popitem(last=False)
('a', 1)
>>> index.popitem()
('b', 2)
>>> index.popitem()
Traceback (most recent call last):
  ...
KeyError: 'dictionary is empty'

Parameters:: last (bool) – pop last item pair (default True)
Returns:: key and value item pair
Raises:: KeyError – if index is empty

pull(prefix=None, default=(None, None), side='front')¶

Pull key and value item pair from side of queue in index.

When prefix is None, integer keys are used. Otherwise, string keys are used in the format “prefix-integer”. Integer starts at 500 trillion.

If queue is empty, return default.

Defaults to pulling key and value item pairs from front of queue. Set side to ‘back’ to pull from back of queue. Side must be one of ‘front’ or ‘back’.

Recipes ¶

class diskcache.Averager(cache, key, expire=None, tag=None)¶

Recipe for calculating a running average.

Sometimes known as “online statistics,” the running average maintains the total and count. The average can then be calculated at any time.

Assumes the key will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the key is not evicted.

>>> import diskcache
>>> cache = diskcache.FanoutCache()
>>> ave = Averager(cache, 'latency')
>>> ave.add(0.080)
>>> ave.add(0.120)
>>> ave.get()
0.1
>>> ave.add(0.160)
>>> ave.pop()
0.12
>>> print(ave.get())
None

add(value)¶: Add value to average.

get()¶: Get current average or return None if count equals zero.

pop()¶: Return current average and delete key.

class diskcache.Lock(cache, key, expire=None, tag=None)¶

Recipe for cross-process and cross-thread lock.

Assumes the key will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the key is not evicted.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> lock = Lock(cache, 'report-123')
>>> lock.acquire()
>>> lock.release()
>>> with lock:
...     pass

acquire()¶: Acquire lock using spin-lock algorithm.

locked()¶: Return true if the lock is acquired.

release()¶: Release lock by deleting key.

class diskcache.RLock(cache, key, expire=None, tag=None)¶

Recipe for cross-process and cross-thread re-entrant lock.

Assumes the key will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the key is not evicted.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> rlock = RLock(cache, 'user-123')
>>> rlock.acquire()
>>> rlock.acquire()
>>> rlock.release()
>>> with rlock:
...     pass
>>> rlock.release()
>>> rlock.release()
Traceback (most recent call last):
  ...
AssertionError: cannot release un-acquired lock

acquire()¶: Acquire lock by incrementing count using spin-lock algorithm.

release()¶: Release lock by decrementing count.

class diskcache.BoundedSemaphore(cache, key, value=1, expire=None, tag=None)¶

Recipe for cross-process and cross-thread bounded semaphore.

Assumes the key will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the key is not evicted.

>>> import diskcache
>>> cache = diskcache.Cache()
>>> semaphore = BoundedSemaphore(cache, 'max-cons', value=2)
>>> semaphore.acquire()
>>> semaphore.acquire()
>>> semaphore.release()
>>> with semaphore:
...     pass
>>> semaphore.release()
>>> semaphore.release()
Traceback (most recent call last):
  ...
AssertionError: cannot release un-acquired semaphore

acquire()¶: Acquire semaphore by decrementing value using spin-lock algorithm.

release()¶: Release semaphore by incrementing value.

@diskcache.throttle(cache, count, seconds, name=None, expire=None, tag=None, time_func=<built-in function time>, sleep_func=<built-in function sleep>)¶

Decorator to throttle calls to function.

Assumes keys will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the keys are not evicted.

>>> import diskcache, time
>>> cache = diskcache.Cache()
>>> count = 0
>>> @throttle(cache, 2, 1)  # 2 calls per 1 second
... def increment():
...     global count
...     count += 1
>>> start = time.time()
>>> while (time.time() - start) <= 2:
...     increment()
>>> count in (6, 7)  # 6 or 7 calls depending on CPU load
True

@diskcache.barrier(cache, lock_factory, name=None, expire=None, tag=None)¶

Barrier to calling decorated function.

Supports different kinds of locks: Lock, RLock, BoundedSemaphore.

Assumes keys will not be evicted. Set the eviction policy to ‘none’ on the cache to guarantee the keys are not evicted.

>>> import diskcache, time
>>> cache = diskcache.Cache()
>>> @barrier(cache, Lock)
... def work(num):
...     print('worker started')
...     time.sleep(1)
...     print('worker finished')
>>> import multiprocessing.pool
>>> pool = multiprocessing.pool.ThreadPool(2)
>>> _ = pool.map(work, range(2))
worker started
worker finished
worker started
worker finished
>>> pool.terminate()

@diskcache.memoize_stampede(cache, expire, name=None, typed=False, tag=None, beta=1, ignore=())¶

Memoizing cache decorator with cache stampede protection.

Cache stampedes are a type of system overload that can occur when parallel computing systems using memoization come under heavy load. This behaviour is sometimes also called dog-piling, cache miss storm, cache choking, or the thundering herd problem.

The memoization decorator implements cache stampede protection through early recomputation. Early recomputation of function results will occur probabilistically before expiration in a background thread of execution. Early probabilistic recomputation is based on research by Vattani, A.; Chierichetti, F.; Lowenstein, K. (2015), Optimal Probabilistic Cache Stampede Prevention, VLDB, pp. 886-897, ISSN 2150-8097

If name is set to None (default), the callable name will be determined automatically.

If typed is set to True, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.

The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.

>>> from diskcache import Cache
>>> cache = Cache()
>>> @memoize_stampede(cache, expire=1)
... def fib(number):
...     if number == 0:
...         return 0
...     elif number == 1:
...         return 1
...     else:
...         return fib(number - 1) + fib(number - 2)
>>> print(fib(100))
354224848179261915075

An additional __cache_key__ attribute can be used to generate the cache key used for the given arguments.

>>> key = fib.__cache_key__(100)
>>> del cache[key]

Remember to call memoize when decorating a callable. If you forget, then a TypeError will occur.

Parameters:

cache – cache to store callable arguments and return values
expire (float) – seconds until arguments expire
name (str) – name given for callable (default None, automatic)
typed (bool) – cache different types separately (default False)
tag (str) – text to associate with arguments (default None)
ignore (set) – positional or keyword args to ignore (default ())

Returns:

callable decorator

Constants ¶

Read the Settings tutorial for details.

diskcache.DEFAULT_SETTINGS¶

statistics (int) default 0 - disabled when 0, enabled when 1.
tag_index (int) default 0 - disabled when 0, enabled when 1.
eviction_policy (str) default “least-recently-stored” - any of the keys in EVICTION_POLICY as described below.
size_limit (int, in bytes) default one gigabyte - approximate size limit of cache.
cull_limit (int) default ten - maximum number of items culled during set or add operations.
sqlite_auto_vacuum (int) default 1, “FULL” - SQLite auto vacuum pragma.
sqlite_cache_size (int, in pages) default 8,192 - SQLite cache size pragma.
sqlite_journal_mode (str) default “wal” - SQLite journal mode pragma.
sqlite_mmap_size (int, in bytes) default 64 megabytes - SQLite mmap size pragma.
sqlite_synchronous (int) default 1, “NORMAL” - SQLite synchronous pragma.
disk_min_file_size (int, in bytes) default 32 kilobytes - values with greater size are stored in files.
disk_pickle_protocol (int) default highest Pickle protocol - the Pickle protocol to use for data types that are not natively supported.

diskcache.EVICTION_POLICY¶

least-recently-stored (default) - evict least recently stored keys first.
least-recently-used - evict least recently used keys first.
least-frequently-used - evict least frequently used keys first.
none - never evict keys.

Disk ¶

Read the Disk tutorial for details.

class diskcache.Disk(directory, min_file_size=0, pickle_protocol=0)¶

Cache key and value serialization for SQLite database and files.

__init__(directory, min_file_size=0, pickle_protocol=0)¶

Initialize disk instance.

Parameters:

directory (str) – directory path
min_file_size (int) – minimum size for file use
pickle_protocol (int) – pickle protocol for serialization

fetch(mode, filename, value, read)¶

Convert fields mode, filename, and value from Cache table to value.

Parameters:

mode (int) – value mode raw, binary, text, or pickle
filename (str) – filename of corresponding value
value – database value
read (bool) – when True, return an open file handle

Returns:

corresponding Python value

Raises:

IOError if the value cannot be read

filename(key=UNKNOWN, value=UNKNOWN)¶

Return filename and full-path tuple for file storage.

Filename will be a randomly generated 28 character hexadecimal string with “.val” suffixed. Two levels of sub-directories will be used to reduce the size of directories. On older filesystems, lookups in directories with many files may be slow.

The default implementation ignores the key and value parameters.

In some scenarios, for example Cache.push, the key or value may not be known when the item is stored in the cache.

Parameters:

key – key for item (default UNKNOWN)
value – value for item (default UNKNOWN)

get(key, raw)¶

Convert fields key and raw from Cache table to key.

Parameters:

key – database key to convert
raw (bool) – flag indicating raw database storage

Returns:

corresponding Python key

hash(key)¶

Compute portable hash for key.

Parameters:: key – key to hash
Returns:: hash value

put(key)¶

Convert key to fields key and raw for Cache table.

Parameters:: key – key to convert
Returns:: (database key, raw boolean) pair

remove(file_path)¶

Remove a file given by file_path.

This method is cross-thread and cross-process safe. If an OSError occurs, it is suppressed.

Parameters:: file_path (str) – relative path to file

store(value, read, key=UNKNOWN)¶

Convert value to fields size, mode, filename, and value for Cache table.

Parameters:

value – value to convert
read (bool) – True when value is file-like object
key – key for item (default UNKNOWN)

Returns:

(size, mode, filename, value) tuple for Cache table

JSONDisk ¶

Read the Disk tutorial for details.

class diskcache.JSONDisk(directory, compress_level=1, **kwargs)¶

Cache key and value using JSON serialization with zlib compression.

__init__(directory, compress_level=1, **kwargs)¶

Initialize JSON disk instance.

Keys and values are compressed using the zlib library. The compress_level is an integer from 0 to 9 controlling the level of compression; 1 is fastest and produces the least compression, 9 is slowest and produces the most compression, and 0 is no compression.

Parameters:

directory (str) – directory path
compress_level (int) – zlib compression level (default 1)
kwargs – super class arguments

fetch(mode, filename, value, read)¶

Convert fields mode, filename, and value from Cache table to value.

Parameters:

mode (int) – value mode raw, binary, text, or pickle
filename (str) – filename of corresponding value
value – database value
read (bool) – when True, return an open file handle

Returns:

corresponding Python value

Raises:

IOError if the value cannot be read

get(key, raw)¶

Convert fields key and raw from Cache table to key.

Parameters:

key – database key to convert
raw (bool) – flag indicating raw database storage

Returns:

corresponding Python key

put(key)¶

Convert key to fields key and raw for Cache table.

Parameters:: key – key to convert
Returns:: (database key, raw boolean) pair

store(value, read, key=UNKNOWN)¶

Convert value to fields size, mode, filename, and value for Cache table.

Parameters:

value – value to convert
read (bool) – True when value is file-like object
key – key for item (default UNKNOWN)

Returns:

(size, mode, filename, value) tuple for Cache table

Timeout ¶

exception diskcache.Timeout¶: Database timeout expired.

DiskCache API Reference ¶

Cache ¶

FanoutCache ¶

DjangoCache ¶

Deque ¶

Index ¶

Recipes ¶

Constants ¶

Disk ¶

JSONDisk ¶

Timeout ¶

DiskCache

Give Support

Table of Contents

Related Topics