Memcached based cooperative token is a improved version of ghetto lock,
see the description of ghetto lock at here:
https://github.com/memcached/memcached/wiki/ProgrammingTricks
It's used to avoid the thundering herd situation which many caching
users face: given a cache item that is popular and difficult to
recreate, in the event of cache misses, users could end up with hundreds
(or thousands) of processes slamming the backend database at the same
time in an attempt to refill the same cache content. This thundering
herd problem not only often leads to unresponsive backend; and also
those writes into memcached cause premature cache eviction under memory
pressure.
With cooperative token, when lots of in-flight callers try to get the
cached item specified by key from memcache and get cache misses, only
the first few query requests (limited by by ``num_tokens``) will be able
get the cooperative tokens by creating or incrementing an internal
memcache key, and then those callers with tokens can send backend
requests to fetch data from backend servers and be able to set data into
memcache; all other cache miss requests without a token should wait for
cache filling to finish, instead of all querying the backend servers at
the same time.
Co-Authored-By: Tim Burke <tim.burke@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Yan Xiao <yanxiao@nvidia.com>
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Signed-off-by: Jianjian Huo <jhuo@nvidia.com>
Change-Id: I50ff92441c2f2c49b3034644aba59930e8a99589
707a65ab3c