Safely setting keys with StackExchange.Redis while allowing deletes -

i trying use redis cache sits in front of sql database. @ high level want implement these operations:

1. read value redis, if it's not there generate value via querying sql, , push in redis don't have compute again.
2. write value redis, because made change our sql database , know might have cached , it's invalid.
3. delete value, because know value in redis stale, suspect nobody want it, it's work recompute now. we're ok letting next client operation #1 compute again.

my challenge understanding how implement #1 , #3, if attempt stackexchange.redis. if naively implement #1 simple read of key , push, it's entirely possible between me computing value sql , pushing in number of other sql operations may have happened , tried push values redis via #2 or #3. example, consider ordering:

1. client #1 wants operation #1 [read] above. tries read key, sees it's not there.
2. client #1 calls sql database generate value.
3. client #2 sql , operation #2 [write] above. pushes newly computed value redis.
4. client #3 comes long, other operation in sql, , wants operation #3 [delete] redis knowing if there's cached there, it's no longer valid.
5. client #1 pushes (now stale) value redis.

so how implement operation #1? redis offers watch primitive makes easy against bare metal able observe other things happened on key client #1, it's not supported stackexchange.redis because of how multiplexes commands. it's conditional operations aren't quite sufficient here, since if try saying "push if key doesn't exist", doesn't prevent race explained above. there pattern/best practice used here? seems common pattern people want implement.

one idea have can use separate key gets incremented each time operation on main key , can use stackexchange.redis' conditional operations way, seems kludgy.

it looks question right cache invalidation strategy rather question redis. why think - redis watch/multi kind of optimistic locking strategy , kind of locking not suitable of cases cache db read query can problem solves cache. in operation #3 description write:

it's work recompute now. we're ok letting next client operation #1 compute again.

so can continue read update case update strategy. here more questions, before continue:

1. that happens when 2 clients starts perform operation #1? both of them can not find value in redis , perform sql query , next both of write redis. should have garanties 1 client update cache?
2. how can shure in right sequence of writes (operation 3)?

why not optimistic locking

optimistic concurrency control assumes multiple transactions can complete without interfering each other. while running, transactions use data resources without acquiring locks on resources. before committing, each transaction verifies no other transaction has modified data has read. if check reveals conflicting modifications, committing transaction rolls , can restarted.

you can read occ transactions phases in wikipedia in few words:

if there no conflict - update data. if there conflict, resolve it, typically aborting transaction , restart if still need update data.

redis watch/multy kind of optimistic locking can't - not know cache key modified before try work them.

what works?

each time listen told locking - after words listen compromises, performance , consistency vs availability. last pair important.

in of high loaded system availability winner. thats means caching? usualy such case:

1. each cache key hold metadata value - state, version , life time. last 1 not redis ttl - if key should in cache x time, life time in metadata has x + y time, there y time garantie process update.
2. you never delete key directly - need update state or life time.
3. each time application read data cache if should make decision - if data has state "valid" - use it. if data has state "invalid" try update or use absolete data.

how update on read(the quite important "hand made" mix of optimistic , pessisitic locking):

1. try set pessimistic locking (in redis setex - read more here).
2. if failed - return absolete data (rememeber still need availability).
3. if success perform sql query , write in cache.
4. read version redis again , compare version readed previously.
5. if version same - mark state "valid".
6. release lock.

how invalidate (your operations #2, #3):

1. increment cache version , set state "invalid".
2. update life time/ttl if need it.

why difficult

1. we can , return value cache , have situatiuon cache miss. not have cache invalidation cascade hell many process try update 1 key.
2. we still have ordered key updates.
3. just 1 process per time can update key.

i have queue!

sorry, have not said before - not write all. if have queue becomes more simple:

1. each modification operation should push job queue.
2. only async worker should execute sql , update key.
3. you still need use "state" (valid/invalid) cache key separete application logic cache.

is answer?

actualy yes , no in same time. 1 of possible solutions. cache invalidation complex problem many possible solutions - 1 of them may simple, other - complex. in of cases depends on real bussines requirements of concrete applicaton.