Schrodinger’s Cache

Generally speaking, repeating the same actions under the same conditions should create the same results (or close enough that the results aren’t a surprise, at least). This tendency is at the root of all scientific pursuits, our development of technology, our understanding of economics (such as it is), and forms a core belief in a big chunk of the population. This is especially true for software developers, because we literally rely on functions at a deep level. In fact, the mathematical definition of a function is “a relation from a set of inputs to a set of possible outputs where each input is related to exactly one output” (mathinsight.org). When examined, this statement simply says that doing the same thing should produce the same result with the same inputs, albeit with a bit more mathematical rigor. However, if you’ve ever had the experience of getting burned by caching, you are more likely to describe the experience as being more like Schrödinger’s cat. This is a concept from a thought experiment posed by Erwin Schrödinger during discussions with Albert Einstein while expressing his objections to the Copenhagen interpretation of quantum mechanics. While a full discussion of this principle is beyond the scope of this podcast, the gist of it, per Schrödinger was that if you put a cat in a box with something that would kill it (in this case, a radioactive particle), that if you couldn’t perceive what was in the box, that one could model that cat as either being “alive” or “dead”. While a somewhat gruesome notion, if you’ve ever dealt with cache invalidation of unknown length, this probably sounds familiar. So, if you suspect a cache issue, because things are behaving in a way that indicates Schrödinger’s Cache might be your issue, you probably need a reasonable way to think about your troubleshooting efforts. Troubleshooting cache issues without a plan is not only unlikely to succeed, but you may find that you have difficulty proving that you actually fixed an intermittent issue. In effect, you need to figure out how to let the cat out of the box so that it is in a provable state. Dealing with caching issues is very frustrating. One of the biggest problems is that caching problems often manifest in a very unpredictable manner, often driven by load or by user behavior. Because of this, troubleshooting caching problems often requires that you first prove that caching is a factor in the problem, before attempting to correct it. Once you have proved that the cache is involved in the problem, you can manipulate your cache in order to make the problem easier to reproduce, which will greatly aid you in troubleshooting and debugging. Episode Breakdown Basics of caching A cache is a higher speed (or lower cost) data storage layer, which stores a subset of data, typically transient in nature so that future requests for this data are faster/cheaper and create less load on the system. When a cache is in use an incoming request will be examined and a cache key generated. This key will be used to look up the result in the cache. If the result is there, it will be returned. If it isn’t, it will be retrieved, stored in cache, and then returned. Items in cache will have a cache window, that is, a period of time that will elapse before the item is removed from cache. Accesses of data subsequent to cache invalidation will cause the data to be reloaded and recached. Sometimes caches have sliding expiration, that is, the item will remain in cache for a certain amount of time since it was last accessed. This lets the cache perform better under load, while possibly reducing accuracy. Most caching systems also let you manually invalidate an item in cache. This is handy if the thing you are caching sometimes exhibits more volatility more frequent than the length of your cache window.