MemoryCache, DistributedCache and HybridCache

The latest preview (.NET 9 preview 4) brought another caching structure to the .NET world - so let's order some things here.

What is a MemoryCache?

The MemoryCache is a datastructure that allows you, well, to cache objects in memory. It is a simple key-value store. A more in detailed blog post can be found here. "Caching in .NET with MemoryCache".

Here a simple example:

public async Task<IActionResult> GetBlogPost(int id)
{
    // Cache key
    var cacheKey = $"BlogPost_{id}";

    // Check if the cache contains the blog post
    if (!_memoryCache.TryGetValue(cacheKey, out BlogPost blogPost))
    {
        // Retrieve the blog post from the repository
        blogPost = await _blogRepository.GetBlogPostByIdAsync(id);

        // Save the blog post in the cache
        _memoryCache.Set(cacheKey, blogPost);
    }

    return Ok(blogPost);
}

This can be simplified to:

public async Task<IActionResult> GetBlogPost(int id)
{
    // Cache key
    var cacheKey = $"BlogPost_{id}";

    var blogPost = await _memoryCache.GetOrCreateAsync(cacheKey, async entry =>
    {
        return await _blogRepository.GetBlogPostByIdAsync(id);
    });

    return Ok(blogPost);
}

Of course there is a whole lot more to it - you can define how long your cache entry lives, with a multitude of strategies. A oversimplied version of the MemoryCache would be: ConcurrentDictionary<string, object>. Keep that in mind!

DistributedCache

The IDistributedCache is generally taken to communicate between multiple services (so multiple instances of an ASP.NET Web API Backend) and/or if you need to persist the data over the lifetime of your application (so after you shutdown and restarted your server). A famous example would be: Redis. So if you register something like this in your code:

builder.Services.AddStackExchangeRedisCache(...);

You register an implementation of IDistributedCache into your application. But you are also free to choose a database as your cache provider. For example an SqlServerCache:

builder.Services.AddDistributedSqlServerCache(options =>
{
    options.ConnectionString = builder.Configuration.GetConnectionString(
        "DistCache_ConnectionString");
    options.SchemaName = "dbo";
    options.TableName = "TestCache";
});

Okay - so far we know that the IMemoryCache lives and dies with the host application lifecycle (aka your app) and isn't well suited for distributed or load-balanced scenarios. Another fundemental difference between those two:

• IMemoryCache - we will persist the live object in the same scope as the host lifetime!
• IDistributedCache - we will have to serialize and deserialize the object (can be off box)!

That might make some scenarios difficult where you have non-serializble objects or objects that are expensive to serialize.

Now, look at the following code:

public class SomeService(IDistributedCache cache)
{
    public async Task<SomeInformation> GetSomeInformationAsync(string name, int id, CancellationToken token = default)
    {
        var key = $"someinfo:{name}:{id}"; // unique key for this combination
        var bytes = await cache.GetAsync(key, token); // try to get from cache
        SomeInformation info;
        if (bytes is null)
        {
            // cache miss; get the data from the real source
            info = await SomeExpensiveOperationAsync(name, id, token);

            // serialize and cache it
            bytes = SomeSerializer.Serialize(info);
            await cache.SetAsync(key, bytes, token);
        }
        else
        {
            // cache hit; deserialize it
            info = SomeSerializer.Deserialize<SomeInformation>(bytes);
        }
        return info;
    }

    // this is the work we're trying to cache
    private async Task<SomeInformation> SomeExpensiveOperationAsync(string name, int id,
        CancellationToken token = default)
    { /* ... */ }

    // ...
}

^{Source: https://github.com/dotnet/AspNetCore.Docs/issues/32361}

The IDistributedCache doesn't have a nice API for simply putting and retrieving elements, giving you a chance to lots of things wrong. There is another problem: While SomeExpensiveOperationAsync might take some time, your GetSomeInformationAsync method is called multiple times. This can lead to Cache Stampede:

A cache stampede is a type of cascading failure that can occur when massively parallel computing systems with caching mechanisms come under a very high load. This behaviour is sometimes also called dog-piling

^{Source: https://en.wikipedia.org/wiki/Cache_stampede}

So we call SomeExpensiveOperationAsync unneccessarily often and put our system under load, while waiting for the cache to populate might be the better strategy! Therefore meet:

HybridCache

That is where the new type HybridCache will come into play. Officially introduced with .NET 9 preview - but available (thanks to netstandard2.0) even for .NET Framework 4.7.2.

The call from above can be done like this:

public class SomeService(HybridCache cache)
{
    public async Task<SomeInformation> GetSomeInformationAsync(string name, int id, CancellationToken token = default)
    {
        return await cache.GetOrCreateAsync(
            $"someinfo:{name}:{id}", // unique key for this combination
            async cancel => await SomeExpensiveOperationAsync(name, id, cancel),
            token: token
        );
    }
}

That looks very much like the API of the IMemoryCache but behaves like the IDistrubtedCache "under the hood". Here are some other features, highlighted in the introduction:

As you might expect for parity with IDistributedCache, HybridCache supports explicit removal by key (cache.RemoveKeyAsync(...)). HybridCache also introduces new optional APIs for IDistributedCache implementations, to avoid byte[] allocations (this feature is implemented by the preview versions of Microsoft.Extensions.Caching.StackExchangeRedis and Microsoft.Extensions.Caching.SqlServer).

Serialization is configured as part of registering the service, with support for type-specific and generalized serializers via the .WithSerializer(...) and .WithSerializerFactory(...) methods, chained from the AddHybridCache(...) call. By default, the library handles string and byte[] internally, and uses System.Text.Json for everything else, but if you want to use protobuf, xml, or anything else: that's easy to do.

^{Source: https://github.com/dotnet/AspNetCore.Docs/issues/32361#issuecomment-2070480937}

A very nice addition to reduce potential issues one can have with IDistributedCache.