MemoryCache, DistributedCache and HybridCache

20/05/2024

The latest preview (.NET 9 preview 4) brought another caching structure to the .NET world - so let's order some things here.

What is a MemoryCache?

The MemoryCache is a datastructure that allows you, well, to cache objects in memory. It is a simple key-value store. A more in detailed blog post can be found here. "Caching in .NET with MemoryCache".

Here a simple example:

public async Task<IActionResult> GetBlogPost(int id)
{
    // Cache key
    var cacheKey = $"BlogPost_{id}";

    // Check if the cache contains the blog post
    if (!_memoryCache.TryGetValue(cacheKey, out BlogPost blogPost))
    {
        // Retrieve the blog post from the repository
        blogPost = await _blogRepository.GetBlogPostByIdAsync(id);

        // Save the blog post in the cache
        _memoryCache.Set(cacheKey, blogPost);
    }

    return Ok(blogPost);
}

This can be simplified to:

public async Task<IActionResult> GetBlogPost(int id)
{
    // Cache key
    var cacheKey = $"BlogPost_{id}";

    var blogPost = await _memoryCache.GetOrCreateAsync(cacheKey, async entry =>
    {
        return await _blogRepository.GetBlogPostByIdAsync(id);
    });

    return Ok(blogPost);
}

Of course there is a whole lot more to it - you can define how long your cache entry lives, with a multitude of strategies. A oversimplied version of the MemoryCache would be: ConcurrentDictionary<string, object>. Keep that in mind!

DistributedCache

The IDistributedCache is generally taken to communicate between multiple services (so multiple instances of an ASP.NET Web API Backend) and/or if you need to persist the data over the lifetime of your application (so after you shutdown and restarted your server). A famous example would be: Redis. So if you register something like this in your code:

builder.Services.AddStackExchangeRedisCache(...);

You register an implementation of IDistributedCache into your application. But you are also free to choose a database as your cache provider. For example an SqlServerCache:

builder.Services.AddDistributedSqlServerCache(options =>
{
    options.ConnectionString = builder.Configuration.GetConnectionString(
        "DistCache_ConnectionString");
    options.SchemaName = "dbo";
    options.TableName = "TestCache";
});

Okay - so far we know that the IMemoryCache lives and dies with the host application lifecycle (aka your app) and isn't well suited for distributed or load-balanced scenarios. Another fundemental difference between those two:

  • IMemoryCache - we will persist the live object in the same scope as the host lifetime!
  • IDistributedCache - we will have to serialize and deserialize the object (can be off box)!

That might make some scenarios difficult where you have non-serializble objects or objects that are expensive to serialize.

Now, look at the following code:

public class SomeService(IDistributedCache cache)
{
    public async Task<SomeInformation> GetSomeInformationAsync(string name, int id, CancellationToken token = default)
    {
        var key = $"someinfo:{name}:{id}"; // unique key for this combination
        var bytes = await cache.GetAsync(key, token); // try to get from cache
        SomeInformation info;
        if (bytes is null)
        {
            // cache miss; get the data from the real source
            info = await SomeExpensiveOperationAsync(name, id, token);

            // serialize and cache it
            bytes = SomeSerializer.Serialize(info);
            await cache.SetAsync(key, bytes, token);
        }
        else
        {
            // cache hit; deserialize it
            info = SomeSerializer.Deserialize<SomeInformation>(bytes);
        }
        return info;
    }

    // this is the work we're trying to cache
    private async Task<SomeInformation> SomeExpensiveOperationAsync(string name, int id,
        CancellationToken token = default)
    { /* ... */ }

    // ...
}

Source: https://github.com/dotnet/AspNetCore.Docs/issues/32361

The IDistributedCache doesn't have a nice API for simply putting and retrieving elements, giving you a chance to lots of things wrong. There is another problem: While SomeExpensiveOperationAsync might take some time, your GetSomeInformationAsync method is called multiple times. This can lead to Cache Stampede:

A cache stampede is a type of cascading failure that can occur when massively parallel computing systems with caching mechanisms come under a very high load. This behaviour is sometimes also called dog-piling

Source: https://en.wikipedia.org/wiki/Cache_stampede

So we call SomeExpensiveOperationAsync unneccessarily often and put our system under load, while waiting for the cache to populate might be the better strategy! Therefore meet:

HybridCache

That is where the new type HybridCache will come into play. Officially introduced with .NET 9 preview - but available (thanks to netstandard2.0) even for .NET Framework 4.7.2.

The call from above can be done like this:

public class SomeService(HybridCache cache)
{
    public async Task<SomeInformation> GetSomeInformationAsync(string name, int id, CancellationToken token = default)
    {
        return await cache.GetOrCreateAsync(
            $"someinfo:{name}:{id}", // unique key for this combination
            async cancel => await SomeExpensiveOperationAsync(name, id, cancel),
            token: token
        );
    }
}

That looks very much like the API of the IMemoryCache but behaves like the IDistrubtedCache "under the hood". Here are some other features, highlighted in the introduction:

As you might expect for parity with IDistributedCache, HybridCache supports explicit removal by key (cache.RemoveKeyAsync(...)). HybridCache also introduces new optional APIs for IDistributedCache implementations, to avoid byte[] allocations (this feature is implemented by the preview versions of Microsoft.Extensions.Caching.StackExchangeRedis and Microsoft.Extensions.Caching.SqlServer).

Serialization is configured as part of registering the service, with support for type-specific and generalized serializers via the .WithSerializer(...) and .WithSerializerFactory(...) methods, chained from the AddHybridCache(...) call. By default, the library handles string and byte[] internally, and uses System.Text.Json for everything else, but if you want to use protobuf, xml, or anything else: that's easy to do.

Source: https://github.com/dotnet/AspNetCore.Docs/issues/32361#issuecomment-2070480937

A very nice addition to reduce potential issues one can have with IDistributedCache.

Keyed Services in the IServiceProvider in .NET 8 preview 7

The .NET 8 preview 7 will bring another exciting feature some of you probably awaiting for a long time: Keyed services.

Caching in .NET with MemoryCache

In this blog post, we will discuss how we can "cache" entries from the database. We will talk about why we would do this in the first place and how to achieve that.

Also, we will talk about some implications and what "cache invalidation" is.

ReadOnlySet<T> in .NET 9

The next preview (preview 6) will bring a new type ReadOnlySet<T>. This is a read-only set that is similar to ReadOnlyCollection<T>. Let's see how it works and why it was introduced.

An error has occurred. This application may no longer respond until reloaded. Reload x