How to benchmark different .NET versions

7/30/2023

With the famous BenchmarkDotNet library you can benchmark a lot - but it doesn't stop with a single .NET version. You can benchmark multiple versions of the same code that targets different runtimes!

The setup

To run multiple versions of your code, you have to select multiple target frameworks in your csproj file:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
        <LangVersion>preview</LangVersion>
        <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
        <TargetFrameworks>net7.0;net8.0</TargetFrameworks>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="BenchmarkDotNet" Version="0.13.6" />
    </ItemGroup>

</Project>

Then inside your BenchmarkDotNet test you can define via the SimpleJobAttribute the target runtime.

[SimpleJob(RuntimeMoniker.Net70, baseline: true)]
[SimpleJob(RuntimeMoniker.Net80)]
[MemoryDiagnoser]
public class InterpolatedEnumBenchmark
{
    [Benchmark]
    public string Interpolated()
        => $"{Weekdays.Monday} {Weekdays.Tuesday} {Weekdays.Wednesday} {Weekdays.Thursday} {Weekdays.Friday} {Weekdays.Saturday} {Weekdays.Sunday}";
}

public enum Weekdays
{
    Monday,
    Tuesday,
    Wednesday,
    Thursday,
    Friday,
    Saturday,
    Sunday
}

The result will look like this:

BenchmarkDotNet v0.13.6, macOS Ventura 13.4.1 (22F82) [Darwin 22.5.0]
Apple M1 Pro, 1 CPU, 10 logical and 10 physical cores
.NET SDK 8.0.100-preview.6.23330.14
  [Host]   : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD
  .NET 7.0 : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD
  .NET 8.0 : .NET 8.0.0 (8.0.23.32907), Arm64 RyuJIT AdvSIMD


|       Method |      Job |  Runtime |      Mean |    Error |   StdDev | Ratio |   Gen0 | Allocated | Alloc Ratio |
|------------- |--------- |--------- |----------:|---------:|---------:|------:|-------:|----------:|------------:|
| Interpolated | .NET 7.0 | .NET 7.0 | 148.66 ns | 2.008 ns | 1.780 ns |  1.00 | 0.0484 |     304 B |        1.00 |
| Interpolated | .NET 8.0 | .NET 8.0 |  61.37 ns | 0.888 ns | 0.831 ns |  0.41 | 0.0216 |     136 B |        0.45 |

Logging Source Code Generators

Since .NET6 we have the possibility to define an easy way of logging common statements:

Meet Compile-time logging source generators. This article will show why we have them and how to use them. Of course a smaller benchmark will also follow.

How not to benchmark!

I came across a recent LinkedIn post about the let statement in LINQ and it's performance implication. And in typically influencer fashion it out right claimed that using let in LINQ is a bad idea and should be avoided. But is it a bad idea?

LINQ on steroids with SIMD

In this blog post, we will explore the use of SIMD instructions to speed up LINQ queries. We will use the Vector type of performing SIMD operations on arrays of data. We will also use the BenchmarkDotNet library to measure the performance of our code. We will also see how this works hand in hand with the new "generic math" feature of C# 10.

Table of Contents

The setup

Want to read more? Check out these related blog posts!

Logging Source Code Generators

How not to benchmark!

LINQ on steroids with SIMD