Skip to main content

How to benchmark different .NET versions

7/30/2023
2 minute read

With the famous BenchmarkDotNet library you can benchmark a lot - but it doesn't stop with a single .NET version. You can benchmark multiple versions of the same code that targets different runtimes!

The setup#

To run multiple versions of your code, you have to select multiple target frameworks in your csproj file:

xml
<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
        <LangVersion>preview</LangVersion>
        <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
        <TargetFrameworks>net7.0;net8.0</TargetFrameworks>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="BenchmarkDotNet" Version="0.13.6" />
    </ItemGroup>

</Project>

Then inside your BenchmarkDotNet test you can define via the SimpleJobAttribute the target runtime.

csharp
[SimpleJob(RuntimeMoniker.Net70, baseline: true)]
[SimpleJob(RuntimeMoniker.Net80)]
[MemoryDiagnoser]
public class InterpolatedEnumBenchmark
{
    [Benchmark]
    public string Interpolated()
        => $"{Weekdays.Monday} {Weekdays.Tuesday} {Weekdays.Wednesday} {Weekdays.Thursday} {Weekdays.Friday} {Weekdays.Saturday} {Weekdays.Sunday}";
}

public enum Weekdays
{
    Monday,
    Tuesday,
    Wednesday,
    Thursday,
    Friday,
    Saturday,
    Sunday
}

The result will look like this:

no-class
BenchmarkDotNet v0.13.6, macOS Ventura 13.4.1 (22F82) [Darwin 22.5.0]
Apple M1 Pro, 1 CPU, 10 logical and 10 physical cores
.NET SDK 8.0.100-preview.6.23330.14
  [Host]   : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD
  .NET 7.0 : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD
  .NET 8.0 : .NET 8.0.0 (8.0.23.32907), Arm64 RyuJIT AdvSIMD


|       Method |      Job |  Runtime |      Mean |    Error |   StdDev | Ratio |   Gen0 | Allocated | Alloc Ratio |
|------------- |--------- |--------- |----------:|---------:|---------:|------:|-------:|----------:|------------:|
| Interpolated | .NET 7.0 | .NET 7.0 | 148.66 ns | 2.008 ns | 1.780 ns |  1.00 | 0.0484 |     304 B |        1.00 |
| Interpolated | .NET 8.0 | .NET 8.0 |  61.37 ns | 0.888 ns | 0.831 ns |  0.41 | 0.0216 |     136 B |        0.45 |
An error has occurred. This application may no longer respond until reloaded.Reload x