TL;DR

Cogenity.Extensions.Hosting.Composition can provide lightweight, runtime composition for the .NET Core 3.0 Generic Host... with some caveats. An issue has been created as an RFC on how best to address these caveats with comments/contributions welcomed.

Did you know...

The .NET Core 3.0 web stack has been "re-platformed" onto the generic host library?

Furthermore, did you know that WebHost allows you to load additional non-referenced assemblies at runtime which can participate in service start-up by implementing the IHostingStartup interface?

Well neither did I until I started looking for a way to perform runtime-composition for a service utilizing the .NET Core 3.0 Generic Host. It turns out that, while both the changes above are great for the web-platform, nothing similar exists out-of-the-box for the Generic Host. So here's what I did...

Background

I'm currently working on a very interesting project which I hope to see open-sourced in a few weeks time. At the moment it's being developed behind closed doors but with various reusable components being segregated into open-source projects/packages for use by the community.

One facet of this project is a .NET Core 3.0 service which needs to be deployable in a variety of environments and across a number of platforms. While this service provides a set of core behaviours, these behaviours need to be augmented by arbitrary functionality specific to the environment/platform the service is being used within.

So it was that I came to look for a means to provide light-weight runtime-composition to services implemented using the .NET Core 3.0 Generic Host and, to my surprise, came up empty handed.

Wait, runtime composition?

Yes, the ability to add functionality/behaviours to a software component, ostensibly by loading additional assemblies at runtime, without requiring re-compilation of said software component.

Specifically, my requirements were:

A simple, fast, lightweight means to safely load and configure additional assemblies at service start-up; and
A means for these assemblies to participate in host composition (i.e. do something like 'IHostBuilder.UseRabbitMq<MyMessageHandler>()').

With a couple of nice-to-haves:

Allow for multiple instances of a specific assembly to be loaded with different configurations.
Allow multiple versions of specific assemblies to be loaded concurrently.

Surely this is a solved problem?

I thought so too. Having used MEF (the Managed Extensibility Framework) in the past I went to see if it had been ported to .NET Core and indeed it had. Unfortunately, it seems to have fallen out of favour and there's very little documentation about it's use in .NET Core and even less about how one might integrate it into the Generic Host.

I then found Scrutor which, while more integrated into the Generic Host eco-system (it provides fantastic assembly scanning and decoration capabilities specfically for Microsoft.Extensions.DependencyInjection), didn't provide a means for the discovered types to participate in host composition.

Finally I discovered Dapplo.Microsoft.Extensions.Hosting which provides the Dapplo.Microsoft.Extensions.Hosting.Plugins package. This was extremely close to what I wanted but relied too heavily on directory scanning (thereby not meeting the "fast and safe" requirements) and, like Scrutor and MEF, also didn't provide a means to participate in host composition (the "plugins" only have access to the HostBuilderContent).

Right, well how hard can this be?

Not that hard it turns out... but with several caveats.

Within a few hours of deciding to roll-my-own solution to the requirements above, and by borrowing extensively from the various projects I'd already encountered, I'd written Cogenity.Extensions.Hosting.Composition. This solution used configuration (rather than directory scanning) to specify the additional assemblies to load and, like the Dapplo project, used .NET Core's AssemblyLoadContext to provide scoping of loaded assemblies providing additional safety and reliability characteristics (in the abscence of AppDomains).

A GenericHostConsole sample project was written to show the library's use and demonstrate it's functionality which worked beautifully. From the host project, all that was required to provide runtime composition was a call to the .UseComposition extension method and some associated configuration (I decided to use a Yaml file but any configuration provider could be used) as shown below:

private static async Task Main(string[] args)
{
    var builder = Host.CreateDefaultBuilder(args)
        .ConfigureHostConfiguration(configurationBuilder => configurationBuilder.AddCommandLine(args))
        .UseComposition(config => config.AddYamlFile(args[0]), "composition");

    await builder
        .Build()
        .RunAsync();
}

composition:
  modules:
    - name: ConsoleWriter
      assembly: GenericHostConsole.Writer
      configurationSection: consolewriterConfiguration
      optional: true

consolewriterConfiguration:
  writeIntervalInSeconds: 2

Then additional assemblies could be provided (in this case GenericHostConsole.Writer) which simply needed to implement the IModule interface as shown here:

public class Module : IModule
{
    public IHostBuilder Configure(IHostBuilder hostbuilder, string configurationSection)
    {
        return hostbuilder
            .ConfigureServices(
                (hostBuilderContext, serviceCollection) =>
                {
                    serviceCollection.AddOptions<Configuration>().Bind(hostBuilderContext.Configuration.GetSection(configurationSection));
                    serviceCollection.AddSingleton<IHostedService, Service>();
                })
            .ConfigureLogging((hostingContext, logging) => logging.AddConsole());
    }
}

After compiling, copying the GenericHostConsole.Writer assembly to the GenericHostConsole project and running the latter's executable, the GenericHostConsole.Writer.Service wrote to the console every two seconds, as configured.

Boom! Done... well... sort of.

I returned to the project I required this for and followed the above pattern, expecting (ok, somewhat naively) everything to be rosy. It failed spectacularly.

You see, while my sample project for Cogenity.Extensions.Hosting.Composition, simply augmented the host by adding a new service with no other dependencies, the original project required new services to interact with core functionality provided by the project, mainly by injection of services defined in a 'common' assembly (i.e. one referenced both by the host project and the composition modules). Starting the project resulting in the DI container reporting that it couldn't locate implementations of required services despite those services being registered with the host's DI container (by throwing a System.InvalidOperationException: Unable to resolve service for type '<service>' while attempting to activate '<consumer>')).

It turned out that I was being bitten by the following:

Due to the requirement of needing to allow additional modules to participate in host composition, assemblies were being loaded at composition, not build time as shown below:

var builder = Host.CreateDefaultBuilder(args)
    .ConfigureHostConfiguration(configurationBuilder => configurationBuilder.AddCommandLine(args))
    .UseComposition(config => config.AddYamlFile(args[0]), "composition"); // <- Assemblies loaded here
    .ConfigureServices(, serviceCollection) => serviceCollection.AddSingleton<IEventBus, EventBus>())

await builder
    .Build() // <- Not here
    .RunAsync();

This meant that the AssemblyLoadContext used to isolate module assemblies was loading new instances of 'common' assemblies rather than using the ones being registered by the host service (i.e. the IEventBus in the above example), thereby explaining why the various implementations couldn't be found.

So what now?

Well, it can be used as-is for basic composition functionality, but it's really not what I need moving forward.

I have considered several possible solutions ranging from simply removing assembly isolation (and no-longer providing the 'nice-to-haves') to working more intimately with the HostBuilder.Build process to ensure shared assemblies are fully loaded prior to loading additional assemblies for composition. Each of the approaches have pros and cons which I am trying to consider from a 'general consumer' point of view before deciding on the solution to adopt.

To this end, I have created an issue in the Github repository for this project in which I describe what I see as the various ways forward and, where possible, links to branches proving out the various approaches. I've labelled it as 'discussion' and would genuinely be interested to hear people's thoughts. If this project is of interest to you and/or you have suggestions about how best to resolve the above issues, please feel free to add a comment with your suggestions and/or requests for modified/additional functionality.

If you require runtime composition for projects utilizing the .NET Core Generic Host then watch this space as I hope to have a more versatile solution in place within the next week or so.

Light-weight run-time composition for the .NET Core 3.0 Generic Host

With an RFC (Request For Comments) from the community