Previous month:
May 2018

June 2018

Entity Framework Core: Isolation of Integration Tests

When working with Entity Framework Core (EF) a lot of code can be tested using the In-Memory database provider but sometimes you want (or have) to go to the real database. For example, you are using not just LINQ but custom SQL statements due to performance reasons or you want to check that a specific exception is thrown by the database under some conditions like when having a primary key violation.

The biggest challenge of integration tests is the isolation of one test from another. In this post we will look at 3 options how to do that.

The code for the 3rd option is on GitHub: PawelGerr/Presentation-EntityFrameworkCore

Remarks: in my demos I'm using 3rd party libs: FluentAssertions and xunit.

Given is a DemoRepository with a method AddProduct that we want to test. (The code kept oversimplified for clarity reasons)

public class DemoRepository
{
...

public void AddProduct(Guid id)
{
_dbContext.Products.Add(new Product { Id = id });
_dbContext.SaveChanges();
}
}

Using Transaction Scopes

EF Core added support for TransactionScope in version 2.1.

The isolation of tests via TransactionScope is very simple just wrap the call AddProduct into a TransactionScope to revert all changes at the end of the test. But, there are few preconditions. The testing method must not starting transactions using BeginTransaction() or it has to use a TransactionScope as well.

Also, I recommend to read my other blog post: Entity Framework Core: Use TransactionScope with Caution!

public DemoRepositoryTests()
{
_dbContext = CreateDbContext();
_repository = new DemoRepository(_dbContext);
}
[Fact]
public void Should_add_new_product()
{
var productId = new Guid("DBD9439E-6FFD-4719-93C7-3F7FA64D2220");

using(var scope = new TransactionScope())
{
_repository.AddProduct(productId);

_dbContext.Products.FirstOrDefault(p => p.Id == productId).Should().NotBeNull();

// the transaction is going to be rolled back because the scope is not completed
// scope.Complete();
}
}

Using new Databases

Creating a new database for each test is very easy but the tests are very time consuming. On my machine each test takes about 10 seconds to create and to delete a database on the fly.

The steps of each test are: generate a new database name, create the database by running EF migrations and delete the database in the end.

public class DemoRepositoryTests : IDisposable
{
private readonly DemoDbContext _dbContext;
private readonly DemoRepository _repository;
private readonly string _databaseName;

public DemoRepositoryTests()
{
_databaseName = Guid.NewGuid().ToString();

var options = new DbContextOptionsBuilder<DemoDbContext>()
.UseSqlServer($"Server=(local);Database={_databaseName};...")
.Options;

_dbContext = new DemoDbContext(options);
_dbContext.Database.Migrate();

_repository = new DemoRepository(_dbContext);
}

// Tests come here

public void Dispose()
{
_dbContext.Database.ExecuteSqlCommand((string)$"DROP DATABASE [{_databaseName}]");
}
}

Using different Database Schemas

The 3rd option is to use the same database but different schemas. The creation of a new schema and running EF migrations usually takes less than 50 ms, which is totally acceptable for an integration test. The prerequisites to run queries with different schemas are schema-aware instances of DbContext and schema-aware EF migrations. Read my blog posts for more information about how to change the database schema at runtime:

The class executing integration tests consists of 2 parts: creation of the tables in constructor and the deletion of them in Dispose().

I'm using a generic base class to use the same logic for different types of DbContext.

In the constructor we generate the name of the schema using Guid.NewGuid(), create DbContextOptions using DbSchemaAwareMigrationAssembly and DbSchemaAwareModelCacheKeyFactory described in my previous posts, create the DbContext and run the EF migrations. The database is now fully prepared for executing tests. After execution of the tests the EF migrations are rolled back using IMigrator.Migrate("0"), the EF history table __EFMigrationsHistory is deleted and newly generated schema is dropped.

public abstract class IntegrationTestsBase<T> : IDisposable
where T : DbContext
{
private readonly string _schema;
private readonly string _historyTableName;
private readonly DbContextOptions<T> _options;

protected T DbContext { get; }

protected IntegrationTestsBase()
{
_schema = Guid.NewGuid().ToString("N");
_historyTableName = "__EFMigrationsHistory";

_options = CreateOptions();
DbContext = CreateContext();
DbContext.Database.Migrate();
}

protected abstract T CreateContext(DbContextOptions<T> options,
IDbContextSchema schema);

protected T CreateContext()
{
return CreateContext(_options, new DbContextSchema(_schema));
}

private DbContextOptions<T> CreateOptions()
{
return new DbContextOptionsBuilder<T>()
.UseSqlServer($"Server=(local);Database=Demo;...",
builder => builder.MigrationsHistoryTable(_historyTableName, _schema))
.ReplaceService<IMigrationsAssembly, DbSchemaAwareMigrationAssembly>()
.ReplaceService<IModelCacheKeyFactory, DbSchemaAwareModelCacheKeyFactory>()
.Options;
}

public void Dispose()
{
DbContext.GetService<IMigrator>().Migrate("0");
DbContext.Database.ExecuteSqlCommand(
(string)$"DROP TABLE [{_schema}].[{_historyTableName}]");
DbContext.Database.ExecuteSqlCommand((string)$"DROP SCHEMA [{_schema}]");

DbContext?.Dispose();
}
}

The usage of the base class looks as follows

public class DemoRepositoryTests : IntegrationTestsBase<DemoDbContext>
{
private readonly DemoRepository _repository;

public DemoRepositoryTests()
{
_repository = new DemoRepository(DbContext);
}

protected override DemoDbContext CreateContext(DbContextOptions<DemoDbContext> options,
IDbContextSchema schema)
{
return new DemoDbContext(options, schema);
}

[Fact]
public void Should_add_new_product()
{
var productId = new Guid("DBD9439E-6FFD-4719-93C7-3F7FA64D2220");

_repository.AddProduct(productId);

DbContext.Products.FirstOrDefault(p => p.Id == productId).Should().NotBeNull();
}
}

 

Happy testing! 


Entity Framework Core: Changing DB Migration Schema at Runtime

In the first part of this short blog post series we looked at how to change the database schema of a DbContext, now it is all about changing the schema of the EF Core Migrations at runtime.

The samples are on Github: PawelGerr/Presentation-EntityFrameworkCore

Given is a DemoDbContext implementing our interface IDbContextSchema from the first part of this series.

public interface IDbContextSchema
{
string Schema { get; }
}
public class DemoDbContext : DbContext, IDbContextSchema
{
public string Schema { get; }

public DbSet<Product> Products { get; set; }

...
}

At first we create a migration the usual way: dotnet ef migrations add Initial_Migration

And we get the following:

public partial class Initial_Migration : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable("Products",
table => new { Id = table.Column<Guid>() },
constraints: table => table.PrimaryKey("PK_Products", x => x.Id));
}

protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable("Products");
}
}

Next, we add a constructor to provide the migration with IDbContextSchema and pass the schema to CreateTable and DropTable.

public partial class Initial_Migration : Migration
{
private readonly IDbContextSchema _schema;

public Initial_Migration(IDbContextSchema schema)
{
_schema = schema ?? throw new ArgumentNullException(nameof(schema));
}

protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable("Products",
table => new { Id = table.Column<Guid>() },
constraints: table => table.PrimaryKey("PK_Products", x => x.Id),
schema: _schema.Schema);
}

protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable("Products", _schema.Schema);
}
}

If we try to run the migration then we get a MissingMethodException: No parameterless constructor defined for this object. because EF Core needs a parameterless constructor to be able to create an instance of the migration. Luckily, we can adjust the part that is responsible for the creation of new instances. For that we derive from MigrationsAssembly and override the method CreateMigration. In CreateMigration we check if the migration requires an instance of IDbContextSchema and whether the current DbContext is implementing this interface. If so, then we create new instance of the migration by ourselves and return this instance to the caller, otherwise we pass the call to the default implementation.

public class DbSchemaAwareMigrationAssembly : MigrationsAssembly
{
private readonly DbContext _context;

public DbSchemaAwareMigrationAssembly(ICurrentDbContext currentContext,
        IDbContextOptions options, IMigrationsIdGenerator idGenerator,
        IDiagnosticsLogger<DbLoggerCategory.Migrations> logger)
: base(currentContext, options, idGenerator, logger)
{
_context = currentContext.Context;
}

public override Migration CreateMigration(TypeInfo migrationClass,
        string activeProvider)
{
if (activeProvider == null)
throw new ArgumentNullException(nameof(activeProvider));

var hasCtorWithSchema = migrationClass
            .GetConstructor(new[] { typeof(IDbContextSchema) }) != null;

if (hasCtorWithSchema && _context is IDbContextSchema schema)
{
var instance = (Migration)Activator.CreateInstance(migrationClass.AsType(), schema);
instance.ActiveProvider = activeProvider;
return instance;
}

return base.CreateMigration(migrationClass, activeProvider);
}
}

The last step is to register the DbSchemaAwareMigrationAssembly with the dependency injection of EF Core.

Remarks: to change the schema (or the table name) of the migration history table you have to use the method MigrationsHistoryTable

var optionsBuilder = new DbContextOptionsBuilder<DemoDbContext>()
.UseSqlServer("..."
                      // optional
//, b => b.MigrationsHistoryTable("__EFMigrationsHistory", schema)
                          )
.ReplaceService<IModelCacheKeyFactory, DbSchemaAwareModelCacheKeyFactory>()
.ReplaceService<IMigrationsAssembly, DbSchemaAwareMigrationAssembly>();

 

That's all!  


Entity Framework Core: Use TransactionScope with Caution!

One of the new features of Entity Framework Core 2.1 is the support of TransactionScopes. The usage of a TransactionScope is very easy, just put a new instance in a using, write the code inside the block and when you are finished then call Complete() to commit the transaction:

using (var scope = new TransactionScope())
{
var groups = MyDbContext.ProductGroups.ToList();

scope.Complete();
}

But, before changing your code from using BeginTransaction() to TransactionScope you should know some issues caused by them.

The demos are on GitHub: github.com/PawelGerr/Presentation-EntityFrameworkCore

In all examples we will select ProductGroups from a DemoDbContext.

public class DemoDbContext : DbContext
{
public DbSet<ProductGroup> ProductGroups { get; set; }

public DemoDbContext(DbContextOptions<DemoDbContext> options)
: base(options)
{
}
}

public class ProductGroup
{
public Guid Id { get; set; }
public string Name { get; set; }
}

Async methods

EF has for (almost?) every synchronous operation an asynchronous one. So, it is nothing special (even recommended) to use async-await for I/O operations.

In the first example we are using await inside a TransactionScope.

using (var scope = new TransactionScope())
{
var groups = await Context.ProductGroups.ToListAsync().ConfigureAwait(false);
}

Looks harmless but it throws a System.InvalidOperationException: A TransactionScope must be disposed on the same thread that it was created.

The reason is that the TransactionScope doesn't flow from one thread to another by default. To fix that we have to use TransactionScopeAsyncFlowOption.Enabled:

using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
var groups = await Context.ProductGroups.ToListAsync().ConfigureAwait(false);
}

Does it work now? It depends.

If the calls with and without TransactionScopeAsyncFlowOption are using the same database connection and the call without the option is executed first, then we get another exception: System.InvalidOperationException: Connection currently has transaction enlisted. Finish current transaction and retry.

In other words, the first call is the culprit but the second one breaks:

try
{
using (var scope = new TransactionScope())
{ // We know this one - System.InvalidOperationException:
// A TransactionScope must be disposed on the same thread that it was created.
var groups = await Context.ProductGroups.ToListAsync().ConfigureAwait(false);
}
}
catch (Exception e)
{
// error handling
}

using (var scope = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled))
{
// Implemented correctly but throws anyways // System.InvalidOperationException:
// Connection currently has transaction enlisted. Finish current transaction and retry.
var groups = await Context.ProductGroups.ToListAsync().ConfigureAwait(false);
}

Imagine the first call is done in a 3rd party lib or a framework you are using, i.e. you don't know the code - you will be searching for the cause forever, if you haven't seen this error before.

BeginTransaction within TransactionScope

The transaction scopes can be nested. For example, if the outer scope is rolled back then the changes made in the inner scope are reverted as well. The following example works without problems:

using (var scope = new TransactionScope())
{
// some code
Do();
}

public void Do()
{
using (var anotherScope = new TransactionScope())
{
var groups = Context.ProductGroups.ToList();
}
}

Let's try to change the inner scope to BeginTransaction().

using (var scope = new TransactionScope())
{
// some code
Do();
}

public void Do()
{
using (var tx = Context.Database.BeginTransaction())
{
var groups = Context.ProductGroups.ToList();
}
}

The shown use case is not supported, and we get a System.InvalidOperationException: An ambient transaction has been detected. The ambient transaction needs to be completed before beginning a transaction on this connection.

 Yet again, if Do() is part of a 3rd party lib or a framework then this method has be moved out of outer TransactionScope.

Multiple instances of DbContext (or rather DB connections)

Depending on the project we could end up having multiple instances of DbContext. The instances could be of the same or different type and it may be that the other context doesn't even belong to your application but is being used by a framework you are using.

The use case is the following, we are having a TransactionScope with 2 database accesses using different database connections.

using (var scope = new TransactionScope())
{
var groups = Context.ProductGroups.ToList();
var others = AnotherCtx.SomeEntities.ToList();
}

This use case is not supported as well because a distributed transaction coordinator is required and there is none besides on Windows, so EF team has dropped the support altogether. The exception we get on Windows and Linux is System.PlatformNotSupportedException: This platform does not support distributed transactions.

Conclusion 

The issues mentioned in this blog post are neither new nor specific to Entity Framework Core. I recommend putting some research into this matter before deciding to use or not to use transaction scopes.


Entity Framework Core: Changing Database Schema at Runtime

At the moment there is no built-in support for changing the database schema at runtime. Luckily, Entity Framework Core (EF) provides us with the right tools to implement it by ourselves.

The demos are on GitHub: github.com/PawelGerr/Presentation-EntityFrameworkCore

Given are a database context DemoDbContext and an entity Product.

public class DemoDbContext : DbContext
{
public DbSet<Product> Products { get; set; }

public DemoDbContext (DbContextOptions<DemoDbContext> options)
: base(options)
{
}
}
public class Product
{
public Guid Id { get; set; }
}

There are 2 ways to change the schema, either by applying the TableAttribute or by implementing the interface IEntityTypeConfiguration<TEntity>.

The first option won't help us because the schema is hard-coded.

[Table("Products", Schema = "demo")]
public class Product
{
public Guid Id { get; set; }
}

The second option gives us the ability to provide the schema from DbContext to the EF model configuration. At first we implement the entity configuration for Product.

public class ProductEntityConfiguration : IEntityTypeConfiguration<Product>
{
private readonly string _schema;

public ProductEntityConfiguration(string schema)
{
_schema = schema;
}

public void Configure(EntityTypeBuilder<Product> builder)
{
if (!String.IsNullOrWhiteSpace(_schema))
builder.ToTable(nameof(DemoDbContext.Products), _schema);

builder.HasKey(product => product.Id);
}
}

Now we use the entity configuration in OnModelCreating and pass the schema to it via constructor. Additionally, we create the interface IDbContextSchema containing just the schema (i.e. a string) to be able to inject it into DemoDbContext.

public interface IDbContextSchema
{
string Schema { get; }
}
// DbContext implements IDbContextSchema as well
// so we know it is "schema-aware"
public class DemoDbContext : DbContext, IDbContextSchema
{
public string Schema { get; }

public DbSet<Product> Products { get; set; }

public DemoDbContext(DbContextOptions<DemoDbContext> options,
IDbContextSchema schema = null)
: base(options)
{
Schema = schema?.Schema;
}

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);

modelBuilder.ApplyConfiguration(new ProductEntityConfiguration(Schema));
}
}

We are almost done. The last task is to change how EF is caching database model definitions. By default just the type of the DbContext is used but we need to differentiate the models not just by type but by the schema as well. For that we implement the interface IModelCacheKeyFactory.

public class DbSchemaAwareModelCacheKeyFactory : IModelCacheKeyFactory
{
public object Create(DbContext context)
{
return new {
        Type = context.GetType(),
        Schema = context is IDbContextSchema schema ? schema.Schema : null
    };
}
}

No we have to replace the default implementation with ours and to register the IDbContextSchema. In current example the IDbContextSchema is just a singleton but it can be provided by anything we want like read from a database or derived from a JWT bearer token during an HTTP request, etc.

IServiceCollection services = ...;

services
.AddDbContext<DemoDbContext>(
     builder => builder.UseSqlServer("...")
                  .ReplaceService<IModelCacheKeyFactory, DbSchemaAwareModelCacheKeyFactory>())
  .AddSingleton<IDbContextSchema>(new DbContextSchema("demo"));

--------------------------------------------

// just a helper class public class DbContextSchema : IDbContextSchema
{
public string Schema { get; }

public DbContextSchema(string schema)
{
Schema = schema ?? throw new ArgumentNullException(nameof(schema));
}
}

 

Voila! 

 

PS: There is one special use case for that feature - isolation of integration tests due to missing support of ambient transactions. For that we need schema-aware migrations we will look at in the next blog post.

Stay tuned!