Testing

Quino provides a lot of support for building powerful, fast and concise tests. As with everything else in Quino, there are several layers of support and a lot of components that can be used with or without the test-fixture base classes.

One of the central tasks for a testing fixture is to prepare an environment in which code can be run without a lot of additional setup. Quino's philosophy is to avoid mocking and faking wherever possible. That is, tests can be executed against the exact same back-end as the application itself uses. This means that Quino tests generally run with a real database since a database is quite a lot of work to mock (and is an essential component of many applications). These are called application-based tests.

Some tests will elect to use less of the application support in order to guarantee that as little code as possible is executed before executing a given test. These are called service-based tests.

Assertions & Expectations

The TestsBase provides basic assertion support in addition to that provided by the testing framework used by the application. For example,

• AssertTextExpectation: Asserts that a given sequence of items or text matches the contents of the expectations filename for a given test
• GetExpectationFilename: Gets the name of the expectations file for a given test
• TestingTools: A property of type ITestingTools that provides support for advanced assertions
• ProjectTools: A property of type IProjectTools that provides support for getting paths and filenames relative to the test fixture

These features taken together provide test-framework–independent support for advanced assertions and expectations.

Application-based Tests

A central feature of Quino is the IApplication. A product will want most test fixtures to run with an application configured as closely to that of the "main" product as possible.

The ApplicationBasedTestsBase provides the following services:

• Application: Provides access to the main instance of the product's application for a test fixture
• CreateApplication(): Should be overridden in a product's testing base class to return the base application for that product.
• InitializePerFixture(): Descendants can override this to complete setup
• CreateAndStartUpApplication(): The method to call from a testing framework to start a fixture (e.g. in NUnit, a method decorated with the OneTimeSetUp attribute)
• ShutDownApplication(): The method to call from a testing framework to stop a fixture (e.g. in NUnit, a method decorated with the OneTimeTearDown attribute)
• SetUpApplication(): Any of the three overloads can be overridden to customize the application returned by CreateApplication. Products should use the overload with the fewest parameters possible (see example below).
• RunApplication(): Executes a "sub"-Application in a test (see below for use cases and examples).

An example base class from the Quino Sandbox is shown below.

public class SandboxTestsBase : NUnitMetaApplicationBasedTestsBase
{
  /// <inheritdoc />
  protected override IApplication CreateApplication(bool isFixtureApplication = false) => new SandboxApplication();

  /// <inheritdoc />
  protected override IApplication SetUpApplication(IApplication application)
  {
    return base
      .SetUpApplication(application)
      .UseTitle("Quino Sandbox Tests")
      .UseTestingMainConfigurationFilename(typeof(SandboxTestsBase), SandboxConstants.ConfigurationFilename);
  }
}

Metadata-based Tests

The MetaApplicationBasedTestsBase extends the basic application-based support to provide access to a Session property that is global to the fixture. Most tests will use this feature for data access.

An example is shown below.

[Test]
[RunInTransaction]
public void TestDelegatePropertyReload()
{
  var person = new Person(Session)
  {
    FirstName = "Hans",
    LastName = "Muster",
    Company = new Company(Session) { Name = "Muster AG" }
  };
  person.Save();

  Assert.That(person.FirstName, Is.EqualTo("Hans"));

  person.Reload();

  Assert.That(person.FirstName, Is.EqualTo("Hans"));
}

With the standard support, tests are very clear and concise: there isn't any extra setup and mocking to do once you've defined the base fixture for your product. The RunInTransaction attribute used above is discussed below.

Testing Attributes

Quino defines some special testing attributes for tests and fixtures.

• RunInTransaction: Use this attribute on a test or fixture to put the main Session into a transaction for the duration of a test or all tests in a fixture. Quino uses this attribute widely to decouple the effects of tests. In the example above, you can see that an additional person would have been saved to the database had the effects of the test not been rolled back.
• AssertNoOpenConnectionsAttribute: Use this attribute on a test or fixture to assert that no database connections have been left open during the run of the test or fixture. This attribute is used more rarely, but can come in handy.

Running "sub"-Applications

Most test fixtures run all tests with a single Application (and in metadata-based applications, with a single Session ). However, some fixtures will have tests that should logically be in a test fixture, but also need to modify some part of the application that will break other tests in the fixture.

One solution is to move the test to its own fixture. This is an acceptable solution, but it unnecessarily spreads tests among smaller fixtures that were created not for semantic but for technical reasons.

Another approach is to work with RunApplication to launch a new application context within a test. An example is shown below.

[Test]
public void TestLanguageSettingsInReleaseMode()
{
  void TestLanguageSettings(IApplication application)
  {
    var languageSettings = application.GetInstance<ILanguageSettings>();

    Assert.That(languageSettings.FallbackTemplate, Is.EqualTo("{0}"));
    Assert.That(languageSettings.MissingTemplate, Is.EqualTo(string.Empty));
  }

  RunApplication(s => CreateAndSetUpSubApplication(s).UseRunMode(RunMode.Release), TestLanguageSettings);
}

• Note how the test calls CreateAndSetUpSubApplication to create a copy of the application used by the fixture itself and then configures that application to run in release mode.
• The run parameter is a local function TestLanguageSettings.

Current User in Tests

By default, the application started with a test fixture is logged in with the current user running the tests. However, most products with authorization will want to create and log in with a testing user that has appropriate access to the data needed by a fixture.

Quino provides the following extension method for an IApplication:

application.UseTestingLogin("TestingUserName");

Service-based Tests

There is a class of fixtures that should run without having started an entire application. In these cases, though, the IOC should still be available and configured as the product would expect it to be configured.

Quino provides support for this with the ServicesBasedTestsBase fixture.

• Services: Provides access to the main instance of the product's IOC for a test fixture
• CreateServiceRegistrationHandler: Creates an instance of the product's IOC (e.g. SimpleInjectorServiceLocator).
• CreateServicesAndObjects: The method to call from a testing framework to start a fixture (e.g. in NUnit, a method decorated with the OneTimeSetUp attribute)
• ConfigureServices: Can be overridden to customize the services (analogous to ApplicationBasedTestsBase.SetUpApplication()).
• ConfigureObjects: Can be overridden to set up objects in the IOC after all registrations have been made (analogous to ApplicationBasedTestsBase.InitializePerFixture()).

The MetaServicesBasedTestsBase extends the basic services-based support to include standard Quino registrations.

An example is shown below. This test shows how an application can write test fixtures that focus on a very specific component without setting up an application (and also knowing that the component runs with only the setup included in the fixture).

[TestFixture, Category("Security")]
public class CredentialsTests : EncodoServicesBasedTestsBase
{
  [Test]
  public void TestCurrentCredentials()
  {
    var credentials = Services.GetInstance<ICurrentUserCredentialsFactory>().CreateCredentials();

    Assert.That(credentials.PasswordEncryptionType, Is.EqualTo(EncryptionType.None));

    // More assertions, etc. ...
  }

  // Tests removed...

  /// <inheritdoc />
  protected override IServiceRegistrationHandler ConfigureServices(IServiceRegistrationHandler serviceRegistrationHandler)
  {
    return base
      .ConfigureServices(serviceRegistrationHandler)
      .RegisterWindowsConnectionServices();
  }
}

Another concrete example is for testing model integrity (discussed below) without setting up database access or triggering schema migration.

NUnit

The actual testing framework is a final, thin layer on top of all of this testing support. Quino provides an implementation for NUnit. Other testing frameworks can be relatively easily supported. See the NUnit implementation for an example.

• The class NUnitMetaApplicationBasedTestsBase in assembly Quino.Testing.NUnit is the base class that most products will use.
• The class NUnitMetaServicesBasedTestsBase in assembly Quino.Testing.NUnit is the base class products can use for services-only tests.

Test Fixtures

Core Tests

Quino provides a lot of support for making a product and its services and data available in automated tests.

A product can/should configure the following features:

• Which name should the testing application use?
• Where is the configuration file (e.g. for database/server connections)?
• What is the name of the default user in tests?

public class SandboxTestsBase : NUnitMetaApplicationBasedTestsBase<SandboxApplication>
{
  /// <inheritdoc />
  protected override IApplication SetUpApplication(IApplication application)
  {
    return base
      .SetUpApplication(application)
      .UseTitle("Quino Sandbox Tests")
      .UseTestingMainConfigurationFilename(typeof(SandboxTestsBase), SandboxConstants.ConfigurationFilename)
      .UseTestingLogin(SandboxSecurityConstants.TestUser)
      .UseDataGenerator<TestingDataGenerator>();
  }
}

The testing base class above does the following:

• Uses the NUnit testing framework
• Uses the SandboxApplication as a base application
• Configures the title to Quino Sandbox Tests
• Uses the configuration filename found next to the assembly
• Sets up the testing login name (optional; the default username is Test)
• Includes testing data generated with the TestingDataGenerator

The following sample shows a test fixture for this product that uses this base class.

[TestFixture]
public class OfficeTests : SandboxTestsBase
{
  [Test]
  public void TestCount()
  {
    Assert.That(Session.GetCount<Office>(), Is.EqualTo(5));
  }
}

Web Tests

Quino-WebAPI provides support for web-testing that builds on the testing support in Quino-Standard.

For example, to test controllers, a product will want to create a base class that extends its own testing base fixture to include web-testing support.

public class SandboxControllerTestsBase<TController> : SandboxTestsBase
  where TController : ApiController
{
  protected T CreateController<T>()
      where T : ApiController
  {
      return Application.GetInstance<IWebApiTestingControllerFactory>().CreateController<T>();
  }

  protected TController CreateController() => CreateController<TController>();

  /// <inheritdoc />
  protected override IApplication SetUpApplication(IApplication application, IApplicationCreationSettings applicationCreationSettings) =>
    base
      .SetUpApplication(application, applicationCreationSettings)
      .UseMetaWeb()
      .UseWebTesting()
      .UseWebTestingLogin(JobVortexTestingConstants.TestUser);
}

The testing base class above does the following:

• Declares a CreateController<T>() method to create any controller
• Declares a CreateController() method to create the main controller for the test fixture
• Includes standard web, web-testing and authentication support

The following sample shows a test fixture for this product that uses this base class.

[TestFixture]
public class OfficeControllerTests : SandboxControllerTestsBase<OfficeController>
{
  [Test]
  public void TestCount()
  {
    using (var controller = CreateController())
    {
      Assert.That(controller.GetCount(), Is.EqualTo(5));
    }
  }

  [Test]
  public void TestClear()
  {
    using (var adminController = CreateController<AdminController>())
    {
      using (var officeController = CreateController())
      {
        Assert.That(officeController.GetCount(), Is.EqualTo(5));

        adminController.ClearOffices();

        Assert.That(officeController.GetCount(), Is.EqualTo(0));
      }
    }
  }
}

Product-specific Test Fixtures

Quino includes several test fixtures that provide standard testing coverage for products.

• QuinoApplicationIntegrityTestsBase: Tests and verifies that the product's IOC container is valid (all registrations can be instantiated) and also audits the startup/shutdown actions.
• QuinoModelCodeGenerationTestsBase: Tests whether the code generators run against the product. The tests store expectation files, so that a product can verify and audit code-generation without overwriting actually real code.
• QuinoModuleCodeGenerationTestsBase: Same as the model-based fixture, but for individual modules.
• QuinoModelValidationTestsBase: Runs the product's model against standard model validators.
• QuinoModelIntegrityTestsBase: Audits the overall model as well as the schema-relevant changes. This allows a product to detect subtle changes to the model after larger upgrades or refactoring and avoid inadvertent schema-migration.

QuinoModelIntegrityTestsBase Example

Most products will have extra dependencies in order to generate a model (e.g. when metadata builders inject components), generate code or start up. Products can override ConfigureServices(), as Sandbox does with the reporting components and its own commands below.

[TestFixture]
public class SandboxModelIntegrityTests : QuinoModelIntegrityTestsBase<SandboxGeneratedModelBuilder>
{
  /// <inheritdoc />
  protected override IServiceRegistrationHandler ConfigureServices(IServiceRegistrationHandler serviceRegistrationHandler)
  {
    return base
      .ConfigureServices(serviceRegistrationHandler)
      .RegisterMetaReportingServices()
      .RegisterSandboxCommands();
  }
}

These model-integrity tests emit expectation files for both the schema-relevant and full models. These tests will fail whenever the model changes, but provide a confirmation of the changes you've made to the model. Once a model is mature, these tests become very important as they will alert a developer to inadvertent changes to either the database schema or the view layouts.

At the very least, a product will have a protocol in its source-control repository that details which changes were made to the model and when.

Quino Framework Tests

This section covers base classes that are of relevance to Quino-framework developers only. That is, only code that extends Quino with new components should use these fixtures.

• Quino.Testing.Models and Quino.Testing.Models.Generated: Contains several model builders for models used throughout Quino's test suite (e.g. Punchclock, Testing, Main, etc.)
• Quino.Testing.Models.Fixtures: Contains base testing fixtures that provide easy access to the testing models in Quino.Testing.Models

Other Quino solutions, like Quino-WebApi and Quino-Windows, use these fixtures and models.