Testing images

Concreate is able to run behave based tests for images. We suggest you read the Behave documentation before reading this chapter.

An image can be tested by running:

$ concreate test

Test options

  • --test-wip – only run tests tagged with the @wip tag.
  • --test-steps-url – a git repository url containing steps for tests.
  • --tag altname – Overrides the name of the Image used for testing to altname. Only the first occurrence of this argument is honoured.

About Tests

Behave tests are defined in two separate parts: steps and features.

You can place the files defining tests in a tests directory next to the image descriptor, module descriptor or in a root of a git repository which contains the modules.

The tests directory is structured as follows:

tests/features
tests/features/amq.feature
tests/steps
tests/steps/custom_steps.py

The tests/features directory is the place where you can drop your behave features.

The tests/steps directory is optional and contains custom steps for the specific image/module.

We strongly recommend that a test is written for every feature that is added to the image.

Concreate comes with a list of build-in steps that are available for use in tests. See the steps repository.

Where necessary we encourage people to add or extend these steps.

Tags

Concreate selects which tests to run via the tags mechanism. Here are several examples of ways ways that tags could be used for managing tests across a set of related images:

  1. Product tags

    Tags based on image names. Concreate derives two test tag names from the name of the Image being tested. The whole image name is converted into one tag, and everything before the first ‘/’ character is converted into another. Example: If you are testing the jboss-eap-7/eap7 image, tests will be invoked with tags @jboss-eap-7 and @jboss-eap-7/eap7.

    If --tag is specified, then the argument is used in place of the Image name for the process above. Example If you provided --tag foo/bar, then the tags used would be @foo and @foo/bar.

  2. Wip tags

    This is very special behavior used mainly in development. Its purpose is to to limit the tests to be run to a subset you are working on. To achieve this you should mark your in-development test scenarios with the @wip tag and run concreate test --test-wip. All other scenarios not tagged @wip will be ignored.

  3. The @ci tag

    If concreate is not running as a user called jenkins, the tag @ci is added to the list of ignored tags, meaning any tests tagged @ci are ignored and not executed.

    The purpose of this behavior is to ease specifying tests that are only executed when run within Jenkins CI.