SoftwareEntwicklung Beratung Schulung

A coders, hackers heaven.....Hm...I do not think so...

Maven Plugin Testing - in a Modern Way - Part IV

In the prevous part of the series – Maven Plugin Testing – In a Modern way – Part III we have seen how to define command line options. In this part we will take a deeper look which goals will run for each test case and how we can change that.

Let us start with simple example test case like the following:

1
2
3
4
5
6
7
8
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

If we ran that integration test Maven will be called like the following:

1
mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors package

We will concentrate on the part package in the above example. This is a life cycle phase of Maven. So what can we do if we like to call something like mvn .. verify instead? This can simply being achieved by using the @MavenGoal annotation like this:

1
2
3
4
5
6
7
8
9
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

So let us take a deeper look onto the following example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

The test case first will be called with the phase verify whereas second and third will be called with the package. So this means you can overwrite the default behaviour for each test case separately.

Sometimes you want to execute Maven during an integration test like this: mvn clean verify or in general with multiple life cycle phase. This can be achieved by using multiple MavenGoal annotations as in the following example:

1
2
3
4
5
6
7
8
9
10
11
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("clean")
  @MavenGoal("verify")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

Of course there are situations where you have a bunch of integration tests which needed to be executing the previously defined goals. This can handled by defining the @MavenGoal annotation on a class level instead like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
@MavenJupiterExtension
@MavenGoal("clean")
@MavenGoal("verify")
class BaseIT {

  @MavenTest
  void first(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void second(MavenExecutionResult result) {
     ...
  }

  @MavenTest
  void third(MavenExecutionResult result) {
     ...
  }
}

This also gives the opportunity to let run a single test (or more than one) case within a test class with different goals depending on what you like to achieve.

Another example on how to define the @MavenGoal annotation on a class level which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
@MavenJupiterExtension
@MavenGoal("clean")
class GoalsOnClassIT {

  @MavenTest
  @DisplayName("This will check the goal which is defined on the class.")
  void goal_clean(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "Scanning for projects...",
            "-------------------< com.soebes.katas:kata-fraction >-------------------",
            "Building kata-fraction 1.0-SNAPSHOT",
            "--------------------------------[ jar ]---------------------------------",
            "--- maven-clean-plugin:3.1.0:clean (default-clean) @ kata-fraction ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn().isEmpty();
  }
}

The next logical step is to create a meta annotation to make life easier. We would like to combine clean and verify within a single annotation @GoalsCleanVerify which can be done like this:

1
2
3
4
5
6
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenGoal({"clean", "verify"})
public @interface GoalsCleanVerify {
}

Such kind of meta annotation can be used on class level (defined by the annotation itself) as well as on method level like this:

1
2
3
4
5
@MavenJupiterExtension
@GoalsCleanVerify
class MetaAnnotationGoalIT {
 ...
}

So now lets think about Part III where we defined this meta annotation:

1
2
3
4
5
6
7
8
9
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

This meta annotation can now being enhanced by the needed @MavenGoal annotations.

1
2
3
4
5
6
7
8
9
10
11
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
@MavenGoal("clean")
@MavenGoal("verify")
public @interface MavenTestOptions {
}

This means you can define easily your set of annotation or combination of command line options and goals or more sophisticated combine it with @MavenJupiterExtension like this:

1
2
3
4
5
6
7
8
9
10
11
12
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
@MavenGoal("clean")
@MavenGoal("verify")
public @interface MavenTestOptions {
}

This will give us the option to use it like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

This combines the given options with the defined goals in one single annotation. If you need to change something you have to fix only a single point.

So did we miss something? Yes we did. Sometimes you want to call your plugin with a separate goal like this:

1
mvn org.test.maven.plugin:maven-x-plugin:goal

This can be achieved by using the @MavenGoal annotation like this:

1
2
3
4
5
6
7
8
9
10
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("org.test.maven.plugin:maven-x-plugin:goal")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

Now let us assume the given plugin is the one which should be tested via the given integration test. Then you need to define the correct groupId, artifactId, version and the correct goal. Unfortunately with each release of your plugin the version changes etc.

1
2
3
4
5
6
7
8
9
10
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  @MavenGoal("${project.groupId}:${project.artifactId}:${project.version}:goal-to-test")
  void first(MavenExecutionResult result) {
     ...
  }
  ...
}

The placeholders ${project.groupId}, ${project.artifactId} and ${project.version} are exactly the information from your project pom in which you define the coordinates of the plugin your are developing. This makes sure only this plugin version is being used and not any other version is being tried to download from central or other repositories during your integration tests.

Good examples can be found in maven-invoker-plugin based integration tests.

So this it is for Part IV. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example project which shows the previous example can be found on GitHub.

Maven Plugin Testing - in a Modern Way - Part III

In the second part of the series – Maven Plugin Testing – In a Modern way – Part II we have seen how to make the basic integration test while checking the log output of Maven builds.

In this third part we will dive into how Maven will be called by default during the integration tests and how we can influence that behaviour.

Let us begin with the following basic integration test (we ignore the project which is used for testing at the moment.)

1
2
3
4
5
6
7
8
@MavenJupiterExtension
class BaseIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
}

The above test will execute Apache Maven by using the following options by default:

  • --batch-mode
  • --show-version
  • --errors

That means that each execution within the Integration Testing Framework will be done like this:

1
mvn --batch-mode --show-version --errors

To get that correctly working and in particular having a local cache for each integration test case the integration testing framework will add: -Dmaven.repo.local=... to each call as well. This is necessary to get the following result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
.
└──target/
   └── maven-it/
       └── org/
           └── it/
               └── FirstMavenIT/
                   └── the_first_test_case/
                       ├── .m2/
                       ├── project/
                       │   ├── src/
                       │   ├── target/
                       │   └── pom.xml
                       ├── mvn-stdout.log
                       ├── mvn-stderr.log
                       └── mvn-arguments.log

The option -Dmaven.repo.local=... can’t be changed at the moment. The used command line arguments are wrote into the mvn-arguments.log file which can be consulted for later analysis. So in the end a command line for an integration test looks like this:

1
mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors package

The goal which is used (package) and how it can be changed will be handled in Part IV of the series.

So far so good, but sometimes or maybe more often it is needed to add other command line options to a Maven call in particular in relationship with integration tests.

Sometimes your own plugin/extension will printout some useful information on debug level but how to test that? We have mentioned before that the options which are used for an integration test does not contain the debug option.

We can now express the need via this (basic_configuration_with_debug):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
@MavenJupiterExtension
class FailureIT {
  ...
  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void basic_configuration_with_debug(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ basic_configuration_with_debug ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ basic_configuration_with_debug ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ basic_configuration_with_debug ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ basic_configuration_with_debug ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ basic_configuration_with_debug ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ basic_configuration_with_debug ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ basic_configuration_with_debug ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ basic_configuration_with_debug ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ basic_configuration_with_debug ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn()
        .containsSubsequence(
            "Neither executionException nor failureException has been set.",
            "JAR will be empty - no content was marked for inclusion!");

    assertThat(result)
        .isSuccessful()
        .out()
        .debug()
        .containsSubsequence(
            "Created new class realm maven.api",
            "Project: com.soebes.itf.maven.plugin.its:basic_configuration_with_debug:jar:1.0",
            "Goal:          org.apache.maven.plugins:maven-resources-plugin:3.1.0:resources (default-resources)"
        );

  }
}

It’s important to say that by using the @MavenOption(..) automatically all other previously mentioned command line options will not being used anymore. In this example the final command line looks like this for the test case basic_configuration_with_debug:

1
mvn -Dmaven.repo.local=<path> --debug package

So based on turning on debugging output it means that you can check debugging output like this:

1
2
3
4
5
6
7
8
9
assertThat(result)
    .isSuccessful()
    .out()
    .debug()
    .containsSubsequence(
        "Created new class realm maven.api",
        "Project: com.soebes.itf.maven.plugin.its:basic_configuration_with_debug:jar:1.0",
        "Goal:          org.apache.maven.plugins:maven-resources-plugin:3.1.0:resources (default-resources)"
    );

If you like to have the --batch-mode option, --show-version as well as the --error option back in your test case have to add them like this:

1
2
3
4
5
6
7
8
  @MavenTest
  @MavenOption(MavenCLIOptions.BATCH_MDOE)
  @MavenOption(MavenCLIOptions.SHOW_VERSION)
  @MavenOption(MavenCLIOptions.ERRORS)
  @MavenOption(MavenCLIOptions.DEBUG)
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

The result will be that your Maven command line now looks like before including the supplemental --debug option:

1
mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors --debug package

That shows that you can easily combine several command line options for a test case via the @MavenOption annotation.

Some command line options in Maven need supplemental information like --log-file <arg> which requires the name of the log file to redirect all the output into. How can we express this with the @MavenOption annotation? This can simply being achieved like the following:

1
2
3
4
5
6
7
  @MavenTest
  @MavenOption(QUIET)
  @MavenOption(SHOW_VERSION)
  @MavenOption(value = LOG_FILE, parameter = "test.log")
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

As you can see in the above example you have to give the option via the value of the annotation. The parameter of the option has to be given via parameter. In this case we have used static imports to make it more readable. You can of course work without static imports like this (Just a matter of taste):

1
2
3
4
5
6
7
  @MavenTest
  @MavenOption(MavenCLIOptions.QUIET)
  @MavenOption(MavenCLIOptions.SHOW_VERSION)
  @MavenOption(value = MavenCLIOptions.LOG_FILE, parameter = "test.log")
  void basic_configuration_with_debug(MavenExecutionResult result) {
  ...
  }

So what about using the same command line options for several test cases? You can simply add the command line options onto the test class level which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_four(MavenExecutionResult result) {
    ..
  }

}

This means that for each given test case ( case_one, case_two, case_three and case_four) the same set of command line options will be used. Apart from that it is much more convenient to define the command line options only once and not for every single test case.

Wait a second. I want to execute case_four with different command line options? Ok no problem just define the set command line options onto that particular test case like in the following example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

The test case case_four will not inherit the command line options defined on the class level. It will use only those defined on the test case itself.

Now a usual developer question: I’m really lazy I don’t want to write all those four MavenOption annotation for each test class. In particular if I need to change those command line options I have to go through each test class one by one?

There is of course a more convenient solution for that problem. This is called a meta annotation. We can simply create a meta annotation which we call MavenTestOptions. This meta annotation looks like this:

1
2
3
4
5
6
7
8
9
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

This meta annotation can now being used to change the test class of the previous example like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
@MavenJupiterExtension
@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

The above can be improved a bit more. We can integrate the annotation @MavenJupiterExtension into our self defined meta annotation like this (In the example project I have named the meta annotation @MavenITExecution to have different examples in one project.):

1
2
3
4
5
6
7
8
9
10
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RUNTIME)
@Inherited
@MavenJupiterExtension
@MavenOption(MavenCLIOptions.FAIL_AT_END)
@MavenOption(MavenCLIOptions.NON_RECURSIVE)
@MavenOption(MavenCLIOptions.ERRORS)
@MavenOption(MavenCLIOptions.DEBUG)
public @interface MavenTestOptions {
}

Based on that we can change the test case to look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
@MavenTestOptions
class FailureIT {

  @MavenTest
  void case_one(MavenExecutionResult project) {
    ..
  }

  @MavenTest
  void case_two(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  void case_three(MavenExecutionResult result) {
    ..
  }

  @MavenTest
  @MavenOption(MavenCLIOptions.DEBUG)
  void case_four(MavenExecutionResult result) {
    ..
  }

}

So this it is for Part III. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example of the shown uses cases can be found on GitHub.

Maven Plugin Testing - in a Modern Way - Part II

In the first part of the series – Maven Plugin Testing – In a Modern way – Part I we have seen how to make the basic setup with The Integration Testing Framework and run very basic integration test.

In this second part we will take a deeper look into other aspects of testing Maven plugins in particular how we check the logging output of a Maven build process.

Let us begin with writing more than a single integration test case. You can of course write multiple test cases within a single test class like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
@MavenJupiterExtension
class SeveralMavenIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
     ...
  }
  @MavenTest
  void the_second_test_case(MavenExecutionResult result) {
     ...
  }
  @MavenTest
  void the_third_test_case(MavenExecutionResult result) {
     ...
  }
}

Apart from the test cases them self we need the according projects which are used as test projects which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
.
└── src/
    └── test/
        └── resources-its/
            └── org/
                └── it/
                    └── SeveralMavenIT/
                        ├── the_first_test_case/
                        │   ├── src/
                        │   └── pom.xml
                        ├── the_second_test_case/
                        │   ├── src/
                        │   └── pom.xml
                        └── the_this_test_case/
                            ├── src/
                            └── pom.xml

So after we have executed the integration tests (mvn verify) the resulting directory structure will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
.
└──target/
   └── maven-it/
       └── org/
           └── it/
               └── SeveralMavenIT/
                   ├── the_first_test_case/
                   │   ├── .m2/
                   │   ├── project/
                   │   │   ├── src/
                   │   │   ├── target/
                   │   │   └── pom.xml
                   │   ├── mvn-stdout.log
                   │   ├── mvn-stderr.log
                   │   └── other logs
                   ├── the_second_test_case/
                   │   ├── .m2/
                   │   ├── project/
                   │   │   ├── src/
                   │   │   ├── target/
                   │   │   └── pom.xml
                   │   ├── mvn-stdout.log
                   │   ├── mvn-stderr.log
                   │   └── mvn-arguments.log
                   └── the_third_test_case/
                       ├── .m2/
                       ├── project/
                       │   ├── src/
                       │   ├── target/
                       │   └── pom.xml
                       ├── mvn-stdout.log
                       ├── mvn-stderr.log
                       └── mvn-arguments.log

Based on the resulting directory structure you can see each test case completely separated from each other. This means also that each test case contains it’s own maven cache (.m2/repository). You can find also the separated log file outputs and the separate project directory which contains the test project after the test run. That is very helpful for later issue analysis.

So now let us take a deeper look into the test cases:

1
2
3
4
  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {

  }

In each test you have seen a parameter to the test method MavenExecutionResult result. The injected parameter gives you access to the result of the test build of project.

This class contains appropriate methods to access the result of the build process, the project cache, the project itself (in other words to the directory) and of course to the different logging output files which have been created during the run of the test.

So the first thing you usually check would be if the built has been successful or not. This depends on the type of integration test you are writing. This can be achieved by using the following:

1
assertThat(result).isSuccessful();

This expects the built being successful as you already suspected. You can of course write a test which assumes that your built must fail which can be expressed like this:

1
assertThat(result).isFailure();

So the isSuccessful() means the return code 0 whereas isFailure() represents a return code which is not 0.

You can combine the check for a successful build and no warning output like this:

1
2
3
4
assertThat(result)
    .isSuccessful()
    .out()
    .warn().isEmpty();

So .out() will access the created output file of the build mvn-stdout.log. The .warn() will filter out all lines which start with [WARNING]. The .isEmpty() is part of AssertJ framework for assertion against lists which implies that the result is empty.

So now let us check some output which is produced by a usual build. The output emits [INFO] so the test can use .out().info(). instead which looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
@MavenJupiterExtension
class FirstIT {
  void base_test (MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ kata-fraction ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ kata-fraction ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ kata-fraction ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ kata-fraction ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ kata-fraction ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ kata-fraction ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ kata-fraction ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ kata-fraction ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ kata-fraction ---"
        );
  }
}

The .containsSubsequence(..) checks that the sequence is in the correct order with optional supplemental parts in between.

While writing plugins/extensions it sometimes happens that you emit an information on WARN Level to give some hints what needs to be mentioned but will not fail a build.

The maven-jar-plugin for example will emit a warning if no content will be added into the jar. This could be checked like the following:

1
2
3
4
5
assertThat(result)
    .isSuccessful()
    .out()
    .warn()
    .contains("JAR will be empty - no content was marked for inclusion!");

So this can be combined to create a more comprehensive test case like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
@MavenJupiterExtension
class FailureIT {

  @MavenTest
  void basic_configuration_checking_logout(MavenExecutionResult result) {
    assertThat(result)
        .isSuccessful()
        .out()
        .info()
        .containsSubsequence(
            "--- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ basic_configuration_checking_logout ---",
            "--- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ basic_configuration_checking_logout ---",
            "--- maven-resources-plugin:3.1.0:resources (default-resources) @ basic_configuration_checking_logout ---",
            "--- maven-compiler-plugin:3.8.1:compile (default-compile) @ basic_configuration_checking_logout ---",
            "--- maven-resources-plugin:3.1.0:testResources (default-testResources) @ basic_configuration_checking_logout ---",
            "--- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ basic_configuration_checking_logout ---",
            "--- maven-surefire-plugin:3.0.0-M4:test (default-test) @ basic_configuration_checking_logout ---",
            "--- maven-jar-plugin:3.2.0:jar (default-jar) @ basic_configuration_checking_logout ---",
            "--- maven-site-plugin:3.9.1:attach-descriptor (attach-descriptor) @ basic_configuration_checking_logout ---"
        );
    assertThat(result)
        .isSuccessful()
        .out()
        .warn()
        .contains("JAR will be empty - no content was marked for inclusion!");

  }
}

If you write a plugin which contains a parameter for encoding an output like this (might look familiar to you) should be produced:

1
Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!

This can be checked within a test case like this:

1
2
3
4
assertThat(result)
  .out()
  .warn()
  .containsExactly("Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!");

So this it is for Part II. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example can be found on GitHub.

Maven Plugin Testing - in a Modern Way - Part I

You have decided to write a Maven Plugin. Now the important part comes around the corner: How to test a Maven Plugin? There are in general several options to test a Maven Plugins for example:

You can read about different reason not to use one of those options.

Coming back to the subject and think about to write integration tests for a Maven plugin. Writing integration test means having three parties involved:

  1. The component you would like to test (typically the Maven plugin/Extension etc.).
  2. The testing code by which you check the functionality.
  3. The Project you would like to test with (where your Maven plugin usually is configured in to be used.)

Let us start with a simple test case which simply requires that the build which contains your new plugin to run successful. This looks like this by using the Integration Testing Framework:

1
2
3
4
5
6
7
8
9
@MavenJupiterExtension
class FirstMavenIT {

  @MavenTest
  void the_first_test_case(MavenExecutionResult result) {
    assertThat(result).build().isSuccessful();
  }

}

The written tests is easy to understand? Isn’t it? The foundation of the Integration Testing Framework is the JUnit Jupiter extension mechanism which allows to easily write extensions which can do many things.

Going back to the given simple integration test we need now two other parts to get a working integration test. The code of the plugin (which I assume already exists) and of course the project where you would like to test your plugin with.

The above is an integration test written by using the Integration Testing Framework (ITF for short) which exactly does that.

So a bit more nifty details here. The @MavenJupiterExtension is the annotation to activate the ITF extension. The @MavenTest is the equivalent for unit tests which are annotated with @Test of course it makes it very clear that we are in the context of a test for Maven.

Ah finally we have the parameter of the test method MavenExecutionResult result which gives you access to the result of the build and many more. The assertThat(..) is a customer assertions for AssertJ to make writing assertions more convenient.

So now we have to define the project which is used to use our plugin. As always in Apache Maven you should follow convention over configuration and the ITF is not an exception of that. We locate the integration itself into the usual location of test like src/test/java/.... I recommend to name your integration tests like *IT.java but of course you can change that if you like.

1
2
3
4
5
6
7
.
└── src/
    └── test/
        └── java/
            └── org/
                └── it/
                    └── FirstMavenIT.java

So now we need to put the project into a particular location like this:

1
2
3
4
5
6
7
8
9
10
.
└── src/
    └── test/
        └── resources-its/
            └── org/
                └── it/
                    └── FirstMavenIT/
                        └── the_first_test_case/
                            ├── src/
                            └── pom.xml

It is important to mention that the directory FirstMavenIT represents the integration test class and the the_first_test_case directory represents the method name of the integration test class. This will give us the opportunity to define several test cases within the class. Based on the association between method name and directory name it is necessary to write method names in lowercase and separate by using an underscore. If you use camel case method names this could cause issues with case insensitive file systems.

As you can see that the directory the_first_test_case contains a full fledged project which comprise of a pom.xml file and of course the usual structures for source code and maybe unit test etc.

Let us take a look into the pom.xml of the project which is used to test our plugin:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<build>
  <plugins>
    <plugin>
      <groupId>com.soebes.itf.jupiter.extension</groupId>
      <artifactId>itf-failure-plugin</artifactId>
      <version>@project.version@</version>
      <executions>
        <execution>
          <id>the_first_test_case</id>
          <phase>initialize</phase>
          <goals>
            <goal>failure</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

One interesting thing is important here: @project.version@ which represents the current version of your plugin project.

There are some requirements to use the ITF:

  • JDK 8+
  • Apache Maven 3.1.0 or above.

To get the integration tests running we need to add some parts to the projects pom.xml which is at first the dependencies to ITF:

The dependency com.soebes.itf.jupiter.extension:itf-assertj contains custom assertions of AssertJ in case you want to use AssertJ as your assertion framework. This means you have to include org.assertj:assertj-core as well. If you don’t want to use AssertJ as assertion framework you can omit them both.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
<dependency>
  <groupId>com.soebes.itf.jupiter.extension</groupId>
  <artifactId>itf-extension-maven</artifactId>
  <version>0.9.0</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter-engine</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.soebes.itf.jupiter.extension</groupId>
  <artifactId>itf-assertj</artifactId>
  <version>0.9.0</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.soebes.itf.jupiter.extension</groupId>
  <artifactId>itf-jupiter-extension</artifactId>
  <version>0.9.0</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.assertj</groupId>
  <artifactId>assertj-core</artifactId>
  <scope>test</scope>
</dependency>

You need to add the resource filtering like this:

1
2
3
4
5
6
7
8
9
10
<testResources>
  <testResource>
    <directory>src/test/resources</directory>
    <filtering>false</filtering>
  </testResource>
  <testResource>
    <directory>src/test/resources-its</directory>
    <filtering>true</filtering>
  </testResource>
</testResources>

The first one for src/test/resources might be change based on your own requirements. The second is needed to copy the test projects to the appropriate locations. So now we need to go for the itf-maven-plugin like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<plugin>
  <groupId>com.soebes.itf.jupiter.extension</groupId>
  <artifactId>itf-maven-plugin</artifactId>
  <version>0.9.0</version>
  <executions>
    <execution>
      <id>installing</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>install</goal>
      </goals>
    </execution>
  </executions>
</plugin>

which is responsible to copy your plugin/extension to the correct location and finally the maven-failsafe-plugin to execute the integration tests like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <configuration>
    <systemProperties>
      <maven.version>${maven.version}</maven.version>
      <maven.home>${maven.home}</maven.home>
    </systemProperties>
  </configuration>
  <executions>
  <execution>
    <goals>
      <goal>integration-test</goal>
      <goal>verify</goal>
    </goals>
  </execution>
</executions>
</plugin>

So now you can run the integration tests simply by using:

1
mvn clean verify

After the execution you will find a directory a structure like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
.
└──target/
   └── maven-it/
       └── org/
           └── it/
               └── FirstMavenIT/
                   └── the_first_test_case/
                       ├── .m2/
                       ├── project/
                       │   ├── src/
                       │   ├── target/
                       │   └── pom.xml
                       ├── mvn-stdout.log
                       ├── mvn-stderr.log
                       └── mvn-arguments.log

The directory the_first_test_case represents the test case from our original integration test class FirstMavenIT. The project which has been used to test the plugin is put into the directory project which also contains the target folder after a run of such project. The directory project is the location where you can go into and start manually Maven by using the command line arguments which are stored in mvn-arguments.log. The result of this execution is redirected into mvn-stdout.log and the redirection of stderr is logged into mvn-stderr.log. The directory .m2/repository contains the local cache (aka maven repository) of that build.

So this it is for Part I. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

A full example can be found here: https://github.com/khmarbaise/itf-example-article-part-i

42: The Answer to Life, the Universe and Everything

The story started with a Twitter Post about the JDK method bitCount which is available for Long and Integer types.

If you take a look into the time line, there was a reply https://twitter.com/ScottSelikoff/status/1254185742760280064 by @ScottSelikoff (funny comment of course) who stated 42 as the answer to life, the universe and everything to the magic method which has been followed by Lukas Eder:

For what percentage of longs would that be the correct result?

This thread has inspired me to this article.

So let us summarize that question:

For what percentage of long values is bitCount() == 42?

The first question which arises: What does bitCount do? Let us take a look into the java doc of the function (Integer variant):

Returns the number of one-bits in the two’s complement binary representation of the specified int value. This function is sometimes referred to as the population count.

So in the end bitCount counts the number (as the name implies) of 1s which are in a given value.

Here are some exemplary values for the type Integer:

  • Integer i = 0b1100_0000_0000_0000_0000_0000_0000_0000 bitCount=2
  • Integer i = 0b0000_0000_0000_0000_0000_0000_0000_0011 bitCount=2
  • Integer i = 0b0000_0000_0000_0000_0000_0000_0000_1111 bitCount=4
  • Integer i = 0b1111_1111_1111_0000_0000_0000_0000_0000 bitCount=12
  • Integer i = 0b1010_1010_1010_1010_1010_1010_0000_0000 bitCount=12
  • Integer i = 0b1111_1111_1111_1111_1111_1111_1111_1111 bitCount=32

A very naive way of trying to solve that question would be to write code like this:

1
2
3
4
5
6
7
8
9
BigDecimal pow = BigDecimal.valueOf(2L).pow(64);

Map<Integer, Long> collect = LongStream.range(Long.MIN_VALUE, Long.MAX_VALUE)
  .boxed()
  .collect(groupingBy(Long::bitCount, counting()));

collect.forEach((k, v) -> System.out.printf("k = %4s v=%15s %7.5f\n", k, v, BigDecimal.valueOf(v)
  .divide(pow)
  .multiply(BigDecimal.valueOf(100L))));

The first line will create 2^64 which is equal to 100% and the BigDecimal.valueOf(v).divide(pow).multiply(BigDecimal.valueOf(100L)) will just calculate the percentage of the value (number of values for appropriate bit count) of the resulting map which contains the number.

Ok now we can try that…. Really? Never. This code will run a very very long time…

A long time ago in a galaxy far, far away…

That might be a little long to wait for the result. Ok let’s think a bit about the code. Ah! Yes of course I got it. We should use parallel() for the stream to get it faster.

1
2
3
4
5
6
7
8
9
10
BigDecimal pow = BigDecimal.valueOf(2L).pow(64);

Map<Integer, Long> collect = LongStream.range(Long.MIN_VALUE, Long.MAX_VALUE)
  .boxed()
  .parallel()
  .collect(groupingBy(Long::bitCount, counting()));

collect.forEach((k, v) -> System.out.printf("k = %4s v=%15s %7.5f\n", k, v, BigDecimal.valueOf(v)
  .divide(pow)
  .multiply(BigDecimal.valueOf(100L))));

That version will be faster than the previous one and will take… Sorry but I simply don’t know cause I have not tested it ;–).

Can we make a simpler and faster solution to get a result in the end? Ok we change the Long into Integer? Now the code looks like this. As you see I already added the parallel() in the code to get faster also you see I’m using BigDecimal.valueOf(2L).pow(32) (232) instead of 264 based on the usage of Integer:

1
2
3
4
5
6
7
8
9
10
BigDecimal pow = BigDecimal.valueOf(2L).pow(32);

Map<Integer, Long> collect = IntStream.range(Integer.MIN_VALUE, Integer.MAX_VALUE)
  .boxed()
  .parallel()
  .collect(groupingBy(Integer::bitCount, counting()));

collect.forEach((k, v) -> System.out.printf("k = %4s v=%15s %7.3f\n", k, v, BigDecimal.valueOf(v)
  .divide(pow)
  .multiply(BigDecimal.valueOf(100L))));

So this code will run in ca. 15 seconds (Hexa Core CPU’s) with the following result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
k =    0 v=              1   0.000
k =    1 v=             32   0.000
k =    2 v=            496   0.000
k =    3 v=           4960   0.000
k =    4 v=          35960   0.001
k =    5 v=         201376   0.005
k =    6 v=         906192   0.021
k =    7 v=        3365856   0.078
k =    8 v=       10518300   0.245
k =    9 v=       28048800   0.653
k =   10 v=       64512240   1.502
k =   11 v=      129024480   3.004
k =   12 v=      225792840   5.257
k =   13 v=      347373600   8.088
k =   14 v=      471435600  10.976
k =   15 v=      565722720  13.172
k =   16 v=      601080390  13.995
k =   17 v=      565722720  13.172
k =   18 v=      471435600  10.976
k =   19 v=      347373600   8.088
k =   20 v=      225792840   5.257
k =   21 v=      129024480   3.004
k =   22 v=       64512240   1.502
k =   23 v=       28048800   0.653
k =   24 v=       10518300   0.245
k =   25 v=        3365856   0.078
k =   26 v=         906192   0.021
k =   27 v=         201376   0.005
k =   28 v=          35960   0.001
k =   29 v=           4960   0.000
k =   30 v=            496   0.000
k =   31 v=             31   0.000
k =   32 v=              1   0.000

The first column k= is the bitCount so the first line means:

We have a single value v=1 where the bitCount==0 and 0.000 percent.

Let use take a value on the line for k=16 and v=601,080,390 and 13,995 percent of the values of Integer. So it means for bitCount=16 we have 601,080,390 values which are 13,995 percent of all integer values.

One interesting observation which can be made here is that the number of values is rising up to a maximum value at k=16 (bitCount=16) which you might not have expected. Another thing is that the number of values is going down to the higher number with bitCount=32 etc.

Based on the runtime of this example you could roughly estimate the runtime for the variant with Long: 15 s * 2^32 = 64.424.509.440 seconds / 86400 seconds / day = 745.654 days / 365 days / year which results in ca. 2,000 years. So having a machine which has 1000 times more CPUs it could be dropped down to about 2 years (theoretically) or maybe you could use more power by using GPU’s on AWS cloud but in the end no realizable solution.

So we need to reconsider if there is a faster or easier solution to answer the question? Yes there is one.

The answer can be found by using some mathematics.

Let us take a known example from the above result output:

We have 32 bits (Integer). Now we have 16 bits which should be 0 and 16 bits which should be 1. By expressing that like this:

1
2
3
   32!
   ---------- = 601080390
   16! * 16!

You can calculate that via WolframAlpha and the result is: 601080390 which is exact the number in the above example. Let us check some other values:

1
2
3
   32!
   ---------- = 28048800
   9! * 23!

The resulting value will be two times in the table one for k=9 (bitCount=9) and for k=23 (bitCount=23). The bitCount=9 means 9 1s (and 23 0s) are in the integer value and bitCount=23 means 23 1s (and 9 0) are in the integer values.

So based on the mathematics we could really answer the question via:

1
2
3
   64!
   ---------- = 80,347,448,443,237,920
   42! * 22!

So this means we have 80,347,448,443,237,920 numbers where the bitCount=42 and this means

1
2
3
   80,347,448,443,237,920
   ---------------------- * 100 = 0,435 %
          2^64

So 0,435 percent of all long values having a bitCount=42.

Apache Maven Archetype Plugin 3.2.0 Released

The Apache Maven team is pleased to announce the release of the Apache Maven Archetype Plugin, version 3.2.0.

In short, Archetype is a Maven project templating toolkit. An archetype is defined as an original pattern or model from which all other things of the same kind are made. The names fits as we are trying to provide a system that provides a consistent means of generating Maven projects. Archetype will help authors create Maven project templates for users, and provides users with the means to generate parameterized versions of those project templates.

https://maven.apache.org/archetype/maven-archetype-plugin/index.html

You should specify the version in your project’s plugin configuration:

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-archetype-plugin</artifactId>
  <version>3.2.0</version>
</plugin>

You can download the appropriate sources etc. from the download page.

Apache Maven WAR Plugin Version 3.3.1 Released

The Apache Maven team is pleased to announce the release of the Apache Maven WAR Plugin, version 3.3.1.

The WAR Plugin is responsible for collecting all artifact dependencies, classes and resources of the web application and packaging them into a web application archive.

You should specify the version in your project’s plugin configuration:

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-war-plugin</artifactId>
  <version>3.3.1</version>
</plugin>

You can download the appropriate sources etc. from the download page.

Apache Maven Site Plugin Version 3.9.1 Released

The Apache Maven team is pleased to announce the release of the Apache Maven Site Plugin, version 3.9.1.

The Site Plugin is used to generate a site for the project. The generated site also includes the project’s reports that were configured in the POM.

You can download the appropriate sources etc. from the download page:

https://maven.apache.org/plugins/maven-site-plugin/download.cgi

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-site-plugin</artifactId>
  <version>3.9.1</version>
</plugin>

Apache Maven Shared Component: Maven Reporting Exec Version 1.5.1

The Apache Maven team is pleased to announce the release of the Apache Maven Shared Component: Maven Reporting Exec Version 1.5.1.

Classes to prepare report plugins execution with Maven 3, through MavenReportExecutor (implementation).

You should specify the version in your project’s plugin configuration:

1
2
3
4
5
<dependency>
  <groupId>org.apache.maven.reporting</groupId>
  <artifactId>maven-reporting-exec</artifactId>
  <version>1.5.1</version>
</plugin>

You can download the appropriate sources etc. from the download page.

Apache Maven Surefire Plugin Version 3.0.0-M5 Released

The Apache Maven team is pleased to announce the release of the Apache Maven Surefire Plugin, version 3.0.0-M5.

The release contains 40 bug fixes. Again we received contributions from the community in form of bug reports and bug fixes. Thank you and keep them coming!

You should specify the version in your project’s plugin configuration:

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>3.0.0-M5</version>
</plugin>

or for failsafe:

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>3.0.0-M5</version>
</plugin>

or for surefire-report:

1
2
3
4
5
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-report-plugin</artifactId>
  <version>3.0.0-M5</version>
</plugin>

You can download the appropriate sources etc. from the download page.