Find JUnit tests without assertions

1. Juli 2014 6 Kommentare

As a follow up of the last post Find JUnit assertions without a message another useful constraint can be created using a set of cypher queries.

How can we find all test implementations which do not perform any assertion?

Here’s an example Java class:

  @Test
  public void test() {
    // prepare data
    int a = 1;
    int b = 2;
    // call the method which shall be tested
    int result = myService.add(a, b);
    // no verification if the result is correct
  }

The following set of jQAssistant rules report such kind of test implementations, they consist of two concepts and the constraint:

    <concept id="junit4:TestClassOrMethod">
        <description>Finds test methods (i.e. annotated with
        "@org.junit.Test") and labels them and their containing
        classes with "Test" and "Junit4".</description>
        <cypher><![CDATA[
            match
              (c:Type:Class)-[:DECLARES]->(m:Method),
              (m)-[:ANNOTATED_BY]-()-[:OF_TYPE]->(a:Type)
            where
              a.fqn="org.junit.Test"
            set
              c:Test:Junit4, m:Test:Junit4
            return
              c as TestClass, collect(m) as TestMethods
        ]]></cypher>
    </concept>

    <concept id="junit4:AssertMethod">
        <description>Labels all assertion methods declared
          by org.junit.Assert with "Assert".</description>
        <cypher><![CDATA[
            match
                (assertType:Type)-[:DECLARES]->(assertMethod)
            where
                assertType.fqn = 'org.junit.Assert'
                and assertMethod.signature =~ 'void assert.*'
            set
                assertMethod:Assert
            return
                assertMethod
            ]]></cypher>
    </concept>

    <constraint id="junit4:TestMethodWithoutAssertion">
        <requiresConcept refId="junit4:TestClassOrMethod"/>
        <requiresConcept refId="junit4:AssertMethod"/>
        <description>All test methods must perform assertions.</description>
        <cypher><![CDATA[
            match
              (testType:Type)-[:DECLARES]->(testMethod:Test:Method)
            where
              not (testMethod)-[:INVOKES*]->(:Method:Assert)
            return
              testType as DeclaringType,
              testMethod as Method
        ]]></cypher>
    </constraint>

The concept „junit4:TestClassOrMethod“ adds a label „Test“ to all test methods annotated with @org.junit.Test, the concept „junit4:AssertMethod“ adds a label „Assert“ to all assert methods provided by org.junit.Assert.

Both are required by the constraint „junit4:TestMethodWithoutAssertion“ which does nothing more than checking if within the call graph starting at a test method (i.e. traversal over all outgoing INVOKE relations) at least one „Assert“ labeled method can be found.

The constraint has been added to the jQAssistant JUnit4 plugin and thus will be officially available with the next release.

About the author:

Dirk Mahler

@dirkmahler

Tags: JUnit

6 Comments

Nico - 24. November 2017 - 21:01

Antworten /

Unfortunately, this constraint produces a false positive if the test method does not have an assertion, but has the @Test’s attribute ‚expected‘ set which should be valid IMO. Any clue how to adapt the AssertMethod concept to cope with that? Thanks!

Nico - 27. November 2017 - 9:12

Antworten /

In my case, it was sufficient to extend the concept ‚junit4:AssertMethod‘ above to include ‚fail‘ method invocations, i.e. extend the condition above as follows:

where
assertType.fqn = ‚org.junit.Assert‘
and (assertMethod.signature =~ ‚void assert.*‘ or assertMethod.signature =~’void fail.*‘)

Dirk Mahler - 27. November 2017 - 10:58

Antworten /

Hi Nico,

you’re right, the „expected“ attribute of @Test could be considered as an assertion. Therefore another concept could be introduced, e.g. „junit4:ExpectedException“:

match
  (testType:Type)-[:DECLARES]->(testMethod:Test:Method),
  (testMethod)-[:ANNOTATED_BY]->(a)-[:OF_TYPE]->(:Type{fqn:"org.junit.Test"}),
  (a)-[:HAS]->(v:Value{name:"expected"})-[:IS]->(expectedException:Type)
MERGE
  (testMethod)-[:EXPECTS_EXCEPTION]->(expectedException)
return
  testType as DeclaringType,
  testMethod as Method,
  expectedException as ExpectedException

Now the constraint „junit4:TestMethodWithoutAssertion“ can be enhanced:

match
  (testType:Type)-[:DECLARES]->(testMethod:Test:Method)
where not (
  (testMethod)-[:INVOKES*]->(:Method:Assert)
  or (testMethod)-[:EXPECTS_EXCEPTION]->(:Type)
)
return
  testType as DeclaringType,
  testMethod as Method

Would be a good addition to the pre-defined concepts!

Cheers,

Dirk

PS: Actually I dislike the „expected“ attribute because you cannot verify if the exception has been thrown for the right reason (e.g. by checking the message or a contained status code, etc.)

Nico - 27. November 2017 - 17:47

Antworten /

Great. Thanks a ton!

BTW, I do agree that using @Test’s expected attribute is far from perfect.

Sid - 7. August 2018 - 17:07

Antworten /

Hi,
Can we use JQAssistant for verifying „verify“ method? It is present in org.mockito.Mockito. below is my concept but i not sure it works

match
(assertType:Type)-[:DECLARES]->(assertMethod)
where
assertType.fqn = ‚org.mockito.Mockito‘
and assertMethod.signature =~ ‚verify*‘
set
assertMethod:Mockito
return
assertMethod

Dirk Mahler - 9. August 2018 - 9:31

Antworten /

This should work but I’d recommend to add a more specific label „Assert“ or „Verify“ in addition to „Mockito“, i.e.

... set assertMethod:Mockito:Verify return ...