Skip to content

Test generation fails for Optional<T> class in the plugin #226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dtim opened this issue Jun 16, 2022 · 0 comments · Fixed by #305
Closed

Test generation fails for Optional<T> class in the plugin #226

dtim opened this issue Jun 16, 2022 · 0 comments · Fixed by #305
Assignees
Labels
comp-symbolic-engine Issue is related to the symbolic execution engine ctg-bug Issue is a bug priority-top-focus Top priority chosen by dev team

Comments

@dtim
Copy link
Collaborator

dtim commented Jun 16, 2022

Description

No test cases are generated for methods involving Java Optional<T> class. UtErrors are generated by the engine during traverse due to ClassNotFound exception: UtOptional class can't be loaded. This behavior may be reproduced using the Idea plugin, but models are successfully generated for the same code in the engine unit tests.

To Reproduce

Generate unit tests for the sample method:

import java.util.Optional;

public class OptionalExamples {
    public Optional<Integer> nonEmptyIfPositive(int n) {
        if (n > 0) {
            return Optional.of(n);
        } else {
            return Optional.empty();
        }
    }
}

Expected behavior

Two tests are generated, one for each branch in the code (see Additional context for more details).

Actual behavior

Errors are reported:

public class OptionalExamplesTest {
    ///region Test suites for executable collections.OptionalExamples.nonEmptyIfPositive


    ///region Errors report for nonEmptyIfPositive

    public void testNonEmptyIfPositive_errors() {
        // Couldn't generate some tests. List of errors:
        // 
        // 2 occurrences of:
        // org.utbot.engine.overrides.collections.UtOptional

    }
    ///endregion

    ///endregion
}

Environment

Mockito: any configuration. Test framework: JUnit5.

Additional context

The primary error reason: UtOptional wrapper is used instead of Optional, but the class loader used by the test generator in the plugin can't find it. It's OK (engine-specific wrappers should never leak to the user), but shouldMock function checks the class and fails with ClassNotFound exception.

Adding a check to shouldMock fixes the exception, but the execution path corresponding to Optional.empty() return value is lost unless checkNpeForFinalFields setting is set to true (it is false by default). See SootField.shouldBeNotNull() declared in Extensions.kt.

A similar code sample that uses OptionalInt instead of Optional<Integer> is processed without any problems, because primitive field types are never mocked so no class check is performed.

Code sample:

public OptionalInt nonEmptyOptionalIntIfPositive(int n) {
    if (n > 0) {
        return OptionalInt.of(n);
    } else {
        return OptionalInt.empty();
    }
}

Generated tests:

///region SUCCESSFUL EXECUTIONS for method nonEmptyOptionalIntIfPositive(int)

/**
 * <pre>
 * Test executes conditions:
 *     {@code (n > 0): True }
 * invokes:
 *     OptionalInt::of once
 * returns from: {@code return OptionalInt.of(n); }
 * </pre>
 */
@Test
@DisplayName("nonEmptyOptionalIntIfPositive: n > 0 : True -> return OptionalInt.of(n)")
public void testNonEmptyOptionalIntIfPositive_NGreaterThanZero() {
    OptionalExamples optionalExamples = new OptionalExamples();

    OptionalInt actual = optionalExamples.nonEmptyOptionalIntIfPositive(1);

    OptionalInt expected = of(1);

    assertTrue(deepEquals(expected, actual));
}

/**
 * <pre>
 * Test executes conditions:
 *     {@code (n > 0): False }
 * </pre>
 */
@Test
@DisplayName("nonEmptyOptionalIntIfPositive: -> n > 0 : False")
public void testNonEmptyOptionalIntIfPositive_NLessOrEqualZero() throws ClassNotFoundException, Exception {
    Class optionalIntClazz = Class.forName("java.util.OptionalInt");
    OptionalInt prevEMPTY = ((OptionalInt) getStaticFieldValue(optionalIntClazz, "EMPTY"));
    try {
        OptionalInt empty = empty();
        setStaticField(optionalIntClazz, "EMPTY", empty);
        OptionalExamples optionalExamples = new OptionalExamples();

        OptionalInt actual = optionalExamples.nonEmptyOptionalIntIfPositive(0);


        assertTrue(deepEquals(empty, actual));
    } finally {
        setStaticField(OptionalInt.class, "EMPTY", prevEMPTY);
    }
}
///endregion
@dtim dtim added ctg-bug Issue is a bug comp-symbolic-engine Issue is related to the symbolic execution engine labels Jun 16, 2022
@dtim dtim self-assigned this Jun 16, 2022
@korifey korifey moved this to Todo in UTBot Java Jun 16, 2022
@dtim dtim moved this from Todo to In Progress in UTBot Java Jun 21, 2022
@denis-fokin denis-fokin added the priority-top-focus Top priority chosen by dev team label Jun 22, 2022
dtim added a commit that referenced this issue Jun 24, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 27, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 27, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 28, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 28, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 28, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
dtim added a commit that referenced this issue Jun 28, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
@dtim dtim closed this as completed in #305 Jun 28, 2022
dtim added a commit that referenced this issue Jun 28, 2022
Added a check for overridden classes in `shouldMock` to avoid
access to engine classes that are not available in the plugin.

Implemented more accurate speculative marking of final fields
as not null to avoid losing paths involving `Optional.empty()`,
to enable NPE checks for final fields in user code, and to avoid
generating non-informative NPE tests for final fields in system
classes.

UtSettings.checkNpeForFinalFields is now set default (false)
in `AbstractTestCaseGEneratorTest` and `SummaryTestCaseGeneratorTest`.
Repository owner moved this from In Progress to Done in UTBot Java Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-symbolic-engine Issue is related to the symbolic execution engine ctg-bug Issue is a bug priority-top-focus Top priority chosen by dev team
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants