Testability: add log records on test generation phases

[alisevych]

Description

In order to check phases and their duration we need to have log info records.
Currently we can measure only overall test generation process.
It's needed to analyze which phase is time consuming.

Expected behavior

Add info log records for different phases of UnitTestBot test generation.

• Test generation initiated
• Collecting information (classes)
• Fuzzing phase
• Symbolic analysis phase
• Minimization
• Summaries
• Code generation

Environment

IntelliJ IDEA (probably add records to idea.log?)

Potential alternatives

or separate UnitTestBot log ?

Context

Originally posted by @alisevych in #1028 (comment)

[Vassiliy-Kudryashov]

Seems to be fixed in branch Vassiliy-Kudryashov/IDEA_hangs_during_test_generation_#1207

[Domonion]

New log system description

Based on meeting with @korifey and @alisevych

Target users for whom it should be convenient:

1. Developers - runIde - utbot-intellij/log4j2.xml
2. QA - built plugin - settings from ~/.utbot/settings.properties
3. User scenario logs - built plugin- ~/.utbot/settings.properties

Settings from utsettings:

1. Log level for concrete executor
2. Log config path for engine and concrete executor

Requirements:

1. Users should not send full utbot directory in issue - only useful logs from engine, idea and related concretes
2. Always logs for concrete, engine and idea
3. Info by default
4. Easy to lift loglevel to debug/trace - using ~/.utbot/settings.properties
5. Easy to navigate to log files
6. Do not use console appender as it pollutes stdout and slows os - use file everywhere. Most probably RollingFileAppender.
7. For each concrete executor it should be easy to deduce from which moment in engine process this concrete executor
   was started - from fuzzing/concrete and/or flex
8. Docs about all logging settings and example of usages
9. Navigation from one logfile to another should be easy - for example by click as in Rider
10. May be idea debug log for changing plugin log4j2.xml without rebuilding plugin
11. Concrete executor prints log to stdout which is redirected to file, may be better to file directly

Useful:

1. parralel generations are prohibited! - file appender is suitable

[alisevych]

Additional information

Requirements

1. Users cannot upload files to GitHub via browser larger than 25MB
   It can be LOG or ZIP - here is the github docs
   If files were larger, a workaround - Dropbox for example - would be needed.

[Domonion]

Discussed with @alisevych:
It is required to add more entries about phases execution, at least to indicate when each phase started and ended.

AFAIK following people should be notified:
@amandelpie @Vassiliy-Kudryashov @Markoutte @EgorkaKulikov

In case you need help with current logging - see doc and ask me

[denis-fokin]

As far as most work is delivered to master. The rest will complete in future release.

[amandelpie]

The logging for summaries was not implemented