Scala-cli tries to read classfiles from the current directory when not asked for it

[prolativ]

Version(s)
0.1.18

Describe the bug
Scala-cli tries to read classfiles from the current directory when not asked for it

To Reproduce

mkdir xyz
cd xyz
echo '@main def run() = println("Hello!")' > Hello.scala
scala-cli compile -d out -S 3.2.1 Hello.scala
cp out/* .
scala-cli repl -S 3.1.3

scala> println()

Output:

error while loading Hello$package$,
class file Hello$package.class is broken, reading aborted with class dotty.tools.tasty.UnpickleException
TASTy signature has wrong version.
 expected: {majorVersion: 28, minorVersion: 1}
 found   : {majorVersion: 28, minorVersion: 2}

This TASTy file was produced by a more recent, forwards incompatible release.
To read this TASTy file, please upgrade your tooling.
The TASTy file was produced by Scala 3.2.1.
1 error found

Expected behaviour
Scala-cli should ignore classfiles in the current directory unless they are explicitly added to the classpath. scala command itself doesn't have this problem.

[lwronski]

@prolativ Thanks for reporting,

I'm not convinced this is a problem on the scala-cli side, when you run scala repl via cs I got the same issue as in scala-cli:

➜  xyz git:(main) ✗ cs launch scala:3.1.1 
Welcome to Scala 3.1.1 (17.0.2, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                                                                                                                        
scala> println()
error while loading Hello$package$,
class file Hello$package.class is broken, reading aborted with class dotty.tools.tasty.UnpickleException
TASTy signature has wrong version.
 expected: {majorVersion: 28, minorVersion: 1}
 found   : {majorVersion: 28, minorVersion: 2}

This TASTy file was produced by a more recent, forwards incompatible release.
To read this TASTy file, please upgrade your tooling.
The TASTy file was produced by Scala 3.2.1.
1 error found

[lwronski]

As a workaround, in scala-cli we can run repl in tmp directory. This should fix the problem. @prolativ What do you think about this idea?

[prolativ]

I hope you didn't mean /tmp 😆. A temporary directory might be an option but then should scala-cli create a new one for each invocation of scala-cli repl? And would that change the current working directory from a user's perspective? That would be a problem e.g. if someone tried to use the repl for interacting with the file system

[prolativ]

And IMO it is an issue with scala-cli (and with coursier as well if you say so). If I run

mkdir xyz
cd xyz
echo '@main def run() = println("Hello!")' > Hello.scala
scala-cli compile -d out -S 3.2.1 Hello.scala
cp out/* .

and then start the REPL for scala 3.1.3 using sdkman

sdk use scala 3.1.3
scala

and then I run

scala> println()

then I don't get the error

[lwronski]

@prolativ Thanks for response, I investigated this in more detail. After checking the help documentation for the scala runner from dotty, I found some interesting options that might be useful for us.

     -classpath  Specify where to find user class files.
                    Default .

and then I tried using the --classpath option and passed cwd (.).

➜  xyz git:(main) ✗ scala -classpath .
Welcome to Scala 3.1.3 (17.0.2, Java OpenJDK 64-Bit Server VM).
Type in expressions for evaluation. Or try :help.
                                                                                                                                                                                                                                        
scala> println()
error while loading Hello$package$,
class file Hello$package.class is broken, reading aborted with class dotty.tools.tasty.UnpickleException
TASTy signature has wrong version.
 expected: {majorVersion: 28, minorVersion: 1}
 found   : {majorVersion: 28, minorVersion: 2}

This TASTy file was produced by a more recent, forwards incompatible release.
To read this TASTy file, please upgrade your tooling.
The TASTy file was produced by Scala 3.2.1.
1 error found
                                                                                                                                                                                                                                        
scala> 

Then, I got the same error. In this case, I used the scala that was installed by the sdk.

It appears that the repl from scala-cli works as described in the help documentation, with classpath set to cwd. It might be a problem with scala installed by sdk, because the --classpath option isn't set to the cwd.

It seems that this isn't a bug. The repl should read the class files from the current directory, as described in the help documentation.

[prolativ]

At this point I'm not sure myself what behaviour we should expect then. It seems that scala installed by sdk is not special in any way - I tried downloading scala 3.1.3 directly from a github release and when I start the repl from the binary in xyz then I still get no error unless I explicitly add -classpath ., which is strange, because the help message claims . is the default value for -classpath

[lwronski]

It looks like there is a bug in the scala runner. There is a difference in the classPath options when running scala runner as a JVM or a binary. Therefore, it doesn't seem to be a scala-cli bug and I think we can close this issue.

[prolativ]

It looks like this behaviour started in scala 3.1.0 but it seems to be unintentional. Before that (including scala 2) scala indeed did add . to the classpath by default.
So we might leave the current behaviour of scala-cli for consistency between scala versions.
However it's quite strange that now if there are some classfiles in the current working directory, scala-cli repl will add them to the classpath but scala-cli repl . will not

[lwronski]

So we might leave the current behaviour of scala-cli for consistency between scala versions.
However it's quite strange that now if there are some classfiles in the current working directory, scala-cli repl will add them to the classpath but scala-cli repl . will not

I have new observation that explains the difference between the scala-cli repl and scala-cli repl . commands. Let's start with the first command:

$ scala-cli repl
// here is function to print classpath
println(urls.filterNot(_.toString.contains("ivy")).mkString("\n"))
jrt:/
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.2.1/scala3-library_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-compiler_3/3.2.1/scala3-compiler_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-interfaces/3.2.1/scala3-interfaces-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/tasty-core_3/3.2.1/tasty-core_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/9.3.0-scala-1/scala-asm-9.3.0-scala-1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/compiler-interface/1.3.5/compiler-interface-1.3.5.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.19.0/jline-reader-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.19.0/jline-terminal-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.19.0/jline-terminal-jna-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.7.0/protobuf-java-3.7.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/util-interface/1.3.0/util-interface-1.3.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.3.1/jna-5.3.1.jar
file:/private/tmp/scala-demo/xyz/./

so classes from the current directory were added here - file:/private/tmp/scala-demo/xyz/./.

So now let's check the classpath for scala-cli repl .

$ scala-cli repl .
scala> println(urls.filterNot(_.toString.contains("ivy")).mkString("\n"))
jrt:/
file:/private/tmp/scala-demo/xyz/.scala-build/project_103be31561/classes/main/
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.2.1/scala3-library_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-compiler_3/3.2.1/scala3-compiler_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-interfaces/3.2.1/scala3-interfaces-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/tasty-core_3/3.2.1/tasty-core_3-3.2.1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/9.3.0-scala-1/scala-asm-9.3.0-scala-1.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/compiler-interface/1.3.5/compiler-interface-1.3.5.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.19.0/jline-reader-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.19.0/jline-terminal-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.19.0/jline-terminal-jna-3.19.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.7.0/protobuf-java-3.7.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/util-interface/1.3.0/util-interface-1.3.0.jar
file:/Users/lwronski/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.3.1/jna-5.3.1.jar
file:/private/tmp/scala-demo/xyz/./

the same as before, cwd is on the classpath - file:/private/tmp/scala-demo/xyz/./. But this time, the classpath also includes file:/private/tmp/scala-demo/xyz/.scala-build/project_103be31561/classes/main/. This classes is compiled by the scala-cli using specified scala version, in options.

The classpath contains the same classes but they are compiled with different versions of Scala, therefore the second command works for you, because repl uses classes compiled by scala-cli, but not from cwd.

[lwronski]

@julienrf I think that this issue is interest from the SIP perspective.

I tried to shortly explain the problem. In scala help, we can find the following entry:

     -classpath  Specify where to find user class files.
                    Default .

so I could be surprised for some users, as it was described in the issue. Additionally, it appears that the scala runner has a regression in options -classpath behavior since version 3.1.0

It looks like this behaviour started in scala 3.1.0 but it seems to be unintentional. Before that (including scala 2) scala indeed did add . to the classpath by default.
So we might leave the current behaviour of scala-cli for consistency between scala versions.
However it's quite strange that now if there are some classfiles in the current working directory, scala-cli repl will add them to the classpath but scala-cli repl . will not

@julienrf what is your opinion on -classpath option in the scala runner? Should it include classes from cwd to classPath by default?

[julienrf]

I don’t think -classpath should default to . (by the way, java does not do that either). If nobody complains about the “regression” in Scala 3, it probably means that few people rely on that anyway?

[lwronski]

It appears, that only few people rely on -classpath.

@julienrf, would you be able to report this bug to the compiler team and ask them for change the behavior of the -classpath option? I will close this issue as it works from the Scala CLI side and requires action from the compiler team.

[julienrf]

Done in scala/scala3#16715