"Java member module without member class" when a directory with text file exists in classpath

[eed3si9n]

This was reported originally as "broken test:compile when a package with same name is present in resources folder" sbt/sbt#4517 by @tyrcho

steps

Unzip demo.zip which contains minimal sources to reproduce the issue.

build.sbt

organization := "com.github.writethemfirst"
version := "0.1-SNAPSHOT"
scalaVersion := "2.12.8"

project/build.properties

sbt.version=1.2.8

src/main/scala/com/demo/file/FileContentTest.scala

package com.demo.file

class FileContentTest extends App {
  println("hellsd qs")
}

src/main/resources/com/demo/file/FileContentTest/toto/aa.txt

Empty file.

Run

sbt Test/compile

problem

Project cannot be compiled and the error message is cryptic:

[info] Loading project definition from /private/tmp/approvals-demo-scala/project
[info] Loading settings for project approvals-demo-scala from build.sbt ...
[info] Set current project to approvals-demo-scala (in build file:/private/tmp/approvals-demo-scala/)
[info] Compiling 1 Scala source to /private/tmp/approvals-demo-scala/target/scala-2.12/classes ...
[error] Error while emitting FileContentTest.scala
[error] assertion failed:
[error]   Java member module without member class: package toto - List(package toto)
[error]      while compiling: /private/tmp/approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
[error]         during phase: jvm
[error]      library version: version 2.12.8
[error]     compiler version: version 2.12.8
[error]   reconstructed args: -bootclasspath /Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/classes:/Users/michel/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar -classpath /private/tmp/approvals-demo-scala/target/scala-2.12/classes
[error]
[error]   last tree to typer: TypeTree(class ListBuffer)
[error]        tree position: line 3 of /private/tmp/approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
[error]             tree tpe: scala.collection.mutable.ListBuffer[() => Unit]
[error]               symbol: final class ListBuffer in package mutable
[error]    symbol definition: final class ListBuffer extends AbstractBuffer with Buffer with GenericTraversableTemplate with BufferLike with ReusableBuilder with SeqForwarder with Serializable (a ClassSymbol)
[error]       symbol package: scala.collection.mutable
[error]        symbol owners: class ListBuffer
[error]            call site: constructor FileContentTest$delayedInit$body in class FileContentTest$delayedInit$body in package file
[error]
[error] == Source file context for tree position ==
[error]
[error]      1 package com.demo.file
[error]      2
[error]      3 class FileContentTest extends App {
[error]      4   println("hellsd qs")
[error]      5 }
[error]      6
[error] Error while emitting FileContentTest.scala
[error] assertion failed: ClassBType.info not yet assigned: Lcom/demo/file/FileContentTest;
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 3 s, completed Jan 25, 2019 10:18:46 AM

expectation

sbt should compile the sources and copy the content from resources.

note

When I run it with debug log, here's what I see:

[info] Done updating.
[debug] Copy resource mappings:
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com)
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com/demo,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com/demo)
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com/demo/file,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com/demo/file)
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com/demo/file/FileContentTest,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com/demo/file/FileContentTest)
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com/demo/file/FileContentTest/toto,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com/demo/file/FileContentTest/toto)
[debug] 	(/private/tmp/approvals-demo-scala/src/main/resources/com/demo/file/FileContentTest/toto/aa.txt,/private/tmp/approvals-demo-scala/target/scala-2.12/classes/com/demo/file/FileContentTest/toto/aa.txt)
...

[debug] [zinc] The Scala compiler is invoked with:
[debug] 	-bootclasspath
[debug] 	/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/classes:/Users/eed3si9n/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar
[debug] 	-classpath
[debug] 	/private/tmp/approvals-demo-scala/target/scala-2.12/classes
[error] Error while emitting FileContentTest.scala
[error] assertion failed:
[error]   Java member module without member class: package toto - List(package toto)
[error]      while compiling: /private/tmp/approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
[error]         during phase: jvm
[error]      library version: version 2.12.8
[error]     compiler version: version 2.12.8
[error]   reconstructed args: -bootclasspath /Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home/jre/classes:/Users/eed3si9n/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar -classpath /private/tmp/approvals-demo-scala/target/scala-2.12/classes

In other words, resource copying happens concurrently with the compilation when Test/compile is invoked, and the compiler gets confused by the directory structure that happens to have the same name as the Scala file it's about to compile /com/demo/file/FileContentTest/, /com/demo/file/FileContentTest/toto/, etc..

The assertion happens here:
https://github.com/scala/scala/blob/c4575fd03e27b54d187e4c81fd3aa1e0d7b219c9/src/compiler/scala/tools/nsc/backend/jvm/BTypesFromSymbols.scala#L414

Compiling using Scala 2.11.12 works: sbt "++2.11.12\!" Test/compile

[som-snytt]

Normally, it notices both in the classpath dir; this error is when only the spurious dir is on the classpath.

➜  t11385 scalac -d classes approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
➜  t11385 scalac -cp classes approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala:1: error: package file contains object and package with same name: FileContentTest
one of them needs to be removed from classpath
package com.demo.file
                 ^
one error found
➜  t11385 rm classes/com/demo/file/*.class
➜  t11385 scalac -d /tmp -cp classes approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
error: Error while emitting FileContentTest.scala
assertion failed:
  Java member module without member class: package toto - List(package toto)
     while compiling: approvals-demo-scala/src/main/scala/com/demo/file/FileContentTest.scala
        during phase: jvm

[som-snytt]

Not sure if the localized fix is great, but the other behavior ("one of them needs to be removed") should be ameliorated in general: an empty directory is not a package! If I have a random util dir in my cwd, it shouldn't shadow scala.util!

[SethTisue]

If I have a random util dir in my cwd, it shouldn't shadow scala.util

see also scala/scala3#7635

[som-snytt]

The underlying issue is #6039