Compilation of Stdlib failed

[liufengyun]

Minimized code

Compile the following two files together (from stdlib) results in compiler crash:

volatile.scala:

package scala

final class volatile extends scala.annotation.StaticAnnotation

VolatileByteRef.java

public final class VolatileByteRef implements java.io.Serializable {
    volatile public byte elem;
}

If we remove volatile keyword in the Java file, everything works fine.

Output (click arrow to expand)

exception caught when loading class volatile: java.lang.AssertionError: assertion failed
exception occurred while typechecking community-build/community-projects/stdLib213/src/library/scala/runtime/VolatileByteRef.java
exception occurred while compiling community-build/community-projects/stdLib213/src/library/scala/runtime/VolatileByteRef.java, community-build/community-projects/stdLib213/src/library/scala/volatile.scala
java.lang.AssertionError: assertion failed while compiling community-build/community-projects/stdLib213/src/library/scala/runtime/VolatileByteRef.java, community-build/community-projects/stdLib213/src/library/scala/volatile.scala
Exception in thread "main" java.lang.AssertionError: assertion failed
at dotty.DottyPredef$.assertFail(DottyPredef.scala:16)
at dotty.tools.dotc.core.unpickleScala2.Scala2Unpickler.(Scala2Unpickler.scala:136)
at dotty.tools.dotc.core.classfile.ClassfileParser.unpickleScala$1(ClassfileParser.scala:781)
at dotty.tools.dotc.core.classfile.ClassfileParser.unpickleOrParseInnerClasses(ClassfileParser.scala:888)
at dotty.tools.dotc.core.classfile.ClassfileParser.parseClass(ClassfileParser.scala:165)
at dotty.tools.dotc.core.classfile.ClassfileParser.run(ClassfileParser.scala:90)
at dotty.tools.dotc.core.ClassfileLoader.load(SymbolLoaders.scala:407)
at dotty.tools.dotc.core.ClassfileLoader.doComplete(SymbolLoaders.scala:402)
at dotty.tools.dotc.core.SymbolLoader.complete(SymbolLoaders.scala:343)
at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:166)
at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeOnce(SymDenotations.scala:344)
at dotty.tools.dotc.core.SymDenotations$SymDenotation.unforcedDecls(SymDenotations.scala:368)
at dotty.tools.dotc.core.SymDenotations$ClassDenotation.typeParamsFromDecls(SymDenotations.scala:1641)
at dotty.tools.dotc.core.SymDenotations$ClassDenotation.typeParams(SymDenotations.scala:1654)
at dotty.tools.dotc.core.TypeApplications$.typeParams$extension(TypeApplications.scala:160)
at dotty.tools.dotc.core.TypeApplications$.typeParamSymbols$extension(TypeApplications.scala:191)
at dotty.tools.dotc.typer.Typer.adaptType$1(Typer.scala:3487)
at dotty.tools.dotc.typer.Typer.adapt1(Typer.scala:3553)
at dotty.tools.dotc.typer.Typer.op$3(Typer.scala:2883)
at dotty.tools.dotc.typer.Typer.adapt(Typer.scala:2884)
at dotty.tools.dotc.typer.Typer.op$1(Typer.scala:2553)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2554)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2557)
at dotty.tools.dotc.typer.Namer.typedAheadType$$anonfun$1(Namer.scala:1223)
at dotty.tools.dotc.typer.Namer.typedAhead(Namer.scala:1215)
at dotty.tools.dotc.typer.Namer.typedAheadType(Namer.scala:1223)
at dotty.tools.dotc.typer.Namer.typedAheadAnnotationClass(Namer.scala:1238)
at dotty.tools.dotc.typer.Namer$Completer.addAnnotations$$anonfun$1(Namer.scala:725)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
at scala.collection.immutable.List.foreach(List.scala:333)
at dotty.tools.dotc.typer.Namer$Completer.addAnnotations(Namer.scala:732)
at dotty.tools.dotc.typer.Namer$Completer.completeInCreationContext(Namer.scala:799)
at dotty.tools.dotc.typer.Namer$Completer.complete(Namer.scala:712)
at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:166)
at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
at dotty.tools.dotc.core.SymDenotations$SymDenotation.ensureCompleted(SymDenotations.scala:352)
at dotty.tools.dotc.typer.Typer.retrieveSym(Typer.scala:2365)
at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:2390)
at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2483)
at dotty.tools.dotc.typer.Typer.op$1(Typer.scala:2553)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2554)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2557)
at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2579)
at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2627)
at dotty.tools.dotc.typer.Typer.typedClassDef(Typer.scala:2081)
at dotty.tools.dotc.typer.Typer.typedTypeOrClassDef$2(Typer.scala:2411)
at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:2415)
at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2483)
at dotty.tools.dotc.typer.Typer.op$1(Typer.scala:2553)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2554)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2557)
at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2579)
at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2627)
at dotty.tools.dotc.typer.Typer.typedPackageDef(Typer.scala:2207)
at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2455)
at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2484)
at dotty.tools.dotc.typer.Typer.op$1(Typer.scala:2553)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2554)
at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2557)
at dotty.tools.dotc.typer.Typer.typedExpr(Typer.scala:2671)
at dotty.tools.dotc.typer.FrontEnd.liftedTree1$1(FrontEnd.scala:78)
at dotty.tools.dotc.typer.FrontEnd.typeCheck$$anonfun$1(FrontEnd.scala:83)
at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at dotty.tools.dotc.typer.FrontEnd.monitor(FrontEnd.scala:42)
at dotty.tools.dotc.typer.FrontEnd.typeCheck(FrontEnd.scala:84)
at dotty.tools.dotc.typer.FrontEnd.runOn$$anonfun$3(FrontEnd.scala:119)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
at scala.collection.immutable.List.foreach(List.scala:333)
at dotty.tools.dotc.typer.FrontEnd.runOn(FrontEnd.scala:119)
at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:181)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
at dotty.tools.dotc.Run.runPhases$5(Run.scala:191)
at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:199)
at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:64)
at dotty.tools.dotc.Run.compileUnits(Run.scala:206)
at dotty.tools.dotc.Run.compileSources(Run.scala:143)
at dotty.tools.dotc.Run.compile(Run.scala:125)
at dotty.tools.dotc.Driver.doCompile(Driver.scala:38)
at dotty.tools.dotc.Driver.process(Driver.scala:195)
at dotty.tools.dotc.Driver.process(Driver.scala:164)
at dotty.tools.dotc.Driver.process(Driver.scala:176)
at dotty.tools.dotc.Driver.main(Driver.scala:203)
at dotty.tools.dotc.Main.main(Main.scala)

Initial diagnosis

The following commit is dubious e88c5b7. Reverting the change fixes the issue:

     final def scalacLinkedClass(using Context): Symbol =
       if (this.is(ModuleClass)) companionNamed(effectiveName.toTypeName)
-      else if (this.isClass) companionNamed(effectiveName.moduleClassName)
+      else if (this.isClass) companionNamed(effectiveName.moduleClassName).sourceModule.moduleClass
       else NoSymbol

Need to dive deeper to the root cause.

[odersky]

It looks like a bootstrap problem. What happens is: When JavaParser reads an annotation, it calls addAnnot, which takes the annotation symbol and puts it in a typed tree. At that time, we still use the previous definition of volatile, loaded from the classfile. Then we typecheck the new definition of volatile and enter it into the symbol table. That sort of double vision leads to the crash.

To fix, it, parsers should never look up symbols from definitions. Instead, they should produce untyped trees with the proper names. Object untpd has a bunch of utility methods that can be used for this:

  def rootDot(name: Name)(implicit src: SourceFile): Select = Select(Ident(nme.ROOTPKG), name)
  def scalaDot(name: Name)(implicit src: SourceFile): Select = Select(rootDot(nme.scala), name)
  def scalaAnnotationDot(name: Name)(using SourceFile): Select = Select(scalaDot(nme.annotation), name)
  def scalaRuntimeDot(name: Name)(using SourceFile): Select = Select(scalaDot(nme.runtime), name)
  def scalaUnit(implicit src: SourceFile): Select = scalaDot(tpnme.Unit)
  def scalaAny(implicit src: SourceFile): Select = scalaDot(tpnme.Any)
  def javaDotLangDot(name: Name)(implicit src: SourceFile): Select = Select(Select(Ident(nme.java), nme.lang), name)

[liufengyun]

Thanks for the tip @odersky . I'll try the solution and see how it works.