Cannot generate java-compatible abstract varargs method #10658

marcprux · 2017-12-16T15:25:11Z

I need to generate a java-compatible varargs method in an interface in order to access a JNA native method (which relies on the varargs signature). This appears to still be an issue in 2.12.4, despite some other similar issues (such as #1459) listing this as having been fixed.

10:12 scl$ scalac -version
Scala compiler version 2.12.4 -- Copyright 2002-2017, LAMP/EPFL and Lightbend, Inc.
10:12 scl$ cat Vargs.scala 

trait Vargs {
  @scala.annotation.varargs def varg(args: AnyRef*): Unit
  def noop(): Unit
}

10:12 scl$ scalac Vargs.scala

10:16 scl$ javap Vargs.class 
Compiled from "Vargs.scala"
public interface Vargs {
  public static void varg$(Vargs, java.lang.Object...);
  public void varg(java.lang.Object...);
  public abstract void varg(scala.collection.Seq<java.lang.Object>);
  public abstract void noop();
}

10:12 scl$ cat VargsImpl.java

public class VargsImpl implements Vargs {
    public void noop() {
    }

    public void varg(java.lang.Object... args) {
        System.out.println("varg, matey!");
    }
}

10:12 scl$ javac -cp . VargsImpl.java 
An exception has occurred in the compiler (1.8.0_151). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.lang.NullPointerException
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.visitIdent(Flow.java:2403)
	at com.sun.tools.javac.tree.JCTree$JCIdent.accept(JCTree.java:2011)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scanExpr(Flow.java:1635)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.visitApply(Flow.java:2258)
	at com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1465)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.tree.TreeScanner.visitExec(TreeScanner.java:175)
	at com.sun.tools.javac.tree.JCTree$JCExpressionStatement.accept(JCTree.java:1296)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:57)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.visitBlock(Flow.java:1883)
	at com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:909)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.visitMethodDef(Flow.java:1811)
	at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:778)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.visitClassDef(Flow.java:1749)
	at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:693)
	at com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49)
	at com.sun.tools.javac.comp.Flow$BaseAnalyzer.scan(Flow.java:404)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.scan(Flow.java:1382)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.analyzeTree(Flow.java:2446)
	at com.sun.tools.javac.comp.Flow$AssignAnalyzer.analyzeTree(Flow.java:2429)
	at com.sun.tools.javac.comp.Flow.analyzeTree(Flow.java:211)
	at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1327)
	at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1296)
	at com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901)
	at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860)
	at com.sun.tools.javac.main.Main.compile(Main.java:523)
	at com.sun.tools.javac.main.Main.compile(Main.java:381)
	at com.sun.tools.javac.main.Main.compile(Main.java:370)
	at com.sun.tools.javac.main.Main.compile(Main.java:361)
	at com.sun.tools.javac.Main.compile(Main.java:56)
	at com.sun.tools.javac.Main.main(Main.java:42)

The text was updated successfully, but these errors were encountered:

SethTisue · 2017-12-19T00:50:03Z

How does this (does it even?) relate to #3899?

marcprux · 2017-12-19T16:55:06Z

It is a bit different, since scalac is, in fact, generating the correct Java varargs signature:

public void varg(java.lang.Object...);

but the problem is that it isn't making it abstract; it is generating it as a concrete implementation that (I presume) is calling the generated abstract:

public abstract void varg(scala.collection.Seq<java.lang.Object>);

I'm suspected that this is new as of scala's support for JDK 1.8 interfaces with default implementations. So I tried targeting JDK 1.7 with scalac's 2.11 compiler and I see that the method is indeed output in a way that JNA can use it:

$ scalac -target:jvm-1.7 Vargs.scala && javap Vargs.class
Compiled from "Vargs.scala"
public interface Vargs {
  public abstract void varg(java.lang.Object...);
  public abstract void varg(scala.collection.Seq<java.lang.Object>);
  public abstract void noop();
}

But since "-target:jvm-1.7" is unavailable in scalac 2.12, this isn't a viable workaround.

ekrich · 2018-10-11T13:25:01Z

I am running up against this as well. I have the following method in a trait.

@varargs def checkValid(reference: Config, restrictToPaths: String*): Unit

And try to overrride with the following from Java.

@Override public void checkValid(Config reference, String... restrictToPaths)

I get the following error:

[error] /Users/eric/workspace/config/config/src/main/java/com/typesafe/config
/impl/SimpleConfig.java:41:1: com.typesafe.config.impl.SimpleConfig
 is not abstract and does not override abstract method
 checkValid(com.typesafe.config.Config,scala.collection.Seq<java.lang.String>) in
 com.typesafe.config.Config

See update below.

See scala/bug#10658

ekrich · 2018-11-26T13:51:14Z

My comment above was a misunderstanding about the way the code was generated.

Using @varargs generates one method for Java and one for Scala where in the byte code the Java one delegates to the Scala one.
The Java class that implemented the code needed to implement both the methods since the Java one was already implemented (overridden) and the Scala one needed to delegate to the Java one. This was found because property based testing was calling both methods.
When the implementing class was converted to Scala, the code from the Java one needed to be moved from the Java one to the Scala one as there is no way (AFAIK) to implement a method with the Java varargs signature in Scala. The following does not override Java's String...:
override def checkValid(reference: Config, restrictToPaths: Array[String]): Unit

Once number 3 was complete, only one method was needed. If the forwarding method in Java was not there, property based testing would have found it I think as you can pass an array to a Java varargs method otherwise property based testing would not test it at all from Scala to Java.

I think the key take home is that if you are using @varargs so that a Java class can call the method, the Java class should implement the Scala signature and the Java signature can be left unimplemented. The Java code will have to deal with the signature below and handle the Scala collection class.

void checkValid(com.typesafe.config.Config, scala.collection.Seq<java.lang.String>)

ekrich · 2018-12-05T13:38:07Z

Update: Things are a bit more complex but seem to work. Steps followed.

Converted Java interface to Scala trait with the following method.

public void checkValid(Config reference, String... restrictToPaths)
@varargs def checkValid(reference: Config, restrictToPaths: String*): Unit

The Java class that implements the trait needed both the Java and Scala methods implemented.

@Override public void checkValid(Config reference, String... restrictToPaths)
@Override public void checkValid(Config reference, Seq<String> restrictToPaths)

When changing the implementation from Java to Scala only the Scala one needed implementing.
override def checkValid(reference: Config, restrictToPaths: String*): Unit
This fails at runtime when running a Java application.

[error] (run-main-6) java.lang.AbstractMethodError: com.typesafe.config.impl.SimpleConfig.checkValid(Lcom/typesafe/config/Config;[Ljava/lang/String;)V
[error] java.lang.AbstractMethodError: com.typesafe.config.impl.SimpleConfig.checkValid(Lcom/typesafe/config/Config;[Ljava/lang/String;)V
[error] 	at simplelib.SimpleLibContext.<init>(SimpleLibContext.java:21)
[error] 	at simplelib.SimpleLibContext.<init>(SimpleLibContext.java:27)
[error] 	at SimpleApp.main(SimpleApp.java:22)
[error] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.lang.reflect.Method.invoke(Method.java:498)

You can not override the method with the Java signature but you can add a method to allow the Java code to run.
def checkValid(reference: Config, restrictToPaths: Array[String]): Unit

I didn't investigate the byte code but the solution is not very intuitive but can be used it seems.

ekrich · 2019-09-12T18:25:01Z

Update working code is here. Dead link removed above.
https://github.com/ekrich/sconfig/blob/v1.0.0/sconfig/shared/src/main/scala/org/ekrich/config/impl/SimpleConfig.scala#L981-L991

Annotation in the interface is here. Edit 3-3-2020 - fix link
https://github.com/ekrich/sconfig/blob/v1.0.0/sconfig/shared/src/main/scala/org/ekrich/config/Config.scala#L394

Issue filed with Dotty: scala/scala3#7212

@SethTisue Can this issue be closed?

hrhino added java interop varargs labels Dec 17, 2017

ekrich added a commit to ekrich/config that referenced this issue Oct 11, 2018

Make the build work and minimize the errors down to one.

e9cd99d

See scala/bug#10658

SethTisue added this to the Backlog milestone Oct 11, 2018

ekrich added a commit to ekrich/config that referenced this issue Nov 16, 2018

Make the build work and minimize the errors down to one.

8f8c12c

See scala/bug#10658

ekrich mentioned this issue Sep 12, 2019

@vararg annotation when called from Java does not compile scala/scala3#7212

Closed

SethTisue closed this as completed Sep 12, 2019

SethTisue removed this from the Backlog milestone Sep 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot generate java-compatible abstract varargs method #10658

Cannot generate java-compatible abstract varargs method #10658

marcprux commented Dec 16, 2017

SethTisue commented Dec 19, 2017 •

edited

Loading

marcprux commented Dec 19, 2017

ekrich commented Oct 11, 2018 •

edited

Loading

ekrich commented Nov 26, 2018 •

edited

Loading

ekrich commented Dec 5, 2018

ekrich commented Sep 12, 2019 •

edited

Loading

Cannot generate java-compatible abstract varargs method #10658

Cannot generate java-compatible abstract varargs method #10658

Comments

marcprux commented Dec 16, 2017

SethTisue commented Dec 19, 2017 • edited Loading

marcprux commented Dec 19, 2017

ekrich commented Oct 11, 2018 • edited Loading

ekrich commented Nov 26, 2018 • edited Loading

ekrich commented Dec 5, 2018

ekrich commented Sep 12, 2019 • edited Loading

SethTisue commented Dec 19, 2017 •

edited

Loading

ekrich commented Oct 11, 2018 •

edited

Loading

ekrich commented Nov 26, 2018 •

edited

Loading

ekrich commented Sep 12, 2019 •

edited

Loading