scaladoc OOMs while building ScalaTest 3.2.7 docs in community build

[griggt]

https://github.com/lampepfl/dotty/runs/2362395012?check_suite_focus=true#step:5:2235

java.lang.RuntimeException: Doc command exited with code 137 for project scalatest. Project details:
SbtCommunityProject(scalatest,scalacticDotty/clean; scalacticDottyJS/clean; scalacticTestDotty/test; scalatestTestDotty/test; scalacticDottyJS/compile; scalatestDottyJS/compile,List(),List(SbtCommunityProject(scala-xml,xml/test,List(),List(),dotty.communitybuild.SbtCommunityProject$$$Lambda$23/1725097945@7cd62f43,xml/publishLocal,xml/doc,List(-Xcheck-macros, -Ysafe-init),false)),dotty.communitybuild.projects$$$Lambda$27/1458849419@6d4b1c02,scalacticDotty/publishLocal; scalatestDotty/publishLocal; scalacticDottyJS/publishLocal; scalatestDottyJS/publishLocal,;scalacticDotty/doc,List(-Xcheck-macros, -Ysafe-init),false)
	at dotty.communitybuild.CommunityProject.doc(projects.scala:79)
	at dotty.communitybuild.CommunityProject.doc$(projects.scala:35)
	at dotty.communitybuild.SbtCommunityProject.doc(projects.scala:104)
	at dotty.communitybuild.Main$.generateDocs(Main.scala:14)
	at dotty.communitybuild.Main$.$anonfun$6(Main.scala:67)
	at scala.collection.immutable.List.map(List.scala:250)
	at dotty.communitybuild.Main$.main(Main.scala:78)
	at dotty.communitybuild.Main.main(Main.scala)

Sometimes it completes without error.

It works fine for me locally, but I have quite a bit more RAM than the 7GB (?) that I think the GitHub hosted runners provide.

[smarter]

Oh, so exit 137 means that docker was OOM, not the JVM (where the max heap for scalatest is set to 11 GB) ? That's going to be hard to work around.

[griggt]

Yep

[abgruszecki]

OK, this was fixed in #12145. Guillaume said that compiling ScalaTest also takes an excessive amount of memory, so I'm not sure if it makes sense to investigate the problems that Scaladoc has with ScalaTest before the compiler's own problems are fixed.

[abgruszecki]

@smarter do we have an issue for the OOM when compiling ScalaTest? If yes, then maybe we should keep this one open and link it there.

[smarter]

I don't think so

[griggt]

IIRC it's compiling the ScalaTest test suite that uses excessive memory, rather than compiling ScalaTest proper.

[abgruszecki]

Ok, let's at least re-open this issue and check whether Scaladoc's problems are the same as compiler's.

[griggt]

This no longer seems to be an issue if we reduce the JVM max heap size for ScalaTest from 11 GB to 5 GB (which is now possible thanks to #12935).

See #13049.