mardi 21 avril 2015

Building Spark 1.3.1 for Scala 2.11 locally fails

Following its guide, my Spark 1.3.1 installation fails after executing these commands:

$ dev/change-version-to-2.11.sh
$ build/sbt -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package assembly

with the error message

sbt.ResolveException: download failed: org.twitter4j#twitter4j-core;3.0.3!twitter4j-core.jar

Then I tried to install normally:

$ mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package

and I get the error:

[error] Required file not found: sbt-interface.jar
...
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first) on project spark-network-common_2.11: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed. CompileFailed -> [Help 1]

Installing in a standard way:

$ build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

gives me the second error shown above.

Any help would be much appreciated!

Aucun commentaire:

Enregistrer un commentaire