Error With H20context Running Pysparkling With Spark 2.1

I'm getting this error when trying to run a Pysparkling script on an AWS EMR cluster. I can get everything to work when downloading Sparkling water 2.1.8 and running it from a pysp

Solution 1:

You do not need to attach Sparkling Water packages (option --packages), but you need to provide pysparkling Python package (it contains all necessary binary dependencies internally).

The best way is to download binary distribution of Sparkling Water from http://h2o.ai/download and use bin/pysparkling script or use spark directly:

$SPARK_HOME/bin/pyspark --py-files h2o_pysparkling_2.1-2.1.8.zip