Apologies this is probably something very obvious. I don't understand why this isn't working:

val df = spark.createDataFrame([("A",1),("B",2),("C",3)],["Letter","Number"])

When I run this I get the below error:

error: illegal start of simple expression

Why isn't this working?

CodePudding user response：

This is Pyspark syntax not Spark Scala. Try this instead:

val df = spark.createDataFrame(Seq(("A",1),("B",2),("C",3))).toDF("Letter", "Number")

Or:

import spark.implicits._

val df = Seq(("A",1),("B",2),("C",3)).toDF("Letter", "Number")