I'm trying to understand below code snippet

import org.apache.spark.sql.functions._

seq : Seq[String] = Seq("A","B")
val op:Seq[Column] = seq.map(col)

my understanding is map is function from trait TraversableLike and Seq[String] is able to call this map function because it has been implemented by Seq class somewhere. I hope upto here is correct.

Now col is function in trait functions and takes a parameter like col(colName: Strig) as per the documentation.

So how come its working just taking col function as parameter but not its respective parameter ?

can anyone please explain how its working behind the scenes ?

Thanks

CodePudding user response：

So how come its working just taking col function as parameter but not its respective parameter ?

The call to seq.map(col) only takes a function as an argument because map will pass each individual element contained in seq to that function (col). Let's walk through the code at a high level to break down what is going on:

1. seq is defined as a Seq[String] containing two strings: "A" and "B"
2. seq.map(col) is called to convert each of the strings contained in seq to a Column instance. map iterates through the contents of seq and does the following:
   • It sees "A" and calls col("A") to produce the first Column value
   • It sees "B" and calls col("B") to produce the second Column value
   • It builds a new Seq[Column] containing all of the Column values generated in the previous steps, and returns it
3. The result of seq.map(col) is stored in the variable op

CodePudding user response：

From the Scaladocs:

map

Builds a new sequence by applying a function to all elements of this sequence.

It has this function signature:

def map[B](f: (A) => B): Seq[B]

Where:

A is the input type of the function f

B is "the element type of the returned sequence."

f is "the function to apply to each element."

and returns "a new sequence resulting from applying the given function f to each element of this sequence and collecting the results."

In this case, you are applying each element in the sequence as an argument to the function col, which expects a String as a parameter. col returns a Column, thus you have mapped your Seq of Strings to a Seq of Columns.

I highly recommend checking out the official Scala documentation:

Seq.map

Also from the docs, maps definition classes are IterableOps → [abstract in] IterableOnceOps