Val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split
Val date=data (0)
Val vid=data (11)
Val duration=data (16)
(date, vid, duration)
}). The filter (t=& gt; T. _1! T==null). The filter (& gt; T. _2! T==null). The filter (& gt; T. _3!=null.) take (10)

Do I want to select three elements are not empty triples how to write, so I write short of purpose, aunt for help

CodePudding user response:

The filter screen is suitable for the key - value such RDD
You can

 val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split 
Val date=data (0) 
Val vid=data (11) 
Val duration=data (16) 
Val key=date 
Val value=https://bbs.csdn.net/topics/vid+, "" + duration 
(key, value) 
}). The filter (t=& gt; T. _1! T==null). The filter (& gt; (t. _2. The split () ", ") (0). T==null). The filter (& gt; (t. _2. The split () ", ") (1)!=null.) take (10) 

CodePudding user response:

 val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split 
Val date=data (0) 
Val vid=data (11) 
Val duration=data (16) 
(date, vid, duration) 
}) filter (RDD=& gt; {
If (RDD _1!=null & amp; & RDD _2!=null & amp; & RDD _3!=null) 
{
True 
} the else 
{
False 
} 

}.) take (10) 

It can be try

CodePudding user response:

Val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split
Val date=data (0)
Val vid=data (11)
Val duration=data (16)
(date, vid, duration)
}). The filter (t=& gt; T. _1!=null& & T. _2!=null & amp; & T. _3!=null.) take (10)

CodePudding user response:

Suggest using third-party tools, confused here recommended tools: https://hutool.cn/docs/#/
Val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split
Val date=data (0)
Val vid=data (11)
Val duration=data (16)
(date, vid, duration)
}) filter (tp=& gt; StrUtil. IsNotBlack (tp) _1) & amp; & StrUtil. IsNotBlack (tp) _2) & amp; & StrUtil. IsNotBlack (tp) _3).) take (10)

CodePudding user response:

Val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split
Val date=data (0)
Val vid=data (11)
Val duration=data (16)
(date, vid, duration)
}). The filter (t=& gt; T. _1!=null& & T. _2!=null& & T. _3!=null)

CodePudding user response:

What has kept you so busy lately

CodePudding user response:

The

references a layer on the second floor flat response:

 val result1=RDD. The map (x=& gt; {
Val data=(", "https://bbs.csdn.net/topics/x.split 
Val date=data (0) 
Val vid=data (11) 
Val duration=data (16) 
(date, vid, duration) 
}) filter (RDD=& gt; {
If (RDD _1!=null & amp; & RDD _2!=null & amp; & RDD _3!=null) 
{
True 
} the else 
{
False 
} 

}.) take (10) 

Can try

this can

CodePudding user response:

The second line: val data=https://bbs.csdn.net/topics/x.split (", ")
That data type Array [String]
the data is obtained by the split it is impossible to contain null, only
So, want to XXX!=null is anyway impossible to filter filtration of any of the data, even if it is an empty string, too!=null

The filter (t=& gt; T. _1! T==null). The filter (& gt; T. _2! T==null). The filter (& gt; T. _3!=null)
If (RDD _1!=null & amp; & RDD _2!=null & amp; & RDD _3!=null)
That no matter how change in two ways, expression results are true, all data will be retained,

CodePudding user response:

Simon, 9/f,

reference brag __Linux reply:

the second line: val data=https://bbs.csdn.net/topics/x.split (", ")
That data type Array [String]
the data is obtained by the split it is impossible to contain null, only
So, want to XXX!=null is anyway impossible to filter filtration of any of the data, even if it is an empty string, too!=null

The filter (t=& gt; T. _1! T==null). The filter (& gt; T. _2! T==null). The filter (& gt; T. _3!=null)
If (RDD _1!=null & amp; & RDD _2!=null & amp; & RDD _3!=null)
That no matter how change in two ways, expression results are true, all data will be retained,

The truth

CodePudding user response:

The filter (t=& gt; T. _1!=null& & T. _2!=null & amp; & T. _3!=null.) take (10)
Your original logic errors, this is feasible

CodePudding user response:

Don't in the spark and null, especially the split () ", "the value is not null,
What also don't say, see code:

 
Def test23 () : the Unit={
Val CFG=new Configuration () 
Val conf=new SparkConf () 
. SetAppName (" spack local test ") 
Set (" spark serializer, "" org. Apache. Spark. Serializer. KryoSerializer") 
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull