I'm using DataBricks Python.

If I save this file

y_pred.to_csv("y_pred.csv")   

it works but I don't know where the file is and I can't find it.

When I try to copy the file in another location

dbutils.fs.cp("/y_pred.csv", 'wasbs://...........')

I get the following error:

java.io.FileNotFoundException: /y_pred.csv

CodePudding user response：

Hi This is the solution:

dbutils.fs.put('shared_uploads/[email protected]/y_pred.csv', pd.DataFrame(y_pred).to_csv())

CodePudding user response：

Based on the databricks concepts , If data frame fits in a driver memory and you want to save to local files system ,that time we can use to_csv. So your file will be stored in driver node . When you are reading the drive node local file through dbutils.fs we should add (file:/<filelocation). So try to read it like dbutils.fs.cp("file:/y_pred.csv", 'wasbs://...........') will it work . You can understand on dbfs location ref