how to show pyspark df with large columns-CodePudding

Home > Software design > how to show pyspark df with large columns

how to show pyspark df with large columns

Time：07-07

I have big pyspark df, with many columns and I need show it correctly. I don't need vertical. Is there way to show with scrolling?

CodePudding user response：

Try with display(sparkDF) else the best workaround is

sparkDF.limit(5).toPandas().head()

CodePudding user response：

Just add:

from IPython.core.display import HTML
display(HTML("<style>pre { white-space: pre !important; }</style>"))

And execute df.show(), you will get the scrollbar.