Data Transformation

How can we view the code/query submitted for Spark pipelines running on the cluster?

In Spark, the query cannot be printed since data frame is used to submit the job. The plan is printed in the logs for all the transformations. Also, you can view the logical plan in SQL Tab in the Spark UI.

Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
On This Page
Data Transformation