Start networking and exchanging professional insights

Register now or log in to join your professional community.

Follow

How to achieve dynamic partition in Spark-sql.?

Is it possible to create dynamic partition in spark-sql

user-image
Question added by Aditya Rajendra Jadhav Jadhav , Software Developer , M/S Sks Sarda group
Date Posted: 2017/09/11
Ahmed Lateef
by Ahmed Lateef , Member Technical Services , eBay

Just add the column name with out "=" but you must Define at least one static partition ( should be last in sequence).

for example

Partition by (Name, date=2017)

Shankar J
by Shankar J , Manager , Standard Chartered Global Business Services

It is to be declared to the field when create a table using syntax 'parition by <fieldname>' 

Partition is the good method to segrate the related values to store into hadoop with multiple areas.

More Questions Like This