Spark DataFrame Data Types
On This Page
The platform currently supports the following Spark DataFrame data types. For a description of the Spark types, see the Spark SQL data types documentation For a general reference of the attribute data types that are supported in the platform, see the Attribute Data Types Reference.
Note
The platform implicitly converts between Spark DataFrame column data types and platform table-schema attribute data types, and converts integer (
IntegerType) and short (ShortType) values to long values (LongType / "long") and floating-point values (FloatType) to double-precision values (DoubleType / "double").
The "Schema of Data Type" column in the following table indicates the matching platform attribute data type for each Spark data type.| Spark Data Type | Schema Data Type |
|---|---|
BooleanType |
"boolean" |
DoubleType |
"double" |
FloatType |
"double" |
IntegerType |
"long" |
LongType |
"long" |
NullType |
"null" |
ShortType |
"long" |
StringType |
"string" |
TimestampType |
"timestamp" |