Using Spark Union and UnionAll you can merge data of 2 Dataframes and create a new Dataframe. Remember you can merge 2 Spark Dataframes only […]
Blogs
SPARK distinct and dropDuplicates
Both Spark distinct and dropDuplicates function helps in removing duplicate records. One additional advantage with dropDuplicates() is that you can specify the columns to be […]
SPARK FILTER FUNCTION
Using Spark filter function you can retrieve records from the Dataframe or Datasets which satisfy a given condition. People from SQL background can also use […]
SPARK DATAFRAME SELECT
Today we will learn how to Select columns from a Spark Dataframe. While selecting we can show complete list of columns or select only few […]
HIVE ALTER TABLE
In Previous chapter we learned about HIVE TABLE CREATION and today lets check out HIVE ALTER TABLE. Here we will list down all the alterations that can be […]
HIVE DATA TYPES
Many of the datatypes you find in Relational Databases can be found in Hive as well. They are called “Primitive Datatypes“. Along with these , […]
SCALA FUNCTIONS
In Previous chapter we learned about SCALA CASE CLASS and today lets check out SCALA FUNCTIONS. Simply put a function is a block of statement which […]
SCALA CASE CLASS
In Previous chapter we learned about SCALA SINGLETON AND COMPANION OBJECT and today lets check out SCALA CASE CLASS. A Case Class is just like a regular […]
SCALA SINGLETON AND COMPANION OBJECT
In Previous chapter we learned about SCALA CLASS and today lets check out SCALA SINGLETON AND COMPANION OBJECT. From previous chapter we now know how to create […]
SCALA CLASS
In Previous chapter we learned about SCALA PATTERN MATCHING and today lets check out SCALA CLASS. A Scala Class is a collection of variables , methods ,objects […]