Apache Pig
What is Apache
Pig?
Pig is a
tool/platform which is used
to analyze larger sets of data representing them as data flows. Pig is
generally used with Hadoop; we can perform all the data
manipulation operations in Hadoop using Apache Pig.
To write data
analysis programs, Pig provides a high-level language known as Pig
Latin. This language provides various operators using which programmers can
develop their own functions for reading, writing, and processing data.
Apache Pig has a
component known as Pig Engine that accepts the Pig Latin
scripts as input and converts those scripts into MapReduce jobs.
Why name PIG?
During development the YAHOO engineers name the
language as "THE LANGUAGE"
and after that the renamed it "PIG".
Pig properties?
pig can eat anything.
pig cant fly.
WHAT PIG DOES?
Pig was designed
for performing a long series of data operations, making it ideal for three
categories of Big Data jobs:
·
Extract-transform-load (ETL) data pipelines,
·
Research on raw data, and
·
Iterative data processing.
HOW PIG WORKS?
·
MapReduce Mode. This is the default mode, which requires access to a Hadoop cluster.
·
Local Mode. With access to a single machine, all files are installed and run using a
local host and file system.
Why Do We Need Apache Pig?
Programmers who
are not so good at Java normally used to struggle working with Hadoop,
especially while
performing any MapReduce tasks.
Apache Pig is a boon for all such programmers.
·
Using Pig Latin, programmers can
perform MapReduce tasks easily without having to type complex codes in Java.
·
Apache Pig
uses multi-query approach,
thereby reducing the length of codes. Pig Latin is SQL-like language and it is easy to learn Apache Pig
when you are familiar with SQL.
·
Apache Pig
provides many built-in operators to support data operations like joins,
filters, ordering, etc. In addition, it also provides nested data types like
tuples, bags, and maps that are missing from MapReduce.
- Define a relation with and without schema
- Define a new relation from an existing relation
- Select specific columns from within a relation
- Join two relations
- Sort the data using ‘ORDER BY’
- FILTER and Group the data using ‘GROUP BY’
Features of Pig
Apache Pig comes
with the following features −
·
Rich set of operators − It provides many operators to perform
operations like join, sort, filer, etc.
·
Ease of programming − Pig Latin is similar to SQL and it is easy to write
a Pig script if you are good at SQL.
·
Optimization opportunities − The tasks in Apache Pig optimize their
execution automatically, so the programmers need to focus only on semantics of
the language.
·
Extensibility − Using the existing operators, users can
develop their own functions to read, process, and write data.
·
UDF’s − Pig provides the facility to create User-defined Functions in other programming languages such as
Java and invoke or embed them in Pig Scripts.
·
Handles
all kinds of data −
Apache Pig analyzes all kinds of data, both structured as well as unstructured.
It stores the results in HDFS.
·
Apache Pig Vs MapReduce
·
Listed below are the major differences
between Apache Pig and MapReduce.
Comments
Post a Comment