Pandas

What is Pandas?

Pandas 🐼 is an open-source Python library used for data munging and data analysis. It is designed to be intuitive and easy to use. It provides a high level interface to work on tabular data.

Pandas is built on top of NumPy, which makes internal computations fast.

What is a DataFrame?

The tabular data is stored in Pandas in a DataFrame—a 2-dimensional data structure. It is modeled on a table and has rows containing data records and columns containing attributes corresponding to those records.

Who uses Pandas?

Everyone from a data analyst to a machine learning engineer would need to get their hands dirty with Pandas at some point or the other. Implementations of almost all machine learning models take in data as Pandas’ DataFrames.

Analysts and data scientists who clean raw data use Pandas as part of their daily workflows. Advanced statistics of entire datasets can be calculated with a variety of Pandas functions. Dropping rows, filling in missing values and generating plots can be done in one line with Pandas’ in-built functions.

How can you get started with Pandas?

Installing Pandas

1. If you use pip, then type the following command to install Pandas as part of your Python environment:

pip install pandas

2. If you use Anaconda, then you don’t have to install anything since it comes pre-installed with the latest version of Pandas.

3. Once installed, import Pandas in your Python shell with the following command:

import pandas as pd

Creating a Pandas DataFrame

The following command will create a Pandas DataFrame which has three columns, namely, Python, Java and C++. It also has two rows which have corresponding values for all three columns.

df = pd.DataFrame(columns=['Python','Java','C++'],data=[[10,2,3],[4,13,6 ]])

This is the resulting DataFrame:

      Python    Java    C++
      
0       10           2        3

1        4          13        6

You can use the describe() function on the DataFrame to get various statistics like min, max, mean, etc. for each column. To do so, type:

df.describe()

To learn more about the various functions that Pandas offers, you can go through the official Pandas User Guide.

Are there any alternatives to Pandas?

If you work on massive data sets—tens and hundreds of gigabytes—then Pandas can be a bit too slow. To facilitate such situations, several projects have sprung up, built on top of Pandas, to provide greater speed and distributed multiprocessing capabilities.

What is Modin?

Modin on Github speeds up Pandas workflows by changing a single line of code.

What is PySpark?

PySpark, written in Python to support Apache Spark, is a distributed framework that can handle big data analysis. Recently, it added support for Pandas DataFrames, which allows you to use distributed systems to do days of Pandas computations in minutes.