Python Pandas Tutorial for Beginners

Python Pandas Tutorial for Beginners: Learn Data Analysis with Pandas

Have you ever found yourself staring at an Excel spreadsheet full of data wondering how to sort out, analyze, and make sense of that data using Python?

Dealing with large volumes of data is quite time-consuming and likely to be full of errors. One of the most widely used Python libraries when it comes to data analysis and data manipulation is Pandas.

Whether it is a novice who is learning the Python programming language, a student who needs to equip himself/herself to become a data analyst, or an individual who wants to enter into the domain of data science, Pandas learning becomes an essential aspect.

In this tutorial on Python Pandas for beginners, we will learn the basics of Pandas, such as the creation of a DataFrame, loading of a dataset, data analysis and saving of the same.

What is Pandas in Python?

Pandas is a Python library that offers convenient ways of handling and analyzing the data.

It enables developers and data analysts to perform all these activities using its easy-to-use yet powerful data structures.

Some of the uses of Pandas are as follows:

Data Analysis
Data Cleaning
Data Preparation
Exploratory Data Analysis (EDA)
Handling CSV and Excel files
Preparing Machine Learning Datasets
Business Reporting
Financial Analysis

Let us assume that we have an employee database of some organization, which consists of the names, departments, salaries, and locations of the employees. Pandas will allow the life of programmers to be easy rather than checking each entry manually one by one.

Why Do You Need to Study Pandas?

The Python language finds wide application in data science, machine learning, artificial intelligence, and data analysis. However, knowledge of the syntaxes of the Python language is not enough when it comes to data analysis.

Pandas serves as a bridge between programming and data analysis using Python.

Using Pandas, one can carry out tasks like:

Importing data from CSV and Excel files

Column selection
Row filtering
Sorting data
Deleting duplicates
Dealing with missing data
Grouping records
Statistics calculations
Combining several datasets
Exporting processed data

Thus, Pandas is an indispensable tool for those starting a career in Data Analytics or Data Science.

How to Install Pandas

In order to use Pandas, first install Pandas.

Open Command Prompt/terminal and run:

pip install pandas

If you are working on Jupyter Notebook, then you may run:

!pip install pandas

Then import Pandas into your Python script:

import pandas as pd

Here, pd is the standard way to refer to Pandas.

The advantage of abbreviation is that it saves lines of code. Instead of writing pandas.DataFrame(), we just have to write pd.DataFrame().

Understanding Pandas Data Structures

Pandas offers mainly two key data structures:

Series

DataFrame

Familiarity with both these data structures is very important for every novice learner.

What is a Pandas Series?

A Series is a one-dimensional data structure.

It would be appropriate to say that a Series represents a single column of data.

For example:

import pandas as pd

marks = pd.Series([85, 90, 78, 92, 88])

print(marks)

The output will give both the values along with the respective index positions.

Series can hold numbers, strings, date or any kind of value.

What is a Pandas DataFrame?

DataFrame is a 2D data structure having rows and columns.

This is similar to an Excel table or database table.

For example:

import pandas as pd

data = {"Nama": ["Amit","Priya","Rahul","Sneha"],

"Usia": [25,28,24,27],

"Kota": ["Pune","Mumbai","Delhi","Bangalore"],

"Gaji": [50000,60000,55000,65000]}

df = pd.DataFrame(data)

print(df)

In the above code,

"Name" is a column,

"Age" is a column,

"City" is a column,

"Salary" is a column.

An individual employee corresponds to one row.

You will be using the DataFrame most commonly while dealing with datasets.

How to Read CSV and Excel Files With Pandas

Practically, in the development process, the data is stored in files and not produced by Python code itself.

Pandas offers easy methods to load these files.

Read a CSV File

import pandas as pd

df = pd.read_csv("employees.csv")

print(df)

The read_csv() method opens a CSV file into a DataFrame.

Read an Excel File

df = pd.read_excel("employees.xlsx")

print(df)

Depending on the Python environment that you are working in, openpyxl library may be needed when dealing with Excel files.

Saving a DataFrame as a CSV file is done this way:

df.to_csv("cleaned_employees.csv", index=False)

With the index=False parameter, Pandas won’t save the index of the DataFrame as a separate column.

How to Explore a Dataset

After loading a dataset, it is time to explore it.

Pandas has many helpful methods for that purpose.

Head() Method

print(df.head())

By default, head() method prints first five rows.

You may define the number of rows:

print(df.head(10))

Tail() Method

print(df.tail())

It will print the last five rows.

Shape Method

print(df.shape)

It returns the number of rows and columns.

Example:

(1000, 6)

It indicates that your dataset contains 1000 rows and 6 columns.

Info() Method

df.info()

It provides highly useful information like column names, data types and much more.

Describe() Method

print(df.describe())

This will generate descriptive statistics for numeric columns such as mean, min, max and quantile values.

Column Names

print(df.columns)

It prints the names of all the columns.

Selecting Data from a DataFrame

One of the common Pandas functions is to select specific columns.

For selecting a single column:

print(df["Name"])

For selecting multiple columns:

print(df[["Name", "Salary"]])

It is helpful especially when you have many columns in your dataset but need to use just some.

Filtering Data Using Pandas

Row filtering with Pandas is easy.

Let’s say we want those employees who have their salary above ₹55,000:

high_salary = df[df["Salary"] > 55000]

print(high_salary)

We can also filter for employees working in Pune:

pune_employees = df[df["City"] == "Pune"]

print(pune_employees)

Various conditions can also be used together:

result = df[

(df["Salary"] > 50000) &

(df["Age"] < 28)]

print(result)

& stands for AND here.

For OR condition, we can use |.

Using loc and iloc

Pandas provides functions like loc and iloc for selecting certain rows and columns.

loc()

loc function is primarily used for label-based indexing.

print(df.loc[0])

We can also choose particular columns:

print(df.loc[0, ["Name", "Salary"]])

iloc()

iloc is an integer-based positioning function.

print(df.iloc[0])

For selecting the first three rows:

print(df.iloc[0:3])

The knowledge of loc and iloc is highly valuable when dealing with large datasets.

Handling Missing Values

Many datasets have missing values or empty values.

Missing values can be identified by:

print(df.isnull().sum())

It indicates the number of missing values in each column.

To drop the rows having missing values:

df_cleaned = df.dropna()

Missing values can also be filled.

For example:

import matplotlib.pyplot as plt

In this case, missing values of salary are filled with median salary.

Different methods depend upon the type of data and reason behind missing values.

Removing Duplicate Records

Duplicate entries might influence the accuracy of the analysis.

There is a method called drop_duplicates() in Pandas:

df = df.drop_duplicates()

But be careful to confirm that the duplicates are indeed duplicates before deleting them. Sometimes in transactional data, two different transaction records can look the same.

Basic Data Analysis

Now when the data is cleaned up, you can use Pandas for doing some basic computations.

Calculate Mean

print(df["Salary"].mean())

Calculate Max

print(df["Salary"].max())

Calculate Min

print(df["Salary"].min())

Calculate Sum

print(df["Salary"].sum())

GroupBy in Pandas

Another important thing about Pandas is that it offers a method called groupby().

Let’s assume the data set has a column called Department. We can get the mean value of salaries in each department:

department_salary = df.groupby("Department")["Salary"].mean()

print(department_salary)

groupby() is helpful for:

Sales analysis

Employee analysis

Customer analysis

Product analysis

Accounting

Performance analysis

For instance, the sales data set can be grouped according to the cities.

Sorting Data

Pandas can also be used to sort the records.

Sorting employees by salary:

df = df.sort_values("Salary")

print(df)

For sorting in descending order:

df = df.sort_values("Salary", ascending=False)

print(df)

Sorting assists in finding out high-performing products, highest salaries, biggest sales, and many more.

Data Visualization with Pandas

Pandas is also a library with plotting capabilities compatible with other libraries such as Matplotlib.

For instance:

import matplotlib.pyplot as plt

df.plot(

x="Name",

y="Salary",

kind="bar",

legend=False

)

plt.title('Employee Salary Comparison')

plt.xlabel("Employee Name")

plt.ylabel("Salary")

plt.show()

Based on the need, you can plot:

Bar graphs

Line graphs

Histograms

Scatter plots

To plot graphs, one can combine Pandas with Matplotlib and Seaborn.

Applications of Pandas

Practical Applications of Pandas

Pandas is not limited to classroom problems only. Pandas is used in many applications.

Data Analytics

Data analysts use Pandas to clean datasets, calculate statistics, spot trends, and create reports.

Data Analytics

Data Analysts use Pandas for data cleaning, calculation, trend identification, and reporting.

Machine Learning

Data may have to be cleaned, filtered, processed, and prepared before creating a machine learning model. Pandas is very popular for such tasks.

Business Intelligence

Pandas will help analysts in data preparation before building the dashboard using PowerBI or Tableau.

Financial Analysis

Databases full of financial data with transactions, prices, expenses, and revenues can be easily analyzed using Pandas.

Customer Analytics

Pandas allows companies to analyze the buying habits, location, spending habits, and other business-related information of their customers.

Pandas vs Excel

One of the frequent questions that newcomers pose about the choice between Pandas and Excel is if Pandas is worth learning in addition to Excel.

The answer is affirmative.

Excel is great for interactive analysis via spreadsheets, whereas Pandas offers the capabilities of programming.

This means that you would be able to clean your data by running a script in Python on several hundred files instead of executing each step of the procedure yourself.

How to Efficiently Learn Pandas as a Beginner

As a beginner, one should not try to cover all the features of Pandas all at once.

An appropriate order is:

Learn Series
Learn DataFrame
Import CSV & Excel files
Study datasets
Columns & rows selection
Learn filtering
Practice loc & iloc
Missing values handling
Deduplication
Learn sorting
Practice groupby()
Basic statistics
Simple visualization
Real world projects

Begin from small datasets and slowly work your way towards bigger datasets.

Conclusion

Pandas is one of the most significant libraries of Python that is very important when doing data analysis. The Pandas library has simple yet powerful functions to import, clean, manipulate, analyze, and export structured data.

As a beginner, there are some important concepts you need to know about, including Series, Data Frame, data importation, filtering, missing data, sorting, group-by, and statistics.

After learning how to work on these concepts, you can begin working on various projects including sales analysis, customer analysis, employee analysis, financial analysis, and exploratory data analysis.

Pandas is also an important building block for those interested in learning Data Analytics, Data Science, Machine Learning, and Artificial Intelligence.

The best approach for mastering Pandas is through continuous practice where you use Python and Pandas to answer any questions you may have regarding a certain data set.

For more learning resources and technology updates, visit the SevenMentor YouTube channel: SevenMentor YouTube Channel

Python Pandas Tutorial for Beginners