Python Pandas Tutorial for Beginners

Python Pandas Tutorial for Beginners

With the rise of a data-centric era, the ability to analyze data has gained utmost importance amongst professionals in domains like data science, machine learning, financial analytics, and business analytics. A popular tool that has been developed for data analysis in Python programming is Pandas. If you are an amateur in data analysis, you need to know about Pandas.

What is Pandas?

It is a robust and free Python library that allows the user to manipulate and analyze data. It is essential for students who want to learn data analysis.

Pandas is a Python library that provides useful functions to manipulate structured data such as tables, CSVs, and Excel files.

If you have any data set that consists of details of employees, like their name, department, salary, and the city, then without manually searching anything, you can filter out the employees, calculate average salary, check for any missing data, and analyze the whole data set using some lines of code.

Pandas is widely used in:

Data analysis and exploratory data analysis (EDA)
Data cleaning and preprocessing
Financial and business reporting
Machine learning data preparation
Statistical analysis and research
Working with CSV and Excel files

How to install pandas?

Prior to working with Pandas, it is required to install Pandas.

Go to your terminal and execute:

pip install pandas

In Jupyter Notebook, you can run:

%pip install pandas

Importing the library into your Python script:

import pandas as pd

Pandas has two significant data structures – Series and DataFrame.

Series :

A Series is a one-dimensional labeled data structure. You can think of it as a single column of data.

Example :

import pandas as pd

marks = pd.Series([85, 90, 78, 92, 88])

print(marks)

The output will be :

0 85

1 90

2 78

3 92

4 88

dtype: int64

The left figures represent index labels while those on the right represent the corresponding values.

DataFrame :

DataFrame is a two-dimensional data structure that consists of rows and columns. It is analogous to the table in Excel or a database.

Example :

data = {

"Name": ["Amit", "Priya", "Rahul", "Sneha"],

"Age": [25, 28, 24, 27],

"City": ["Pune", "Mumbai", "Delhi", "Bangalore"],

"Salary": [50000, 60000, 55000, 65000]

}

df = pd.DataFrame(data)

print(df)

Output :

Output:

Name

Age

City

Salary

Amit

Pune

50000

Priya

Mumbai

60000

Rahul

Delhi

55000

Sneha

Bangalore

65000

In this case, each row stands for an individual employee while each column stands for the attribute.

Importing Data from CSV and Excel Files

In reality, the data is often stored in files instead of being generated manually in Python..

It becomes very easy to import such files using Pandas.

To read a CSV file:

df = pd.read_csv("employees.csv")

For reading Excel file:

df = pd.read_excel("employees.xlsx")

One may also save your output to a different file.

df.to_csv("cleaned_employees.csv", index=False)

Using the index=False option ensures that pandas do not include the DataFrame index as one of the columns of the resulting file.

For the case of Excel files, you need to install another library called openpyxl, depending upon the environment you are working on.

5. Examining a Dataset

After importing the dataset, the next logical step is to examine the structure and the contents of the dataset.

Consider the following commonly used Pandas functions:

df.head()

Displays the first five rows

df.tail()

Displays the last five rows

df.shape

Returns the number of rows and columns

df.columns

Displays column names

df.info()

Shows column data types and non-null counts

df.describe()

Provides descriptive statistics for numeric columns

df.dtypes

Displays the data type of each column

For example:

print(df.head())

print(df.shape)

print(df.info())

print(df.describe())

The above lines of code will help you to understand your data prior to doing any analysis.

6. Selection and Filtering of Data

Pandas helps in selecting particular columns and filtering rows based on some conditions.

Selecting one column:

print(df["Name"])

Selecting multiple columns:

print(df[["Name", "Salary"]])

Filtering employees earning more than ₹55,000:

high_salary = df[df["Salary"] > 55000]

print(high_salary)

Filtering employees who live in Pune:

pune_employees = df[df["City"] == "Pune"]

print(pune_employees)

You can also combine multiple conditions:

result = df[

(df["Salary"] > 50000) &

(df["Age"] < 28)

]

print(result)

& is the symbol for AND, and | for OR. Every criterion needs to be placed within brackets.

Loc and iloc are additional functionalities offered by Pandas for data selection. Loc works on labels, whereas iloc works on integer locations.

7. Handling Missing Data and Duplicate Entries

It is very common to have missing data and duplicate entries in any practical scenario.

Finding missing values:

print(df.isnull().sum())

This gives the number of null values for each column.

Removing rows with missing values:

df_cleaned = df.dropna()

Filling missing values:

df["Salary"] = df["Salary"].fillna(

df["Salary"].median()

)

The use of the median is one way to deal with a numeric field; however, the right way to do it will depend on the nature of the data and the cause of the missing values.

Removing duplicate rows:

df = df.drop_duplicates()

Make sure that what needs to be deleted are really duplicates. In the case of two records which are from the same customer, there may actually be two different transactions.

8. Conducting Basic Data Analysis

It is very easy to compute basic summary statistics using Pandas.

Compute the mean of salary:

Calculate the average salary:

print(df["Salary"].mean())

Find the highest salary:

print(df["Salary"].max())

Find the lowest salary:

print(df["Salary"].min())

Finding out the total salary:

print(df["Salary"].sum())

Grouping of records allows comparing the data belonging to distinct categories.

department_salary = df.groupby("Department")["Salary"].mean()

print(department_salary)

This demonstrates how to compute the mean salary per department, on the assumption that your DataFrame has a column named "Department".

Grouping is highly beneficial when conducting sales analysis, employee analysis, client segmentation, and business performance comparison.

9. Visualization of Data Using Pandas

Visualization of data makes it easy to represent numbers in graph form.

Pandas has functions which facilitate simple plotting, using the power of Matplotlib.

To plot a graph showing employee salaries:

import matplotlib.pyplot as plt

df.plot(

x="Name",

y="Salary",

kind="bar",

legend=False

)

plt.title("Employee Salary Comparison")

plt.xlabel("Employee Name")

plt.ylabel("Salary")

plt.tight_layout()

plt.show()

Some of the graphs that can be made include:

Bar Chart, Line Chart, Histogram, Scatter Plot

Or alternatively, matplotlib and seaborn can be used to draw charts.

10. Why Should Beginners Know Pandas?

By learning Pandas, one becomes able to gain practical skills in dealing with actual data. This will allow you to avoid doing the same things over and over again, thus simplifying everything and allowing you to make data-based decisions.

Pandas are very helpful for those who want to become data analysts because it helps to bridge the gap between programming using Python and manipulating data.

Once you have gained some basic knowledge, you can apply these skills to analyze sales reports, customer data, hospital data, finance data, and other similar things.

Conclusion

Pandas is among the top libraries of Python that is essential for every person who is keen on data analysis. Be it a matter of creating series and dataframes or importing files, filtering data, dealing with missing data, conducting statistical operations, and even visualizing data, Pandas allows a neat and flexible way of doing all that.

The only way of mastering Pandas is practice. Start with smaller datasets, get familiar with all the functions, and then move onto harder things.

Keep this in mind: Get good at using Pandas, use Pandas to manipulate real-world data, and derive insights from raw data.

Related Links :

Do visit our channel : SevenMentor.

Author:

Chaitali Kate