Home Tutorials Videos Q&A MCQ Shortcuts Exercises Appointment Login
Lesson 67 of 70 – Python Pandas
96%

Python Pandas

Pandas is a popular Python library used for data analysis and data manipulation. It provides powerful data structures such as Series and DataFrame.

Note: Pandas is widely used for working with structured data such as CSV files, Excel files, databases and other tabular datasets.
What is Pandas?

Pandas is an open-source Python library designed for working with structured and tabular data.

It provides tools for cleaning, analyzing, transforming and exploring datasets.

  • Data analysis
  • Data cleaning
  • Data filtering
  • Data aggregation
  • Data transformation
  • CSV and Excel file handling
  • Missing-data handling
Installing Pandas

Pandas can be installed using pip. 

pip install pandas

You can also use: 

python -m pip install pandas
Importing Pandas

Pandas is commonly imported using the alias pd. 

import pandas as pd

The alias makes Pandas functions shorter and easier to use.

What is a Series?

A Pandas Series is a one-dimensional labeled data structure. 

import pandas as pd

numbers = pd.Series(
    [10, 20, 30, 40]
)

print(numbers)
Example Output: 
0    10
1    20
2    30
3    40
dtype: int64
Creating a Series with Custom Index

You can specify custom labels using the index parameter. 

import pandas as pd

marks = pd.Series(
    [80, 75, 90],
    index=["Math", "English", "Science"]
)

print(marks)
Output: 
Math       80
English    75
Science    90
dtype: int64
Series from a Dictionary

A dictionary can be converted into a Pandas Series. 

data = {
    "Math": 80,
    "English": 75,
    "Science": 90
}

marks = pd.Series(data)

print(marks)
What is a DataFrame?

A Pandas DataFrame is a two-dimensional labeled data structure with rows and columns.

It is similar to a table in a database or a spreadsheet. 

import pandas as pd

data = {
    "Name": ["Rahul", "Priya", "Amit"],
    "Age": [21, 22, 20],
    "Course": ["Python", "Java", "SQL"]
}

df = pd.DataFrame(data)

print(df)
DataFrame Example 
data = {
    "Name": ["Rahul", "Priya", "Amit"],
    "Marks": [85, 90, 78]
}

df = pd.DataFrame(data)

print(df)
Output: 
    Name  Marks
0  Rahul     85
1  Priya     90
2   Amit     78
Checking DataFrame Information

The info() method displays information about a DataFrame. 

df.info()

It provides information such as the number of entries, column names, non-null counts and data types.

Viewing First Rows

The head() method displays the first rows of a DataFrame. 

print(df.head())

You can specify the number of rows: 

print(df.head(2))
Viewing Last Rows

The tail() method displays the last rows. 

print(df.tail())

For example: 

print(df.tail(2))
Reading CSV Files

Pandas can read CSV files using read_csv(). 

import pandas as pd

df = pd.read_csv("students.csv")

print(df)
Reading Excel Files

Pandas can read Excel files using read_excel(). 

import pandas as pd

df = pd.read_excel("students.xlsx")

print(df)
Note: Depending on the Excel format and your environment, Pandas may require an appropriate Excel engine/package.
Selecting a Column

A DataFrame column can be selected using its column name. 

names = df["Name"]

print(names)

The result is generally a Pandas Series.

Selecting Multiple Columns

Multiple columns can be selected by passing a list of column names. 

result = df[
    ["Name", "Marks"]
]

print(result)
Selecting Rows with loc

The loc accessor is used for label-based selection. 

row = df.loc[0]

print(row)

You can also select specific columns: 

print(
    df.loc[0, "Name"]
)
Selecting Rows with iloc

The iloc accessor selects data using integer positions. 

print(df.iloc[0])
print(df.iloc[1, 0])

The first statement selects the first row and the second selects the value at row position 1 and column position 0.

Filtering Data

Boolean conditions can be used to filter rows. 

result = df[
    df["Marks"] > 80
]

print(result)

This selects rows where the Marks value is greater than 80.

Multiple Conditions

Multiple conditions can be combined using & for AND and | for OR. Each condition should be enclosed in parentheses. 

result = df[
    (df["Marks"] > 70) &
    (df["Age"] >= 20)
]

print(result)
Adding a New Column

A new column can be created by assigning values to a new column name. 

df["Result"] = [
    "Pass",
    "Pass",
    "Pass"
]

print(df)
Updating a Column 
df["Marks"] = df["Marks"] + 5

print(df)

This adds 5 to each value in the Marks column.

Deleting a Column

The drop() method can be used to remove a column. 

df = df.drop(
    columns=["Result"]
)

print(df)
Sorting Data

The sort_values() method sorts rows according to a column. 

result = df.sort_values(
    by="Marks"
)

print(result)

For descending order: 

result = df.sort_values(
    by="Marks",
    ascending=False
)
Checking Missing Values

The isna() method identifies missing values. 

print(df.isna())

You can count missing values in each column: 

print(df.isna().sum())
Handling Missing Values

The dropna() method removes rows or columns containing missing values, depending on the options used. 

df = df.dropna()

print(df)

The fillna() method can replace missing values. 

df = df.fillna(0)

print(df)
Renaming Columns

The rename() method can be used to change column names. 

df = df.rename(
    columns={
        "Marks": "Score"
    }
)

print(df)
Descriptive Statistics

The describe() method generates descriptive statistics for appropriate columns. 

print(df.describe())

It can provide values such as count, mean, standard deviation, minimum and maximum.

Mean, Sum, Minimum and Maximum 
print(df["Marks"].mean())

print(df["Marks"].sum())

print(df["Marks"].min())

print(df["Marks"].max())
Value Counts

The value_counts() method counts the occurrences of each unique value. 

print(
    df["Course"].value_counts()
)

This is useful for understanding the frequency of categories.

GroupBy

The groupby() method groups rows based on one or more columns. 

result = df.groupby(
    "Course"
)["Marks"].mean()

print(result)

This calculates the average Marks for each Course.

Removing Duplicate Rows

The drop_duplicates() method removes duplicate rows. 

df = df.drop_duplicates()

print(df)
Changing Data Types

The astype() method can be used to convert a column to a different data type when the values are compatible. 

df["Age"] = df["Age"].astype(int)

print(df["Age"].dtype)
Writing Data to CSV

The to_csv() method saves a DataFrame as a CSV file. 

df.to_csv(
    "students_output.csv",
    index=False
)

The index=False option prevents the DataFrame index from being written as an additional CSV column.

Writing Data to Excel

A DataFrame can be written to an Excel file using to_excel(). 

df.to_excel(
    "students_output.xlsx",
    index=False
)
Note: Writing Excel files may require an appropriate Excel engine/package in your Python environment.
Working with Dates

The to_datetime() function converts values to Pandas datetime objects when they can be interpreted as dates or times. 

df["Date"] = pd.to_datetime(
    df["Date"]
)

print(df["Date"])
Combining DataFrames

The pd.concat() function can combine DataFrames along an axis. 

df1 = pd.DataFrame({
    "Name": ["Rahul", "Priya"]
})

df2 = pd.DataFrame({
    "Name": ["Amit", "Neha"]
})

result = pd.concat(
    [df1, df2],
    ignore_index=True
)

print(result)
Merging DataFrames

The merge() function combines DataFrames based on related columns, similar to a database join. 

students = pd.DataFrame({
    "ID": [1, 2, 3],
    "Name": ["Rahul", "Priya", "Amit"]
})

fees = pd.DataFrame({
    "ID": [1, 2, 3],
    "Fee": [15000, 18000, 12000]
})

result = pd.merge(
    students,
    fees,
    on="ID"
)

print(result)
Common Pandas Mistakes
  • Forgetting to install Pandas.
  • Forgetting to import Pandas.
  • Using incorrect column names.
  • Confusing Series and DataFrame.
  • Forgetting parentheses when calling methods.
  • Ignoring missing values.
  • Using incorrect data types.
  • Forgetting index=False when exporting a CSV if the index should not be stored.
  • Using & and | incorrectly when combining boolean conditions.
Complete Pandas Example 
import pandas as pd

data = {
    "Name": ["Rahul", "Priya", "Amit", "Neha"],
    "Age": [21, 22, 20, 23],
    "Marks": [85, 92, 76, 88]
}

df = pd.DataFrame(data)

print("Data:")
print(df)

print("\nAverage Marks:")
print(df["Marks"].mean())

print("\nStudents with Marks above 80:")
print(
    df[df["Marks"] > 80]
)

print("\nSorted Data:")
print(
    df.sort_values(
        by="Marks",
        ascending=False
    )
)
Key Points
  • Pandas is a Python library for data analysis and manipulation.
  • Pandas is commonly imported as pd.
  • A Series is a one-dimensional labeled data structure.
  • A DataFrame is a two-dimensional labeled data structure.
  • read_csv() reads CSV data.
  • read_excel() reads Excel data.
  • head() displays the first rows.
  • tail() displays the last rows.
  • info() provides DataFrame information.
  • loc performs label-based selection.
  • iloc performs integer-position based selection.
  • Boolean conditions can be used to filter data.
  • sort_values() sorts DataFrame rows.
  • dropna() removes missing values.
  • fillna() replaces missing values.
  • groupby() groups data for analysis.
  • drop_duplicates() removes duplicate rows.
  • to_csv() writes DataFrame data to CSV.
  • merge() combines related DataFrames.
  • Pandas is widely used in data science and data analysis.

🧠 Quick Quiz

Question: Which Pandas data structure is two-dimensional and contains rows and columns?




← Previous Next →