GroupBy

Data analysis often requires comparing and summarizing values across categories — such as total sales per company, average sentiment by author, or word frequency per decade. Pandas simplifies these tasks through its powerful GroupBy functionality, which lets analysts split data into groups, apply calculations to each, and combine the results into concise summaries.

This chapter introduces the groupby() method and shows how to organize, aggregate, and describe grouped data to uncover meaningful patterns and relationships.

Creating a Sample Dataset

To understand grouping, we use invented sales rows with familiar company labels and fictional employees. These are not company accounts, audited performance data or current corporate identities.

import pandas as pd

# Sample dataset
data = {
    'Company': ['GOOG', 'GOOG', 'MSFT', 'MSFT', 'FB', 'FB'],
    'Person':  ['Sam', 'Charlie', 'Amy', 'Vanessa', 'Carl', 'Sarah'],
    'Sales':   [200, 120, 340, 124, 243, 350]
}

df = pd.DataFrame(data)
print(df)

  Company   Person  Sales
0    GOOG      Sam    200
1    GOOG  Charlie    120
2    MSFT      Amy    340
3    MSFT  Vanessa    124
4      FB     Carl    243
5      FB    Sarah    350

This table contains three columns:

• Company – identifies the organization,
• Person – the employee’s name,
• Sales – the amount of sales they achieved.

The goal is to summarize these results — for example, to find each company’s total or average sales.

The GroupBy Operation

The groupby() method in Pandas groups data based on the values of one or more columns. Once grouped, various aggregation methods such as mean(), sum(), count(), or std() (standard deviation) can be applied to each group.

Let’s begin by grouping the dataset by company:

by_company = df.groupby('Company')
print(by_company)

<pandas.core.groupby.generic.DataFrameGroupBy object at 0x1036dfe00>

This tells us that the data has been grouped — but we haven’t yet asked for a summary. To see the result of the grouping, we can apply an aggregation function such as mean() to compute the average sales per company.

Calculating the Mean

The mean shows the average value of each numeric column within every group.

by_company.mean(numeric_only=True)

Here, Pandas calculates the mean sales for each company by taking the average of their employees’ results. We can see that:

• Facebook (FB) achieved the highest average sales,
• Google (GOOG) recorded the lowest,
• Microsoft (MSFT) stands between them.

Other Common Aggregations

Besides the mean, Pandas provides many built-in methods for summarizing grouped data.

Minimum and Maximum

print(by_company.min())
print(by_company.max())

          Person  Sales
Company                
FB          Carl    243
GOOG     Charlie    120
MSFT         Amy    124
          Person  Sales
Company                
FB         Sarah    350
GOOG         Sam    200
MSFT     Vanessa    340

These methods reduce each column independently. The alphabetical minimum Person and minimum Sales need not come from the same row. For MSFT, min() pairs Amy with 124 even though Vanessa supplied the sales value 124. Use a row selector to find the employee associated with an extreme; see the Pandas GroupBy minimum reference.

Standard Deviation

Standard deviation measures how much the values vary from the mean. It describes the spread of these recorded values in their units. A lower value does not by itself establish a stable organization, a temporal trend or more reliable future performance. Pandas uses ddof=1 by default; the NumPy default is ddof=0.

by_company.std(numeric_only=True)

In this invented two-row-per-group table, the within-group standard deviation is largest for MSFT and smallest for GOOG. Two rows per group provide little evidence about wider employee populations or future periods.

Counting Entries

count() counts non-missing entries separately for each selected column; it is not necessarily the number of rows. size() counts rows. See the Pandas count reference. Here there are no missing entries:

by_company.count()

Each company has two employees in this dataset. Counting is useful for determining sample sizes or checking data completeness per group.

Descriptive Statistics

The describe() method provides a comprehensive statistical summary for each group — including count, mean, standard deviation, min, quartiles, and max.

by_company.describe()

This table gives an overview of the central tendency and spread of values for each company. It is an excellent tool for quickly summarizing numerical datasets.

Transposing the Summary

For better readability, the summary can be transposed using .transpose(), which swaps rows and columns.

by_company.describe().transpose()

Now, each column represents one company, and each row shows a statistical measure. This layout is helpful when comparing groups side by side.

Accessing a Specific Group

It is also possible to access the summary statistics of a specific group, such as Google (GOOG), directly:

by_company.describe().transpose()['GOOG']

Sales  count      2.000000
       mean     160.000000
       std       56.568542
       min      120.000000
       25%      140.000000
       50%      160.000000
       75%      180.000000
       max      200.000000
Name: GOOG, dtype: float64

This isolates Google’s detailed statistics for focused analysis.

Summary

This section showed how groupby() organizes data into meaningful groups and summarizes them using averages, deviations, and descriptive statistics. Grouping transforms raw tables into structured insight — the basis for deeper analysis. The next lesson applies filtering, sorting and pivot summaries. The exploratory-analysis lesson then checks denominators, weighting and aggregation patterns.

Keep the person attached to the selected sale

Predict which employee belongs to the minimum MSFT sale. Then retrieve original rows instead of reducing unrelated columns independently.

minimum_rows = df.loc[df.groupby("Company")["Sales"].idxmin()]
print(minimum_rows[["Company", "Person", "Sales"]].to_string(index=False))
msft = minimum_rows.set_index("Company").loc["MSFT"]
assert msft["Person"] == "Vanessa" and msft["Sales"] == 124

Company  Person  Sales
     FB    Carl    243
   GOOG Charlie    120
   MSFT Vanessa    124

The selected MSFT row belongs to Vanessa. idxmin() returns the first matching index when minima tie; decide whether the question requires all tied rows. This selection assumes the original index labels uniquely identify rows. Change one sale to create a tie and explain what information a one-row-per-company answer would omit.

Chapter exercisesWorked answers after an attempt