Foundations of
Data Science

Spring 2026 | Data 2 (399)
Jeffrey M. Girard | Lecture 11a

Roadmap

Calculate summary statistics to describe an entire dataset
Extract extreme rows using the slice family of functions
Apply these operations to distinct subgroups within your data

Basic Summarize

Summarize

Question: What does the data look like overall?
Although we store data about many observations, we often want to summarize across them.
- This is like folding the tibble down to one row.

We have seen functions that summarize vectors:
- length(), sum(), min(), max()
- mean(), median(), sd(), var()

summarize() lets us use them on tibbles.
- It works very similarly to mutate().
- It always creates a new tibble as output.

Preparing the Data

First, let’s create a small example dataset of sales to practice with.

library(tidyverse)
sales <- tibble(
  customer = c(1, 2, 3, 1, 3),
  store = c("A", "A", "A", "B", "B"),
  quantity = c(25, 20, 16, 10, 5),
  revenue = c(685, 590, 392, 185, 123)
)
sales

# A tibble: 5 × 4
  customer store quantity revenue
     <dbl> <chr>    <dbl>   <dbl>
1        1 A           25     685
2        2 A           20     590
3        3 A           16     392
4        1 B           10     185
5        3 B            5     123

Overall Averages

Question: What is the average transaction size and value?

We can use summarize() to calculate the mean quantity and revenue across all observations in the dataset.

sales |> 
  summarize(
    avg_quantity = mean(quantity),
    avg_revenue = mean(revenue)
  )

# A tibble: 1 × 2
  avg_quantity avg_revenue
         <dbl>       <dbl>
1         15.2         395

The summary() Pitfall

Warning: Be careful not to accidentally use base R’s summary() function when you mean to use dplyr::summarize(). It will ignore your arguments!

sales |> 
  summary(
    avg_quantity = mean(quantity),
    avg_revenue = mean(revenue)
  )

    customer    store              quantity       revenue   
 Min.   :1   Length:5           Min.   : 5.0   Min.   :123  
 1st Qu.:1   Class :character   1st Qu.:10.0   1st Qu.:185  
 Median :2   Mode  :character   Median :16.0   Median :392  
 Mean   :2                      Mean   :15.2   Mean   :395  
 3rd Qu.:3                      3rd Qu.:20.0   3rd Qu.:590  
 Max.   :3                      Max.   :25.0   Max.   :685

Counting Functions

Question: How much data do we actually have?

We can use special functions like n() to count total rows, and n_distinct() to count unique values.

sales |> 
  summarize(
    n_sales = n(),
    n_customers = n_distinct(customer),
    n_stores = n_distinct(store)
  )

# A tibble: 1 × 3
  n_sales n_customers n_stores
    <int>       <int>    <int>
1       5           3        2

Grouped Summarize

Question: How do different categories compare to one another?
We can summarize a tibble by group.
- This is like folding the tibble multiple times.
- We will fold down to one row per group.

We just add the .by argument to summarize().
- We can group by one or many variables.
- With many, we group by their combinations.

Summarizing by One Group

Question: Which store is performing better?

Let’s see how the sales perform on a per-store basis by adding .by.

sales |> 
  summarize(
    .by = store,
    customers = n_distinct(customer),
    total_quantity = sum(quantity),
    total_revenue = sum(revenue),
    avg_quantity = mean(quantity),
    avg_revenue = mean(revenue)
  )

# A tibble: 2 × 6
  store customers total_quantity total_revenue avg_quantity avg_revenue
  <chr>     <int>          <dbl>         <dbl>        <dbl>       <dbl>
1 A             3             61          1667         20.3        556.
2 B             2             15           308          7.5        154

Summarizing by Multiple Groups

Question: How much did each customer purchase at each store?

Now let’s group by customer and store combinations.

sales |> 
  summarize(
    .by = c(customer, store),
    n_visits = n(),
    total_quantity = sum(quantity),
    total_revenue = sum(revenue)
  )

# A tibble: 5 × 5
  customer store n_visits total_quantity total_revenue
     <dbl> <chr>    <int>          <dbl>         <dbl>
1        1 A            1             25           685
2        2 A            1             20           590
3        3 A            1             16           392
4        1 B            1             10           185
5        3 B            1              5           123

Real-World Data

Let’s apply these concepts to a larger, more realistic dataset.

library(nycflights13)

delays <- 
  flights |> 
  select(month, day, carrier, flight, dep_delay)

glimpse(delays)

Rows: 336,776
Columns: 5
$ month     <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, …
$ day       <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, …
$ carrier   <chr> "UA", "UA", "AA", "B6", "DL", "UA", "B6", "EV", "B6", "AA", …
$ flight    <int> 1545, 1714, 1141, 725, 461, 1696, 507, 5708, 79, 301, 49, 71…
$ dep_delay <dbl> 2, 4, 2, -1, -6, -4, -5, -3, -3, -2, -2, -2, -2, -2, -1, 0, …

Average Delay by Carrier

Question: What is the average departure delay for each airline carrier?

delays |> 
  summarize(
    .by = carrier,
    m_delay = mean(dep_delay, na.rm = TRUE)
  )

# A tibble: 16 × 2
  carrier m_delay
  <chr>     <dbl>
1 UA        12.1 
2 AA         8.59
3 B6        13.0 
4 DL         9.26
5 EV        20.0 
6 MQ        10.6 
# ℹ 10 more rows

Slice

The Slice Functions

Question: Who are the extreme winners and losers?

We often want to extract specific rows.

The slice functions do exactly this:
- slice_min() extracts rows based on their lowest values.
- slice_max() extracts rows based on their highest values.

We just tell it which column to look at using order_by and how many rows to return using n.

Best Average Performance

Question: Which 3 carriers have the lowest average departure delay?

We can calculate the summary and then slice the results to get our answer.

delays |> 
  summarize(
    .by = carrier, 
    m_delay = mean(dep_delay, na.rm = TRUE)
  ) |> 
  slice_min(n = 3, order_by = m_delay)

# A tibble: 3 × 2
  carrier m_delay
  <chr>     <dbl>
1 US         3.78
2 HA         4.90
3 AS         5.80

Peak Volume

Question: Which 2 dates had the most NYC flights (in 2013)?

delays |> 
  summarize(
    .by = c(month, day), 
    n_flights = n()
  ) |> 
  slice_max(n = 2, order_by = n_flights)

# A tibble: 2 × 3
  month   day n_flights
  <int> <int>     <int>
1    11    27      1014
2     7    11      1006

Grouped Slice

Grouping inside Slice

Question: What is the extreme value WITHIN each category?
We do not always have to summarize before we slice.
We can slice the raw data directly, and we can apply the by argument to slice_max() and slice_min().
- Note: Notice we use by here, not .by!

Instead of returning the top 1 row overall, this returns the top 1 row for each group.

The Worst Flights per Carrier

Question: What was the single most delayed flight for each airline?

delays |> 
  slice_max(
    n = 1, 
    order_by = dep_delay, 
    by = carrier
  )

# A tibble: 16 × 5
  month   day carrier flight dep_delay
  <int> <int> <chr>    <int>     <dbl>
1     7    26 UA         372       483
2     9    20 AA         177      1014
3     1    16 B6         517       502
4     4    10 DL        2391       960
5    12     5 EV        4711       548
6     6    15 MQ        3535      1137
# ℹ 10 more rows

Grouped Mutate

Question: How does a row compare to its peers?
We have already learned that mutate() creates new columns or alters existing ones.
What happens if we combine mutate() with the .by argument?

The new column is calculated within each group but keeps the original structure of the dataset.
This is useful for calculating deviations from group means or standardizing scores within categories.

Comparing to Group Means

Question: Was this specific flight worse than the carrier’s usual performance?

delays |> 
  mutate(
    .by = carrier,
    carrier_avg = mean(dep_delay, na.rm = TRUE),
    relative_delay = dep_delay - carrier_avg
  )

# A tibble: 336,776 × 7
  month   day carrier flight dep_delay carrier_avg relative_delay
  <int> <int> <chr>    <int>     <dbl>       <dbl>          <dbl>
1     1     1 UA        1545         2       12.1          -10.1 
2     1     1 UA        1714         4       12.1           -8.11
3     1     1 AA        1141         2        8.59          -6.59
4     1     1 B6         725        -1       13.0          -14.0 
5     1     1 DL         461        -6        9.26         -15.3 
6     1     1 UA        1696        -4       12.1          -16.1 
# ℹ 336,770 more rows

Foundations of Data Science

Roadmap

Basic Summarize

Summarize

Preparing the Data

Overall Averages

The summary() Pitfall

Counting Functions

Grouped Summarize

Grouped Summarize

Summarizing by One Group

Summarizing by Multiple Groups

Real-World Data

Average Delay by Carrier

Slice

The Slice Functions

Best Average Performance

Peak Volume

Grouped Slice

Grouping inside Slice

The Worst Flights per Carrier

Grouped Mutate

Grouped Mutate

Comparing to Group Means

Foundations of
Data Science