R Data Cleaning

R is a free programming language and software environment for statistical computing, data analysis, and graphics. RStudio (developed by Posit) is an integrated development environment (IDE) that makes working with R easier by providing a script editor, console, workspace viewer, file manager, package manager, and plotting window in one interface. RStudio requires R to be installed first.

Download links:
R (CRAN)
RStudio Desktop

Installation order:

• Download and install R from CRAN.
• Download and install RStudio Desktop from Posit.
• Open RStudio, which will automatically detect your R installation and provide a complete development environment for writing and running R code.

Install packages once

install.packages(c(   
  "tidyverse", "readxl", "janitor",   
  "lubridate", "stringr", "skimr"   
))

Load packages in every new session

library(tidyverse)   # import, transform and visualise data   
library(readxl)      # read Excel workbooks   
library(janitor)     # standardise names and inspect tables   
library(lubridate)   # parse and work with dates   
library(stringr)     # clean and standardise text   
library(skimr)       # compact data-quality summaries

Remember: install.packages() downloads a package to your computer and usually only needs to be run once. library() makes the package available in the current R session and must be run again after restarting R.

Import the file into a new R object. Use a name such as raw_df so it is clear that the object represents the original imported data. Do not use spreadsheet formatting, colour, merged cells, or blank rows as data-bearing information because R cannot interpret these reliably.

raw_df

When decimal marks, missing-value codes, encodings, or delimiters differ from the defaults, specify them explicitly during import. This is safer than fixing a wrongly imported table later.

raw_df

#For example use a dataset included with R
raw_df 

Inspection shows what R actually imported. Never assume that a column containing numbers has been read as numeric, or that dates have been interpreted correctly. Look at both the structure and the values.

head(raw_df)       # first rows   
tail(raw_df)       # last rows   
View(raw_df)       # spreadsheet-style viewer   
glimpse(raw_df)    # variables, types and example values   
str(raw_df)        # detailed object structure   
summary(raw_df)    # basic numerical and categorical summaries   
skim(raw_df)       # broader data-quality overview   
names(raw_df)      # column names   
dim(raw_df)        # number of rows and columns

During inspection, check for:

• columns imported with the wrong data type;
• missing values represented by several different codes;
• duplicate rows or repeated identifiers;
• unexpected spelling or capitalisation in categories;
• impossible values, such as a negative tree diameter or a date outside the study period;
• units that are missing, mixed, or undocumented;
• rows containing notes, subtotals, or metadata rather than observations.

Consistent names and types are necessary for reliable analysis. Variable names should be short but meaningful, use a single naming convention, and not contain spaces. The janitor::clean_names() function converts names to a consistent snake_case format.

clean_df 
  clean_names()

names(clean_df)

Convert each variable to the type needed for analysis. Always check the result because conversion can introduce missing values if a value cannot be parsed.

clean_df 
  mutate(
    tree_id = as.character(tree_id),
    site = as.factor(site),
    diameter_cm = as.numeric(diameter_cm),
    alive = as.logical(alive)
  )

summary(clean_df)

Caution: Do not convert a factor directly to numeric with as.numeric() unless you want the underlying factor codes. Convert it to character first, then to numeric.

Subsetting creates a focused working table while keeping the complete imported dataset. Use explicit code instead of deleting rows or columns manually.

analysis_df 
  select(ozone, solar_r, wind, temp, month, day) |>
  filter(!is.na(ozone), temp > 70) |>
  arrange(month, day)

Common structural operations include:

select(clean_df, ozone, wind, temp) # keep columns   
filter(clean_df, month == 7)                 # keep matching rows   
slice(clean_df, 1:10)                        # keep rows by position   
arrange(clean_df, desc(temp))                # sort rows   
distinct(clean_df)                           # remove exact duplicate rows   
relocate(clean_df, month, day)               # move columns   
rename(clean_df, temperature_f = temp)       # rename a variable   
mutate(clean_df, temp_c = (temp - 32) * 5/9) # create a variable

Missing values should be handled according to their meaning and the intended analysis. Removing every row with a missing value is mostly a poor choice. First identify where missingness occurs, then decide whether to keep, exclude, recode, or impute values. Record the reason for each choice.

# Count missing values in each column
clean_df |>
summarise(across(everything(), ~ sum(is.na(.))))

# Inspect rows with a missing ozone measurement
clean_df |>
filter(is.na(ozone))

# Remove rows only when ozone is essential for this analysis
analysis_df 
filter(!is.na(ozone))

Use replace_na() only when a replacement has a clear meaning. For example, replacing a missing count with zero is valid only when the absence truly means that none were observed, not when the observation was not made.

# Example: label missing homeworld values explicitly as "Unknown"   
text_example 
select(name, homeworld) |>   
mutate(homeworld = replace_na(homeworld, "Unknown"))   

head(text_example)

Small differences in spacing, spelling, or case create separate categories. Clean strings before converting them to factors or calculating grouped summaries.

# starwars is included with dplyr and contains text categories   
text_example   
  select(name, sex, homeworld) |>   
  mutate(   
  name = str_squish(name),   
  homeworld = str_to_title(homeworld),   
  sex = str_to_lower(sex)   
)   

count(text_df, sex, sort = TRUE)

Keep a documented lookup table when many labels need to be harmonised. This is clearer than putting dozens of replacements in a single script.

Dates imported from text must be converted to a proper Date or date-time class. Choose the parser that matches the source order.

clean_df   
  mutate(   
  sampling_date = make_date(1973, month, day),   
  year = year(sampling_date),   
  month_label = month(sampling_date, label = TRUE),   
  day_of_year = yday(sampling_date)   
  )

Use ymd() for year-month-day, mdy() for month-day-year, and dmy() for day-month-year. After conversion, check the minimum, maximum, and number of missing dates.

range(clean_df$sampling_date, na.rm = TRUE)   
sum(is.na(clean_df$sampling_date))

An exact duplicate is not always an error, and a repeated identifier may be valid in longitudinal or repeated-measures data. Define what should be unique for the study.

# Exact duplicate rows   
sum(duplicated(clean_df))   
   
# Potential duplicate observations based on a key   
clean_df |>   
  count(month, day) |>   
  filter(n > 1)   
   
# Repeated identifiers can be valid: see the built-in ChickWeight data   
ChickWeight |>   
  count(Chick) |>   
  arrange(desc(n)) |>   
  head()

Validate plausible ranges and allowed categories. Flag questionable values instead of deleting them without notes.

quality_flags     
mutate(   
ozone_flag = ozone > 150   
)   

quality_flags |>   
filter(ozone_flag)

Cleaning is finished only after the output has been checked. Compare row counts, review distributions, inspect categories, and make sure that transformations produced the intended result.

nrow(raw_df)   
nrow(clean_df)   

summary(clean_df)   
count(clean_df, month, sort = TRUE)   
sum(duplicated(clean_df))   

clean_df |>   
  summarise(   
  n = n(),   
  mean_ozone = mean(ozone, na.rm = TRUE),   
  minimum_temp = min(temp, na.rm = TRUE),   
  maximum_temp = max(temp, na.rm = TRUE)   
  )

Visual checks can reveal outliers, skewed distributions, gaps, discontinuities, or coding errors that are hard to spot in a table.

ggplot(clean_df, aes(x = temp, y = ozone)) +   
  geom_point() +   
  geom_smooth(method = "loess", se = FALSE) +   
  labs(   
  title = "Relationship between temperature and ozone",   
  x = "Temperature (°F)",   
  y = "Ozone"   
  ) +   
  theme_minimal()

Choosing a suitable plot

The native R pipe, |>, passes the result of one step to the next. This lets the script to read from top to bottom as a sequence of documented decisions. Create a new object instead of overwriting raw_df.

clean_df 
  clean_names() |>   
  mutate(   
  sampling_date = make_date(1973, month, day),   
  temp_c = (temp - 32) * 5/9   
  ) |>   
  filter(!is.na(ozone)) |>   
  distinct() |>   
  arrange(sampling_date)

Export cleaned data to the processed-data folder. Open, machine-readable formats like CSV or TSV are usually suitable for tabular data. Save an R-specific file as an extra working copy when keeping classes and attributes is important.

dir.create("data_processed", showWarnings = FALSE)

write_csv(clean_df, "data_processed/airquality_clean.csv")   
saveRDS(clean_df, "data_processed/airquality_clean.rds")

The project README and data dictionary should explain:

• the purpose and scope of the dataset;
• who collected or created the data and when;
• the meaning, units, and allowed values of every variable;
• the missing-value codes used in the raw data;
• all major cleaning, exclusion, and transformation rules;
• the software and package versions needed to reproduce the workflow;
• the licence, access conditions, and recommended citation;
• the relationship between raw files, processed files, scripts, and outputs.

Living documentation: Update the README and data dictionary whenever the dataset, workflow, or file structure changes. Documentation written only at the end of a project is often incomplete.

The following script combines the main stages into one reproducible workflow. Adapt the variable names and validation rules to your actual dataset rather than copying them without review.

        
library(tidyverse)
library(janitor)
library(lubridate)
library(stringr)
library(skimr)

# 1. Import a dataset included with R
raw_df 
clean_names() |>
mutate(
  sampling_date = make_date(1973, month, day),
  temp_c = (temp - 32) * 5/9
) |>
distinct() |>
arrange(sampling_date)

# 4. Validate
stopifnot(all(clean_df$temp > 0, na.rm = TRUE))
stopifnot(!anyDuplicated(clean_df[c("month", "day")]))

clean_df |>
summarise(across(everything(), ~ sum(is.na(.)))) |>
print()

# 5. Export
dir.create("data_processed", showWarnings = FALSE)
write_csv(clean_df, "data_processed/airquality_clean.csv")
saveRDS(clean_df, "data_processed/airquality_clean.rds")