"DPLYR" -The Data Manipulator
Dirty Data sat me down on a Tuesday morning and said "You I will kill you"😂 but dplyr came to manipulate me out
Seeing the cover photo, you might think this is an action movie😂 However, as a data analyst, you would know that it's more than that.
INTRODUCTION
Data manipulation is a crucial step in the data analysis process. Whether you are exploring, cleaning, aggregating, or transforming data, it is essential to have a powerful and user-friendly tool at your disposal. In the world of R, one such tool that stands out is dplyr, appropriately named "The Data Manipulator". Developed by Hadley Wickham, dplyr is a highly popular and widely used package that offers a comprehensive set of functions for data manipulation tasks. Additionally, dplyr is a grammar of data manipulation in R, providing a consistent set of verbs that can help you solve the most common data manipulation challenges.
I know you were thinking he would soon give us a math example related to data manipulation 😂. But guess what? I have one! Let's check out this calculus question below.
Let's find the derivative of tan(x)
Solution
# Before we find d/dx of tan(x), we have to manipulate it by changing
it to sin(x)/cos(x) from our knowledge of "SOHCAHTOA"
A useful memory aid for remembering the definitions of the trigonometric functions sine, cosine, and tangent
i.e., sine = opposite/hypotenuse,
cosine = adjacent/hypotenuse,
tangent = opposite/adjacent.
Now that we have the derivatives of sine and cosine we use the quotient rule to solve
d/dx(tan(x))= d/dx(sin(x)/cos(x))
= cos(x)cos(x)−sin(x)(−sin(x))/(cos(x))^2
# Multiplying the numerator together gives
= cos^2(x)+sin^2(x)/cos^2(x)
# Now the manipulation comes in again to get our answer
Now, recall that cos^2(x)+sin^2(x) = 1
= 1/cos^2(x)
# In trig function the inverse of cosine is secant, So since have a square
the final answer becomes secant square
= sec^2(x)
So the the derivative of tan(x) equals sec^2(x)
As a mathematician, I don't always need to solve a problem to get the answer. This is because the rules and laws of mathematics allow us to manipulate questions in certain ways to obtain the answers we seek. Similarly, when we are analyzing data, we often need to manipulate it to make our analysis more efficient and effective. By following certain techniques, we can make sure our data analysis is accurate and reliable😎
Why dplyr?
Readability and Expressiveness
Dplyr's syntax is designed for clarity and expressiveness. Its functions read like English sentences, making code more readable. For example:
# Creating a data frame of heights
Height_data <- data.frame(height = c(160, 175, 150, 145, 180, 176))
# Filter the data frame based on the condition 'height > 150'
new_filtered_data <- filter(Height_data, height > 150)
# Print the filtered data frame
print(new_filtered_data)
Consistency
Dplyr functions follow a consistent naming convention, which simplifies the learning process. You will come across functions such as filter(), mutate(), select(), and arrange(), each designed to perform a specific task of data manipulation. This consistency makes it easier to recall and use these functions effectively.
Data Pipelines
The 'Dplyr' package is designed to work seamlessly with the pipe operator (%>%) from the 'magrittr' package. By using pipelines, you can easily create a sequence of data manipulation steps, where the output of one function serves as the input of the next. This approach is highly effective in chaining together multiple data manipulation steps in a neat and organized way, making your code efficient and easy to understand. For example:
# Creating a data frame
student_data <- data.frame(sex = c("M", "F", "M", "F"), age = c(20, 30, 35, 50), height = c(170, 165, 180, 175))
# Filter the data frame based on the condition `age > 30`
student_filtered_data <- filter(student_data, age > 30)
# Grouping the data frame by sex
result <- group_by(student_filtered_data, sex) %>%
# Calculating the mean height for each group
summarise(mean_height = mean(height))
# Print the result
print(result)
Dplyr is a flexible and adaptable tool that offers a wide range of uses. It is also thoroughly documented and backed by support.
Tidyverse Compatibility
The Dplyr package is a part of the tidyverse ecosystem, which includes other packages such as tidyr, ggplot2, and readr. These packages work together seamlessly, providing you with a cohesive and integrated toolkit for data analysis. With this integration, you can easily manipulate and visualize your data, and create detailed reports without any hassle.
Most common dplyr functions
Dplyr provides several core functions to tackle common data manipulation tasks:
filter(): Filter rows based on specified conditions.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where the Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Print the filtered data
print(s_filtered_data)
mutate(): Create new variables or modify existing ones.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Use arrange to sort the filtered data by Age in descending order
arranged_data <- arrange(s_filtered_data, desc(Age))
# Use mutate to create a new column called "Grade" based on Score
s_mutated_data <- mutate(arranged_data, Grade = ifelse(Score >= 90, "A", "B"))
# Print the mutated data
print(s_mutated_data)
select(): Select specific columns from a data frame.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Use arrange to sort the filtered data by Age in descending order
arranged_data <- arrange(s_filtered_data, desc(Age))
# Use mutate to create a new column called "Grade" based on Score
s_mutated_data <- mutate(arranged_data, Grade = ifelse(Score >= 90, "A", "B"))
# Use select to choose specific columns "Name and Grade"
selected_data <- select(s_mutated_data, Name, Grade)
# Print the selected data
print(selected_data)
arrange(): Reorder rows based on variable values.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where the Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Use arrange to sort the filtered data by Age in descending order
arranged_data <- arrange(s_filtered_data, desc(Age))
# Print the arranged data
print(arranged_data)
rename(): Rename columns in a data frame
# Creating a sample data frame
name_data <- data.frame(Name = c("Ayo", "Kayode", "John"),
Age = c(18, 25, 22))
# Rename the "Age" column to "Years"
rename_data <- rename(name_data, Years = Age)
# Display result
rename_data
group_by(): Group data by one or more variables.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Use arrange to sort the filtered data by Age in descending order
arranged_data <- arrange(s_filtered_data, desc(Age))
# Use mutate to create a new column called "Grade" based on Score
s_mutated_data <- mutate(arranged_data, Grade = ifelse(Score >= 90, "A", "B"))
# Use group_by to group the data by Grade
s_grouped_data <- group_by(s_mutated_data, Grade)
# Print the grouped data
print(s_grouped_data)
summarise(): Generate summary statistics for groups of data.
# Creating a sample data frame
sample_data <- data.frame(
Name = c("Ayo", "Kayode", "John", "David"),
Age = c(18, 24, 22, 20),
Score = c(85, 92, 78, 88)
)
# Use filter to select rows where Score is greater than 80
s_filtered_data <- filter(sample_data, Score > 80)
# Use arrange to sort the filtered data by Age in descending order
arranged_data <- arrange(s_filtered_data, desc(Age))
# Use mutate to create a new column called "Grade" based on Score
s_mutated_data <- mutate(arranged_data, Grade = ifelse(Score >= 90, "A", "B"))
# Use group_by to group the data by Grade
s_grouped_data <- group_by(s_mutated_data, Grade)
# Use summarise to calculate the mean score within each group
summary_data <- summarise(s_grouped_data, Mean_Score = mean(Score))
# Print the summary data
print(summary_data)
- Joins: To combine data from multiple tables that have common columns, you can use functions such as
left_join(),inner_join(), andright_join().
Getting Started with dplyr
To use dplyr, you must install and load the package in R :
install.packages("dplyr")
library(dplyr)
After loading the package, you can efficiently clean, transform, and analyze your data.
CONCLUSION
Dplyr is an essential tool in the arsenal of data analysts and scientists, owing to its intuitive syntax and extensive capabilities. It allows you to work with data efficiently, enabling you to discover insights, make informed decisions, and communicate compelling data-driven narratives.
I will be discussing the world of ggplot in my forthcoming article. Where I will provide a comprehensive understanding of how you can effectively use ggplot to create visually appealing and informative data visualizations.✨
I would greatly appreciate your support in sharing and liking this article. It will help to spread the message and reach a wider audience, which is important. Thank you for contributing to this cause🎉
Let’s Connect
Reach out to me on Linkedin
Reach out to me on the X app ( Kindly follow I'll follow back immediately )
“Cover photo” ―Postermywall
“GIF” ―Giphy