Data Wrangling and Visualization with Tidyverse in R | R Programming

In this module, you will learn how to install R and RStudio. You will also learn how to make the best use of the R Markdown within RStudio. RStudio is an IDE (Integrated Development Environment) that makes learning R much easier. In RStudio, you can run not only R codes but also Python codes or codes for other programming languages. RStudio is wonderful. With this preparation done, you will learn what you can do with R. While the program will teach you Base R, it will focus on the Tidyverse approach of data science with the use of the Tidyverse package.

Upon successful completion of this module, you will be able to:

1. Install R and RStudio.
2. Describe the layout and menus of RStudio.
3. Start, Run codes, and Save an R script file.
4. Install R packages and load them up.
5. Start, Run codes, organize the codes, and save the R Markdown file.

The goal of this module is to introduce an overview of the world of data science in R and get you ready for the rest of the modules. 

Specifically, you will learn the universe of R and data science in general in this module.  As R spreads to the academic and research community, more and more college students are learning statistics with R. R was initially developed as a statistical tool.  What else can we do with R? What do data scientists do with R?

For starters, we can create a chart any way we want with ggplot2. You can create and update Word, HTML, PPT, and PDF files right from the R markdown file. You can animate your charts and make your charts interactive. You can create a website and a dashboard with shiny and shiny Dashboard.  You can build machine learning models with Caret or tidy models. You can run Python in R with the reticulate package. Some packages help you create charts with a menu-driven approach (e.g., ggThemeAssist and esquisse).

In this module, you will learn R's capabilities and resources that you can use to learn the skills. There is no way you can learn everything in this short course. Thus, we will focus on some fundamental topics, while you will be led to resources for advanced topics you can tackle in the future.

Upon successful completion of this module, you will be able to:

1. Describe the concept of the Tidverse way of coding.
2. List the Pros and Cons of R and Python for data science.
3. Describe the capability of the Tidyverse package in R.
4. Explain how to use online resources provided by the R community.

The ggplot 2 is a plotting package that provides helpful commands to create complex plots from data. It provides a more programmatic interface for specifying what variables to plot, how they are displayed, and general visual properties. Therefore, we only need minimal changes if the underlying data changes or if we decide to change from a bar plot to a scatterplot. This helps in creating publication-quality plots with minimal amounts of adjustments and tweaking. Therefore, it is not surprising that ggplot2 is included in Tidyverse.

In this module, you will be introduced to concepts of the grammar of graphics when visualizing data with the ggplot2 package. Every ggplot starts with defining data and the aesthetics of variables, which is the foundation. Then, you add a geometry layer over the foundation. The two are the basics of any ggplots. The rest of the layers are optional. You will also be introduced to how to create charts when there is only one (or more) continuous or categorical variable(s) in your data set. Furthermore, you will also be introduced to plot charts after you select and filter the variables you want to plot in your data set.

Upon successful completion of this module, you will be able to:

1. Explain the concept of the grammar of graphics when visualizing data with the ggplot2 package.
2. Be familiar with various types of charts.
3. Visualize counts, proportions, and geospatial data.
4. Select appropriate charts based on strategic considerations (e.g., the characteristics of the data and audience).
5. Create a chart that involves one or two variables. 
6. Create a chart by adding a categorical moderator (3rd variable) to the chart involving two variables. 
7. Create correlation charts.

In this module, you will be introduced to advanced topics of the grammar of graphics and ggplot2 extensions such as patchwork and gganimate. You will also observe how data scientists use ggplot2 to customize charts for better communication, including live screencast demonstrations of clever and creative data visualizations. Advanced users or motivated beginners are encouraged to tackle the advanced visualization codebook with interactive, animated, or geospatial charts. 

 Upon successful completion of this module, you will be able to:

1. Customize correlation charts.
2. Create a chart that involves four or five variables by adding layers of geometry.
3. Modify charts for storytelling by customizing axes, labels, coordinates, themes, etc.
4. Explain when to use interactive or animated charts or dashboards.
5. Visualize geospatial data on a map.
6. Read charts and generate insights.

Understanding the concept of Data Wrangling is important because many professionals inside technology industries have to face different data types and also deal with the various sources of data. In this module, you will be introduced to Data Wrangling. In a broad sense, data wrangling includes (1) importing data, (2) tidying data, and (3) transforming data. After the wrangling process, you can proceed with visualizing data and modeling. 

In this module, we will be focusing on the last two stages of the wrangling process -- tidying and transforming data. You will also be introduced to a modern data type called, tibbles and learn how tibbles are different from data frames. One of the first steps in data wrangling in the Tidyverse ecosystem is to reshape the data to be tidy using "tidyr" package. You will learn the concept of tidy data, and how reshaping the data helps you visualize the data. Once you make your data tidy, you will want to transform your data with the help of "dplyr" package. Learn some key functions from dplyr to transform the data. In base R, one had to use several parentheses together, adding a code inside the parentheses and inside yet another parenthesis, making coding complex. This makes your R code hard to read and understand. In modern R, you can zoom on data, using the pipe operator (%>% or |>), which allows information to move from top to bottom through a pipe. That is, you first start with data, then pipe into the data to tidy or transform it first. Once wrangling is done, you pipe the wrangled data into ggplot() to do all the necessary mapping and geometry as well as other optional layers of coordinates and themes, according to the grammar of graphics. 

Upon successful completion of this module, you will be able to:

1. Describe the concept of Data Wrangling.
2. Describe how Tibbles are different from data frames.
3. Explain how to convert wide or long data to "Tidy" data.
4. Explain how to merge relational data sets using joins.
5. Be familiar with key dplyr verbs and use them to transform data.
6. Use the pipe operator to shape the data to prepare for analysis and visualization.

Continuing from M05, you will expand your horizons by going deeper into the topics with various approaches. 

First, you will learn how data scientists wrangle and visualize data for their projects by watching demonstrations. Next, you will be exposed to more advanced topics and associated functions such as recode(), across(), case_when(), rownames_to_column(), distinct(), rowwise(), and c_across().

Upon successful completion of this module, you will be able to:

1. Describe the concept of Data Wrangling.
2. Describe how Tibbles are different from data frames.
3. Explain how to convert wide or long data to "Tidy" data.
4. Explain how to merge relational data sets using joins.
5. Be familiar with key dplyr verbs and use them to transform data.
6. Use the pipe operator to shape the data to prepare for analysis and visualization.

Data types (numeric, character, logical, factor, dates) are building blocks of data structures (vectors, matrices, data frames, and lists). Most of the time, you are likely to work with data frames or Tibbles in the Tidyverse framework, which is a spreadsheet with rows of observations and columns of features or variables. Matrices are similar to data frames in that they consist of rows and columns. A major difference is that while matrices are composed of variables that are of the same data type, data frames can have a mix of any type of data. You can perform various operations on data to filter and view parts of data or to create new variables. Knowing the differences between the data types and structures will provide you with the basic knowledge needed to manipulate data later. 

In addition to Base R functions, Tidyverse has many packages that help deal with different types of vectors much more efficiently than base R does. With the stringr package, you can wrangle strings (or characters) and regular expressions. With forcats, you can wrangle factor, the ordered data. With the lubridate package, you can wrangle dates and times.  

Upon successful completion of this module, you will be able to:

1. Explain atomic R data types - numeric, character, logical, factors, and dates.
2. Explain the differences between data structures -- vectors, matrices, data frames, and lists.
3. Understand when to use each data type.
4. Create data frames.
5. Use popular functions from base R and Tidyverse to view and manipulate data frames that contain the various data types.
6. Use popular functions from stringr package to handle strings and regular expressions.
7. Use popular functions from the forcats package to manipulate factors in R.
8. Use popular functions from the lubridate package to manipulate dates.

Building on what you learned in the previous module, you will deepen your understanding of the R data types with particular emphasis on tidyverse.

You will develop a practical sense of choosing appropriate functions from various packages associated with certain data types (stringr, forcats, and lubridate) by watching the live performances and well-prepared demonstrations on using those packages.  Further, you will also be given the opportunity to drill with more advanced topics with those packages. 

Upon successful completion of this module, you will be able to:

1. Use advanced functions from base R and Tidyverse to view and manipulate data frames that contain the various data types.
2. Use advanced functions from stringr package to handle strings and regular expressions.
3. Use advanced functions from the forcats package to manipulate factors in R.
4. Manipulate dates and times using advanced functions from the lubridate package.
5. Utilize cheat cheats available online to find the right functions quickly.

Being able to import different formats of data is important because data scientists deal with secondary data collected by others, and secondary data exist everywhere inside and outside the organizations they work for. Therefore, learning how to import and export data is a fundamental step that we are introducing after you learn about data types and structures in Modules 7 and 8.  You must use appropriate tools to import and wrangle the data depending on the data formats.  Also, gathering data outside your company, such as websites, is a great way to extend your organization's capability, and you will be in the center of the action, adding value to your company; thus, web scraping can be another form of importing data that you can have under your took box.  Further, you will learn how to deal with so-called labelled data produced by menu-driven proprietary statistical software such as SAS, SPSS, and STATA.

In addition to the Base R functions, Tidyverse has some packages that help you deal with different formats of data created by well-known software. 

Upon successful completion of this module, you will be able to:

1. Explain how to create a Github repository and collaborate with others on the same R projects.
2. Effectively load and look through built-in datasets in R.
3. Import various formats of data (csv, xlsx, and SPSS) to RStudio.
4. Scrape data from the web using SelectorGadget and rvest.
5. Import multiple external data sets and work with them.
6. Work with labeled data in R.
7. Export/save output data to local pc and push to Github.

Building on what you learned in the previous module about data import and export, you will learn how to share outputs more effectively in this module. That is, you will learn the concept of literate coding and reproducible research using tools such as R Markdown and Quarto. Second, you will learn how to install an interactive visualization dashboard online.

Upon successful completion of this module, you will be able to:

1. Explain how reproducible research with literate coding enhances efficiency, ethics, transparency, and credibility.
2. Explain how Quarter is different from Rmd.
3. Describe how to make an interactive dashboard with Shiny R.
4. Produce customized tables from imported SPSS data effectively in R.
5. Create charts from imported SPSS data in R.

The ability to create a function of your own will make you get even closer to being a data scientist. Although R has several built-in functions of high usage, you will want to be able to create your own functions tailored for your job or tasks, because you found yourself or your team performing a particular task repeatedly. By building your own function, you can help save time. 

In this module, you will be introduced to various types of vectors, logical operators, for-loops, if-else statements. You will also be introduced to good coding practice and see how this positively impacts your code's efficiency. For your code to be more readable, take fewer keystrokes, and execute a batch of jobs faster, you may want to use a family of map() functions from purrr package, which is part of Tidyverse, or use apply() family functions from base R. Furthermore, you will also be introduced to some built-in R functions, which are built by R architecture, using the same sets of principles you will learn in this module. After you finish this module, you should be considered a programmer.

Upon successful completion of this module, you will be able to:

1. Describe good coding practices.
2. Use the various types of logical operators.
3. Use for loops in conjunction with other statements (if-else, next, break, etc).
4. Create your own R functions.
5. Use map functions from the purrr package to increase efficiency.
6. Use a family of apply() functions to simplify repetitive tasks.
7. Be familiar with some built-in R functions.

Building on the fundamentals of programming you learned in the previous module, you will deepen your understanding of R programming in this module. You will also practice more on a family of map() functions from purrr package.  Further, you will be exposed to more packages and tools that allow you to go beyond the familiar Tidyverse ecosystem. 

Specifically, we will go deeper into Functions and iterations with more explanations and examples to practice. Then, you will learn more about the purr package and its family of map functions with demonstrations and exercises, including walk(), walk2(), pwalk().

As the concept is hard, you will be given many examples with which you can appreciate the concepts better. For instance, you will be introduced to a way to test thousands of statistical tests using purr and broom packages. 

Upon successful completion of this module, you will be able to:

1. Explain a good coding style.
2. Describe all components of a function and their roles.
3. Set return values and describe the environment in a function.
4. Explain the differences between various For loop variations.
5. Map over multiple arguments using map2() and pmap().
6. Use walk(), walk2(), and pwalk() in the middle of the pipelines.

Working on a hands-on project is a great way to solidify your learning. Such a comprehensive, practical project can be added to a resume, portfolio, or LinkedIn profile. Use all the tools you learned from all 12 modules. Sometimes, you may have to use additional materials given in each module.

The Capstone project will be challenging due to its scope and nature. If you worked hard on all 12 modules, you will have an idea to try, but you are not likely to remember all the functions you need. Review the codebooks and find the codes that you can apply to the new content. 

Upon successful completion of this module, you will be able to:

1. Be familiar with R data structure.
2. Perform various operations to view and manipulate data
3. Import various types of data and export output data to other types of data.
4. Wrangle data to “tidy” form for visualization and modeling.
5. Write codes to automate/simplify routine operation.
6. Create charts, using R’s built-in functions as well as popular packages such as ggplot2.