Overview

In today’s lab we will first get set up with the accounts and software you will use in this course. Next you will learn how to analyze data using R. At the end of today’s lab you will learn how to import data sets into R, manipulate data and produce visuals of this data using the software. You are encouraged to work with your neighbot on the assignment, however do not copy each other. The work you turn in should be your own.

Lab Submission

At the end of lab today you will upload your assignment as both a R Notebook file (.Rmd) and html file to our Canvas site. There is a template on the Canvas site to get you started with some tips on formatting you document. Your lab must be nicely organized and have a title and a header. - name your file using the format LabName_Lastname - for example for todays lab my file would be names “Lab1_Olson”

A very helpful guide on formatting R Markdown documents for assignments is available at http://www.stat.cmu.edu/~cshalizi/rmarkdown/.

Gettting Started

Go to https://docs.posit.co/cloud/get_started/. Follow the instructions to make a free account and create a new R Studio project.

Part 1 Working with R

You should follow along with instructions below by entering them into your R console as you read through this document.

Below is a code chunk to say hello. If you click the green arrow in the corner of the box it will run the code and print the message.

print('hello')
## [1] "hello"

You can also use R to do arithmetic and R will give you the answer as a calculator would.

100*2+5
## [1] 205

Note that R will follow the order of operations (PEMDAS). If you want to change this you can use parentheses. For example, you will get a different answer from above if we add parentheses to part of the equation.

100*(2+5)
## [1] 700

In the example about the addition ‘+’ is a function. There are many functions within the R base package. There are also functions you can add to your work space environment by installing additional packages. To do this you can use the R Studio interface to search under the Packages tab. Note that the echo = FALSE parameter was added to the code chunk to prevent printing of the R code that generated. This can be useful when you don’t need to see the outcome. Instead of using the Packages tab you can use the command line like below. When you make Rmd documents for future labs you will want to include these commands at the beggining to insure that your code will work on any computer.

## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.5'
## (as 'lib' is unspecified)

Once you have installed a package you can load it with the libary() command.

library('tidyverse')
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.6
## ✔ forcats   1.0.1     ✔ stringr   1.6.0
## ✔ ggplot2   4.0.1     ✔ tibble    3.3.0
## ✔ lubridate 1.9.4     ✔ tidyr     1.3.2
## ✔ purrr     1.2.0     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

An important function to know is the get working directory command. This is where the code is hosted and where it will look for files if not given another file path.

getwd() 
## [1] "/cloud/project"

You can name variable using the <- command. Variable can be numeric or characters.

 name<- "Elizabeth"
  height<- 63 

You can also create lists of data.

list_names<- c('lisa','joe','sam','trevor','eric')
list_heights<- c(63,54,68,71,75)
list_notes<-c(5, '10 ml', 3,2,1)

You can then combine the lists into a dataframe.

heights_df<-data.frame(list_names,list_heights, list_notes)
print(heights_df)
##   list_names list_heights list_notes
## 1       lisa           63          5
## 2        joe           54      10 ml
## 3        sam           68          3
## 4     trevor           71          2
## 5       eric           75          1

Notice how the data frame we created above has listed data type beneath the row headers and . This shows how R will read the data for function input. The way data is coded in is important and will determine how you can use it. We can simplify our workflow by using the tidyverse package and the commands within. The tidyverse package uses tibbles instead of data frames and these are easier to work with.

heights_tib <- tibble(list_names, list_heights, list_notes) 

You can see what files are in your work space using the ls function

ls()
## [1] "height"       "heights_df"   "heights_tib"  "list_heights" "list_names"  
## [6] "list_notes"   "name"

And you can remove objects with the rm function.

rm(heights_df)
ls()
## [1] "height"       "heights_tib"  "list_heights" "list_names"   "list_notes"  
## [6] "name"

You have been working with the code here in your own R console. This is the html output of an Rmd file. You can see how this lab looks as the .Rmd format by clicking the link here(https://utchattanooga.instructure.com/courses/49070/files/9889244?wrap=1).

Part 2 Getting Started with Data Analysis

For this portion of the lab you will create your own Rmd document in Posit and document your work. Follow the lab submission guidelines above. You can also find a helpful R Markdown “cheat-sheet” at https://rstudio.github.io/cheatsheets/rmarkdown.pdf. When you are ready to save and publish your work, click the Knit icon at the top left of the tab. Then in the File tab check the box next to your .Rmd file and select the ‘More’ drop down at the top. Select ‘Export’ to download your file. Do the same thing for your html file.

Today we will investigate the March 15-17, 1973 flood in Chattanooga.

Importing Data

Visit the NOAA Climate DAta Onlien Search https://www.ncdc.noaa.gov/cdo-web/search.

To search for the data select ‘Precipitaion Hourly’ from the drop down. Then for date range select the week of the flood 03/11/73- 03/18/73. Then select ‘Cities’ in search type and Search for ‘Chattanooga’.

Once the search results appear, click on add to cart in the icon box on the left hand side menu. Then you will click on your cart in the upper right hand corner and ‘check out’ your data. Select the csv option for file type. The data link will be sent to the email you enter at the check out. This is a small dataset, it won’t take long for you to get an email telling you the dataset is ready. Download the data by clicking the link in the email.

Each data subset is downloaded with a unique order number. The order number in our example dataset is 4198967. If you are using a dataset you’ve downloaded yourself, make sure to substitute in your own order number in the code below.

First lets import the data and rename the file in our local environment. In the Files tab on the lower right side of the window pane click ‘Upload’. Then navigate to your file and select it to upload. It should now appear in the list on the lower right. Next in the ‘Environment’ tab on the upper right click ‘Import Dataset’ and select ‘From text base’ and then select the file. When you see the import preview select import. Notice that in the Console below the code for import has printed. You can add this code into your Rmd file to import automatically so your code will work everytime without the manual input

`4198967` <- read.csv("/cloud/project/4198967.csv")

We can also change the file name to make it easier to work with.

precip_chat<- `4198967`

You can view your newly imported dataset by clicking the table icon next to it in the Environment tab. Or by typing the command below. Which will show you the first six lines. This also shows the data types and header names.

head(precip_chat)
##       STATION              STATION_NAME           DATE HPCP
## 1 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 07:00 0.01
## 2 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 09:00 0.08
## 3 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 10:00 0.20
## 4 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 11:00 0.71
## 5 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 12:00 0.28
## 6 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 13:00 0.10

You can always check a file’s structure and dat types within by using the str() function.

str(precip_chat)
## 'data.frame':    41 obs. of  4 variables:
##  $ STATION     : chr  "COOP:401656" "COOP:401656" "COOP:401656" "COOP:401656" ...
##  $ STATION_NAME: chr  "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" ...
##  $ DATE        : chr  "19730311 07:00" "19730311 09:00" "19730311 10:00" "19730311 11:00" ...
##  $ HPCP        : num  0.01 0.08 0.2 0.71 0.28 0.1 0.03 0.37 0.01 0.09 ...

Answer the following questions about the dataset:

  1. How many rows and columns are there in the dataset? There is a function that will give you the dimensions of an object (i.e. number of rows and columns). Look at your Base R Cheatsheet (https://iqss.github.io/dss-workshops/R/Rintro/base-r-cheat-sheet.pdf) to find this function. Include the code you used to find this.
#your code here 

Notice that the DATE column is listed as a data class character. We will need to change this to a date class ‘POSIXct’ in order to work with the time series.

# convert to date/time and retain as a new field
precip_chat$DateTime <- as.POSIXct(precip_chat$DATE, 
                                  format="%Y%m%d %H:%M") 
                                  # date in the format: YearMonthDay Hour:Minute 

# double check structure
str(precip_chat$DateTime)
##  POSIXct[1:41], format: "1973-03-11 07:00:00" "1973-03-11 09:00:00" "1973-03-11 10:00:00" ...

We also need to check the dataset for missing values. The column HPCP is the total precipitation given in inches (since we selected Standard for the units), recorded for the hour ending at the time specified by DATE. Importantly, the metadata (see below) notes that the value 999.99 indicates missing data. Also important, hours with no precipitation are not recorded. To clean the data set first lets see if there are any missing data.

hist(precip_chat$HPCP)

The range of values in our histogram above indicates there is no missing data in our data set. If there were we could asign the values an ‘NA’ to correct for the missing values when plotting.

# assing NoData values to NA
precip_chat$HPCP[precip_chat$HPCP==999.99] <- NA

To check how many values are now NA we use the sum function.

sum(is.na(precip_chat))
## [1] 0

Now that the dataset is clean and the date is in the correct format lets plot the data. To do this we will use the ggplot2 package.

install.packages('ggplot2') #install package 
## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.5'
## (as 'lib' is unspecified)
library(ggplot2) # load package to environment 

#plot the data
precPlot_hourly <- ggplot(data=precip_chat,  # the data frame
      aes(DateTime, HPCP)) +   # the variables of interest
      geom_bar(stat="identity") +   # create a bar graph
      xlab("Date") + ylab("Precipitation (Inches)") +  # label the x & y axes
      ggtitle("Hourly Precipitation - Chattanooga Station\n 1973 Flood")  # add a title

precPlot_hourly

You can modify your plot in various ways. See this cheat sheet (https://posit.co/wp-content/uploads/2022/10/data-visualization-1.pdf)for tips on using ggplot2. For example we can make the plot blue by adding the color = command to the geom_car line. There are many colors in R, see this cheat sheet for examples (https://www.nceas.ucsb.edu/sites/default/files/2020-04/colorPaletteCheatsheet.pdf). When submitting your lab, change the color in the plot below to something other than ‘blue’.

#plot the data
precPlot_hourly <- ggplot(data=precip_chat,  # the data frame
      aes(DateTime, HPCP)) +   # the variables of interest
      geom_bar(stat="identity", color ="blue") +   # create a bar graph
      xlab("Date") + ylab("Precipitation (Inches)") +  # label the x & y axes
      ggtitle("Hourly Precipitation - Chattanooga Station\n 1973 Flood")  # add a title

precPlot_hourly

We can also zoom into a specific part of the dataset by filtering before plotting.

precPlot_floodday <- precip_chat %>% 
   filter(DateTime >"1973-03-16" & DateTime< "1973-03-17") %>%
  ggplot( aes(DateTime, HPCP)) +   # the variables of interest
      geom_bar(stat="identity", fill ="chartreuse") +   # create a bar graph
      xlab("Date") + ylab("Precipitation (Inches)") +  # label the x & y axes
      ggtitle("Hourly Precipitation - Chattanooga Station\n 1973")  # add a title

precPlot_floodday

The plot above shows that the highest hourly rainfall occured on the morning of March 16th. We can find out the exact amount by using the summary function.

summary(precip_chat)
##    STATION          STATION_NAME           DATE                HPCP       
##  Length:41          Length:41          Length:41          Min.   :0.0100  
##  Class :character   Class :character   Class :character   1st Qu.:0.0700  
##  Mode  :character   Mode  :character   Mode  :character   Median :0.1100  
##                                                           Mean   :0.2173  
##                                                           3rd Qu.:0.2800  
##                                                           Max.   :1.1700  
##     DateTime                  
##  Min.   :1973-03-11 07:00:00  
##  1st Qu.:1973-03-15 13:00:00  
##  Median :1973-03-15 23:00:00  
##  Mean   :1973-03-15 07:54:08  
##  3rd Qu.:1973-03-16 10:00:00  
##  Max.   :1973-03-17 00:00:00

Answer the following questions about the dataset:

  1. What is the sum of precipitation for the length of our datatset? There is a function that will give you the dimensions of an object (i.e. number of rows and columns). Look at your Base R Cheatsheet (https://iqss.github.io/dss-workshops/R/Rintro/base-r-cheat-sheet.pdf) to find this function. Include your code below.
#your code here 
  1. Plot your data in millimeters instead of inches. You can do this my creating a new column of data and using the calculator functions to multiple the inches by the conversion factor. Show your code below.
# your code here
# precip_chat$newprecipinmm<- precip_chat$oldprecipdataininches * conversion factor
# ggplot(data = ...) + geom_point(aes(x = ... , y = ... , color = ... ))

Part 3 Exploring Historic Data (Graduate Students Mandatory - Undergraduates Extra Credit)

In this section of the lab you will explore data of your choosing.Go back to the NOAA website (https://www.ncdc.noaa.gov/cdo-web/search). And search for a data set of interest. Explore the dataset as you did above and provide your code below. BE sure to include a plot fo the data.

Note that this section of the lab is important and should not be given only a cursory work through. An important learning goal for this term is for you to develop your own independent research skill. Thus, all of our labs this term will have a large, independent component where you are expected to apply what you’ve learned to ask novel and interesting questions and furthermore to try and go beyond what you’ve learned in class.

When trying to get started with something new, remember the copy/paste/tweak approach. This approach can help give you a good starting point for your work.

#your code here

Submit your files on Canvas before lab next week.