In today’s lab we will first get set up with the accounts and software you will use in this course. Next you will learn how to analyze data using R. At the end of today’s lab you will learn how to import data sets into R, manipulate data and produce visuals of this data using the software. You are encouraged to work with your neighbot on the assignment, however do not copy each other. The work you turn in should be your own.
At the end of lab today you will upload your assignment as both a R Notebook file (.Rmd) and html file to our Canvas site. There is a template on the Canvas site to get you started with some tips on formatting you document. Your lab must be nicely organized and have a title and a header. - name your file using the format LabName_Lastname - for example for todays lab my file would be names “Lab1_Olson”
A very helpful guide on formatting R Markdown documents for assignments is available at http://www.stat.cmu.edu/~cshalizi/rmarkdown/.
Go to https://docs.posit.co/cloud/get_started/. Follow the instructions to make a free account and create a new R Studio project.
You should follow along with instructions below by entering them into your R console as you read through this document.
Below is a code chunk to say hello. If you click the green arrow in the corner of the box it will run the code and print the message.
print('hello')
## [1] "hello"
You can also use R to do arithmetic and R will give you the answer as a calculator would.
100*2+5
## [1] 205
Note that R will follow the order of operations (PEMDAS). If you want to change this you can use parentheses. For example, you will get a different answer from above if we add parentheses to part of the equation.
100*(2+5)
## [1] 700
In the example about the addition ‘+’ is a function. There are many
functions within the R base package. There are also functions you can
add to your work space environment by installing additional packages. To
do this you can use the R Studio interface to search under the Packages
tab. Note that the echo = FALSE parameter was added to the
code chunk to prevent printing of the R code that generated. This can be
useful when you don’t need to see the outcome. Instead of using the
Packages tab you can use the command line like below. When you make Rmd
documents for future labs you will want to include these commands at the
beggining to insure that your code will work on any computer.
## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.5'
## (as 'lib' is unspecified)
Once you have installed a package you can load it with the libary() command.
library('tidyverse')
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.4 ✔ readr 2.1.6
## ✔ forcats 1.0.1 ✔ stringr 1.6.0
## ✔ ggplot2 4.0.1 ✔ tibble 3.3.0
## ✔ lubridate 1.9.4 ✔ tidyr 1.3.2
## ✔ purrr 1.2.0
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
An important function to know is the get working directory command. This is where the code is hosted and where it will look for files if not given another file path.
getwd()
## [1] "/cloud/project"
You can name variable using the <- command. Variable can be numeric or characters.
name<- "Elizabeth"
height<- 63
You can also create lists of data.
list_names<- c('lisa','joe','sam','trevor','eric')
list_heights<- c(63,54,68,71,75)
list_notes<-c(5, '10 ml', 3,2,1)
You can then combine the lists into a dataframe.
heights_df<-data.frame(list_names,list_heights, list_notes)
print(heights_df)
## list_names list_heights list_notes
## 1 lisa 63 5
## 2 joe 54 10 ml
## 3 sam 68 3
## 4 trevor 71 2
## 5 eric 75 1
Notice how the data frame we created above has listed data type
beneath the row headers
heights_tib <- tibble(list_names, list_heights, list_notes)
You can see what files are in your work space using the ls function
ls()
## [1] "height" "heights_df" "heights_tib" "list_heights" "list_names"
## [6] "list_notes" "name"
And you can remove objects with the rm function.
rm(heights_df)
ls()
## [1] "height" "heights_tib" "list_heights" "list_names" "list_notes"
## [6] "name"
You have been working with the code here in your own R console. This is the html output of an Rmd file. You can see how this lab looks as the .Rmd format by clicking the link here(https://utchattanooga.instructure.com/courses/49070/files/9889244?wrap=1).
For this portion of the lab you will create your own Rmd document in Posit and document your work. Follow the lab submission guidelines above. You can also find a helpful R Markdown “cheat-sheet” at https://rstudio.github.io/cheatsheets/rmarkdown.pdf. When you are ready to save and publish your work, click the Knit icon at the top left of the tab. Then in the File tab check the box next to your .Rmd file and select the ‘More’ drop down at the top. Select ‘Export’ to download your file. Do the same thing for your html file.
Today we will investigate the March 15-17, 1973 flood in Chattanooga.
Visit the NOAA Climate DAta Onlien Search https://www.ncdc.noaa.gov/cdo-web/search.
To search for the data select ‘Precipitaion Hourly’ from the drop down. Then for date range select the week of the flood 03/11/73- 03/18/73. Then select ‘Cities’ in search type and Search for ‘Chattanooga’.
Once the search results appear, click on add to cart in the icon box on the left hand side menu. Then you will click on your cart in the upper right hand corner and ‘check out’ your data. Select the csv option for file type. The data link will be sent to the email you enter at the check out. This is a small dataset, it won’t take long for you to get an email telling you the dataset is ready. Download the data by clicking the link in the email.
Each data subset is downloaded with a unique order number. The order number in our example dataset is 4198967. If you are using a dataset you’ve downloaded yourself, make sure to substitute in your own order number in the code below.
First lets import the data and rename the file in our local environment. In the Files tab on the lower right side of the window pane click ‘Upload’. Then navigate to your file and select it to upload. It should now appear in the list on the lower right. Next in the ‘Environment’ tab on the upper right click ‘Import Dataset’ and select ‘From text base’ and then select the file. When you see the import preview select import. Notice that in the Console below the code for import has printed. You can add this code into your Rmd file to import automatically so your code will work everytime without the manual input
`4198967` <- read.csv("/cloud/project/4198967.csv")
We can also change the file name to make it easier to work with.
precip_chat<- `4198967`
You can view your newly imported dataset by clicking the table icon next to it in the Environment tab. Or by typing the command below. Which will show you the first six lines. This also shows the data types and header names.
head(precip_chat)
## STATION STATION_NAME DATE HPCP
## 1 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 07:00 0.01
## 2 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 09:00 0.08
## 3 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 10:00 0.20
## 4 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 11:00 0.71
## 5 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 12:00 0.28
## 6 COOP:401656 CHATTANOOGA AIRPORT TN US 19730311 13:00 0.10
You can always check a file’s structure and dat types within by using the str() function.
str(precip_chat)
## 'data.frame': 41 obs. of 4 variables:
## $ STATION : chr "COOP:401656" "COOP:401656" "COOP:401656" "COOP:401656" ...
## $ STATION_NAME: chr "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" "CHATTANOOGA AIRPORT TN US" ...
## $ DATE : chr "19730311 07:00" "19730311 09:00" "19730311 10:00" "19730311 11:00" ...
## $ HPCP : num 0.01 0.08 0.2 0.71 0.28 0.1 0.03 0.37 0.01 0.09 ...
Answer the following questions about the dataset:
#your code here
Notice that the DATE column is listed as a data class character. We will need to change this to a date class ‘POSIXct’ in order to work with the time series.
# convert to date/time and retain as a new field
precip_chat$DateTime <- as.POSIXct(precip_chat$DATE,
format="%Y%m%d %H:%M")
# date in the format: YearMonthDay Hour:Minute
# double check structure
str(precip_chat$DateTime)
## POSIXct[1:41], format: "1973-03-11 07:00:00" "1973-03-11 09:00:00" "1973-03-11 10:00:00" ...
We also need to check the dataset for missing values. The column HPCP is the total precipitation given in inches (since we selected Standard for the units), recorded for the hour ending at the time specified by DATE. Importantly, the metadata (see below) notes that the value 999.99 indicates missing data. Also important, hours with no precipitation are not recorded. To clean the data set first lets see if there are any missing data.
hist(precip_chat$HPCP)
The range of values in our histogram above indicates there is no missing
data in our data set. If there were we could asign the values an ‘NA’ to
correct for the missing values when plotting.
# assing NoData values to NA
precip_chat$HPCP[precip_chat$HPCP==999.99] <- NA
To check how many values are now NA we use the sum function.
sum(is.na(precip_chat))
## [1] 0
Now that the dataset is clean and the date is in the correct format lets plot the data. To do this we will use the ggplot2 package.
install.packages('ggplot2') #install package
## Installing package into '/cloud/lib/x86_64-pc-linux-gnu-library/4.5'
## (as 'lib' is unspecified)
library(ggplot2) # load package to environment
#plot the data
precPlot_hourly <- ggplot(data=precip_chat, # the data frame
aes(DateTime, HPCP)) + # the variables of interest
geom_bar(stat="identity") + # create a bar graph
xlab("Date") + ylab("Precipitation (Inches)") + # label the x & y axes
ggtitle("Hourly Precipitation - Chattanooga Station\n 1973 Flood") # add a title
precPlot_hourly
You can modify your plot in various ways. See this cheat sheet (https://posit.co/wp-content/uploads/2022/10/data-visualization-1.pdf)for
tips on using ggplot2. For example we can make the plot blue by adding
the color = command to the geom_car line. There are many colors in R,
see this cheat sheet for examples (https://www.nceas.ucsb.edu/sites/default/files/2020-04/colorPaletteCheatsheet.pdf).
When submitting your lab, change the color in the plot below to
something other than ‘blue’.
#plot the data
precPlot_hourly <- ggplot(data=precip_chat, # the data frame
aes(DateTime, HPCP)) + # the variables of interest
geom_bar(stat="identity", color ="blue") + # create a bar graph
xlab("Date") + ylab("Precipitation (Inches)") + # label the x & y axes
ggtitle("Hourly Precipitation - Chattanooga Station\n 1973 Flood") # add a title
precPlot_hourly
We can also zoom into a specific part of the dataset by filtering before plotting.
precPlot_floodday <- precip_chat %>%
filter(DateTime >"1973-03-16" & DateTime< "1973-03-17") %>%
ggplot( aes(DateTime, HPCP)) + # the variables of interest
geom_bar(stat="identity", fill ="chartreuse") + # create a bar graph
xlab("Date") + ylab("Precipitation (Inches)") + # label the x & y axes
ggtitle("Hourly Precipitation - Chattanooga Station\n 1973") # add a title
precPlot_floodday
The plot above shows that the highest hourly rainfall occured on the
morning of March 16th. We can find out the exact amount by using the
summary function.
summary(precip_chat)
## STATION STATION_NAME DATE HPCP
## Length:41 Length:41 Length:41 Min. :0.0100
## Class :character Class :character Class :character 1st Qu.:0.0700
## Mode :character Mode :character Mode :character Median :0.1100
## Mean :0.2173
## 3rd Qu.:0.2800
## Max. :1.1700
## DateTime
## Min. :1973-03-11 07:00:00
## 1st Qu.:1973-03-15 13:00:00
## Median :1973-03-15 23:00:00
## Mean :1973-03-15 07:54:08
## 3rd Qu.:1973-03-16 10:00:00
## Max. :1973-03-17 00:00:00
Answer the following questions about the dataset:
#your code here
# your code here
# precip_chat$newprecipinmm<- precip_chat$oldprecipdataininches * conversion factor
# ggplot(data = ...) + geom_point(aes(x = ... , y = ... , color = ... ))
In this section of the lab you will explore data of your choosing.Go back to the NOAA website (https://www.ncdc.noaa.gov/cdo-web/search). And search for a data set of interest. Explore the dataset as you did above and provide your code below. BE sure to include a plot fo the data.
Note that this section of the lab is important and should not be given only a cursory work through. An important learning goal for this term is for you to develop your own independent research skill. Thus, all of our labs this term will have a large, independent component where you are expected to apply what you’ve learned to ask novel and interesting questions and furthermore to try and go beyond what you’ve learned in class.
When trying to get started with something new, remember the copy/paste/tweak approach. This approach can help give you a good starting point for your work.
#your code here