library(tidyverse)
library(rvest)

URL <- "https://history.house.gov/Institution/Party-Divisions/Party-Divisions/"
webpage <- read_html(URL)
cong <- as_tibble(html_table(webpage)[[1]])
onms <- colnames(cong)
newnms <- c("congress", "seats", "D", "R", "other", "delres")
cong <- cong %>% 
  slice((which(grepl("^Republican", `Anti-Administration`))[1] + 1):nrow(.)) %>%
  filter(`Congress (Years)` != colnames(cong)[1]) %>% 
  mutate(year = gsub("(*.*-)|(*.*–)|)|)2", "", `Congress (Years)`)) %>%
  mutate(year = as.numeric(year) - 2) %>% 
  rename_at(vars(onms), ~newnms) %>% select(-other, -delres) %>%
  mutate(seats = substr(seats, 1, 3)) %>% 
  mutate_at(.vars = vars(seats, D, R), as.numeric) %>% 
  mutate(swing = (D - R) / (D + R)) 
cong %>%  
  ggplot() + geom_line(aes(year, swing)) + 
  theme(axis.text.x = element_text(angle = 90, vjust = 0.5)) + 
  geom_hline(yintercept = 0, lty = 2, col = "red") + 
  xlab("") + ylab("(D - R) / (D + R)") + 
  ggtitle("Normalized Party Control Index for US House") + theme_linedraw() + 
  scale_x_continuous(breaks = seq(1859, 2019, 10), expand = c(0, 2))

Today

  • Bring Your Own Data (solved)
  • Continue on manipulating, plotting, analyzing data

BYOD

  • Solution to excel read found!

Reading, reshaping, analyzing, plotting

dat <- readr::read_csv(
  system.file("extdata/FAOSTAT_maize.csv", package = "geospaar")
)

Selecting, renaming, reshaping, mutating

  • Question: how can we describe the shape of this dataset?
  • Follow along:
    • select columns of interest
    • rename to make more tractable
    • reshape: pivot values into different columns, and some light renaming
    • mutate: make a yield column
library(tidyr)
library(dplyr)
dat_wide <- dat %>%
  select(Area, Item, Year, Element, Value) %>%
  rename(country = Area, crop = Item, year = Year, quantity = Element,
         value = Value) %>%
  pivot_wider(names_from = quantity, values_from = value) %>%
  rename(ha = `Area harvested`, prod = Production) %>%
  mutate(yield = prod / ha)

Joining

  • Follow along
    • separate datasets by quantity (ha, prod, yield)
    • join by country, crop, year
      • test different kinds of joins (drop some rows to see effects)
      • arrange joins by country, year, etc

SAC

  • Base
sapply(sort(unique(dat_wide$country)), function(x) {
  colMeans(dat_wide[dat_wide$country == x, c("ha", "prod", "yield")])
})
  • Follow along, dplyr

Plotting

  • Follow along, plotting
    • Maize yield by year for South Africa
      • Base
      • ggplot
    • Graphical SAC: maize yield by country by year
      • Base
      • ggplot
    • Graphical SAC: different quantities by country year, each quantity on its own panel
      • Back to long
      • ggplot
      • How do you reconcile scales?

Regression

  • Follow along
    • Calculate and evaluate maize yield trend across time for Zambia
    • Plot the regression, base and ggplot