What are the variable types? Do they all correspond to what they really are? Which variables have most missing values?

skim(rent)

Variable type: character

Variable type: numeric

glimpse(rent)

## Rows: 200,796
## Columns: 17
## $ post_id     <chr> "pre2013_134138", "pre2013_135669", "pre2013_127127", "pre…
## $ date        <dbl> 20050111, 20050126, 20041017, 20120601, 20041021, 20060411…
## $ year        <dbl> 2005, 2005, 2004, 2012, 2004, 2006, 2007, 2017, 2009, 2006…
## $ nhood       <chr> "alameda", "alameda", "alameda", "alameda", "alameda", "al…
## $ city        <chr> "alameda", "alameda", "alameda", "alameda", "alameda", "al…
## $ county      <chr> "alameda", "alameda", "alameda", "alameda", "alameda", "al…
## $ price       <dbl> 1250, 1295, 1100, 1425, 890, 825, 1500, 2925, 450, 1395, 1…
## $ beds        <dbl> 2, 2, 2, 1, 1, 1, 1, 3, NA, 2, 2, 5, 4, 0, 4, 1, 3, 3, 1, …
## $ baths       <dbl> 2, NA, NA, NA, NA, NA, 1, NA, 1, NA, NA, NA, 3, NA, NA, NA…
## $ sqft        <dbl> NA, NA, NA, 735, NA, NA, NA, NA, NA, NA, NA, 2581, 1756, N…
## $ room_in_apt <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ address     <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
## $ lat         <dbl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 37.5, NA, …
## $ lon         <dbl> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
## $ title       <chr> "$1250 / 2br - 2BR/2BA   1145 ALAMEDA DE LAS PULGAS", "$12…
## $ descr       <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA…
## $ details     <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, "<p class=…

Answer:

• Variable types: chr & dbl
  
• Do they correspond: Date could be formatted as < date >
  
• Most missing variable: descr (197542)
  

Plot to show the top 20 cities in terms of % of classifieds between 2000-2018. (Includes calculation of number of listings by city, and then converted to %)

top20 <- rent %>% 
  count(city, sort=TRUE) %>% 
  mutate(proportion = n/sum(n)) %>% 
  slice_max(order_by = proportion, n=20) %>% 
  mutate(city = fct_reorder(city, proportion))
  
ggplot(data = top20, mapping = aes(x=proportion, y=city)) +
  geom_col() +
  labs(
    title = "San Francisco accounts for more than a quarter of all rental classifieds",
    subtitle = "% of Craigslist listings, 2000-2018",
    x = NULL,
    y = NULL,
    caption="Source: Pennigton, Kate (2018). Bay Area Craiglist Rental Housing Posts, 2000-2018"
  ) +
  scale_x_continuous(labels = scales::percent) +
  theme_light() +
  theme(panel.border = element_blank())+
  theme(plot.title = element_text(hjust = -0.35))+
  theme(plot.subtitle = element_text(hjust = -0.15))

Visual depiction of evolution of median prices in San Francisco for 0, 1, 2, and 3 bedrooms listings.

median_per_bed <- rent %>% 
  filter(beds <= 3, city == "san francisco") %>% 
  group_by(beds, year) %>% 
  summarize(median_price = median(price))

ggplot(median_per_bed, aes(x=year, y=median_price, color = factor(beds))) +
  geom_line() +
  facet_wrap(~beds, nrow = 1) +
  labs(
    title = "San Francisco rents have steadily been increasing",
    subtitle = "0 to 3-bed listings, 2000-2018",
    x=NULL,
    y=NULL,
    caption = "Source: Pennigton, Kate (2018). Bay Area Craiglist Rental Housing Posts, 2000-2018"
  ) +
  xlim(2003,2018) +
  theme_light() +
  theme(legend.position="none") +
  theme(plot.title = element_text(hjust = 0))+
  theme(plot.subtitle = element_text(hjust = 0)) +
  theme(strip.text.x = element_text(colour = "black")) +
  theme(panel.border = element_rect(color = "black", fill = NA, size = 0.5)) +
  theme(strip.background = element_rect(color = "black", size = 0.5))

Visualization of median rental prices for the top 12 cities in the Bay area.

cities_to_include <- rent %>% 
  group_by(city) %>% 
  summarize(number_ads = n()) %>%
  slice_max(order_by = number_ads, n=12)

cities_to_include <- cities_to_include$city

median_per_bed <- rent %>% 
  filter(beds == 1, city %in% cities_to_include) %>% 
  group_by(city, year) %>% 
  summarize(median_price = median(price)) 

ggplot(median_per_bed, aes(x=year, y=median_price, color = factor(city))) +
  geom_line() +
  facet_wrap(~city, nrow = 3) +
  labs(
    title = "Rental prices for 1-bedroom flats in the Bay Area",
    x=NULL,
    y=NULL,
    caption="Source: Pennigton, Kate (2018). Bay Area Craiglist Rental Housing Posts, 2000-2018"
  ) +
  theme_light() +
  theme(legend.position="none") +
  theme(plot.title = element_text(hjust = 0)) +
  theme(strip.text.x = element_text(colour = "black")) +
  theme(panel.border = element_rect(color = "black", fill = NA, size = 0.5)) +
  theme(strip.background = element_rect(color = "black", size = 0.5))

Inference

Looking at the graphs from the last exercise we can see that rental prices have increased since the year 2000. While one major cause could be inflation, it is also true that large tech companies have increased the attractiveness of the Bay Area making living there more expensive.
The figures also reveal that there has been a decline in the growth rate (or even negative growth) in the median rent after 2015. It could be due to the fact that more and more people are leaving California as the boom seems to lose momentum. A CNBC video outlines the reasons for this exodus of people.