Main
- in R:
y ~ 1 + year, in eq: \(y = w_1x_1 + w_2x_2\)
- in R:
y ~ 1 + sex, in eq: \(y = w_1x_1 + w_2x_2\)
- in R:
y ~ 1 + year, in eq: \(y = w_1x_1 + w_2x_2 + w_3x_3\)
Week 07: Model Specification
Data Science for Studying Language and the Mind
2024-10-08
Announcements
- You did great on the exam!
- You can replace your lowest exam score with the optional final
- The final exam is cumulative: another opportunity to show mastery of the material.
Is there a relationship?
If there was no relationship, we’d say there are independent: knowing the value of one provides no information about about the other. But that’s not the case here.
Yes, a linear relationship
In a linear relationship, when one variable goes up the other goes up (positive); or when one goes up the other goes down (negative).
Quantify with correlation
One way to quantify linear relationships is with correlation (\(r\)). Correlation expresses the linear relationship as a range from -1 (perfectly negative) to 1 (perfectly positive).
💪 In-class Exercise 7.1
Take a few minutes to try this yourself!
Use the infer way to visualize the sampling distribution and shade the confidence interval we just computed. Change the x-axis label to stat (correlation) as pictured below.
Hypothesis testing our correlation
How do we test whether the correlation we observed is significantly different from zero? Hypothesis test!
Step 3: Decide whether to reject the null!
Interpret our p-value. Should we reject the null hypothesis?
Correlation is model building
Correlation is a simple case of model building, in which we use one value (\(x\)) to predict another (\(y\)).
Correlation is model building
Even more specifically — formally, the model specification — we are fitting the linear model \(y = ax+b\), where \(a\) and \(b\) are free parameters.
- Model specification: \(y = ax + b\)
- Estimate free parameters: \(a\) and \(b\)
- Fitted model: \(y = 0.39x + 2.02\)
Correlation is the slope of the model
Correlation is the slope of the line that best predicts \(y\) from \(x\) (after z-scoring)
💪 In-class Exercise 7.2
Take a few minutes to try this yourself!
Ask ChatGPT what type of model it is made with?
Linear models
- Linear models are models in which the output (y) is a weighted sum of the inputs
- Easy to understand and fit
- \(y=\sum_{i=1}^{n}w_ix_i\)
- \(y = ax + b\) is this!
Nonlinear models
Output (y) cannot be expressed as a weighted sum of inputs(\(y=\sum_{i=1}^{n}w_ix_i\) ); pattern is better captured by more complex functions. (But often we can linearize them!)
Specify our first model
To illustrate how this simple equation scales up to complex models, let’s start with a simple case (“toy”, “tractable”).
# A tibble: 2 × 2
x y
<dbl> <dbl>
1 1 3
2 2 5
Enter the matrix \(y = Xβ + ε\)
When we have more equations than unknowns we cannot solve the system directly (we have an overdetermined system), but we can find a soulution with linear algebra.
\[\begin{aligned}
\begin{bmatrix}
3 \\
5
\end{bmatrix}
=
\begin{bmatrix}
1 & 1 \\
1 & 2
\end{bmatrix}
\begin{bmatrix}
w_1 \\
w_0
\end{bmatrix}
\end{aligned}\]
Response variable \(y\)
What is the thing you are trying to understand?
Explanatory variable(s) \(x_i\)
What could explain the variation in your response variable?
Intercept
- in R:
y ~ 1, in eq: \(y=w_1x_1\)
Interaction
- in R:
y ~ 1 + year + gender + year:gender- or the short way:
y ~ 1 + year * gender
- or the short way:
- in eq: \(y = w_1x_1 + w_2x_2 + w_3x_3 + w_4x_4\) where \(x_4\) is \(x_2x_3\)
Transformation
- in R:
y ~ 1 + year * sex + I(year^2) - in eq: \(y = w_1x_1 + w_2x_2 + w_3x_3 + w_4x_4\) + \(w_5x_5\)
- where \(x_4\) is \(x_2x_3\) and \(x_5\) is \(x_2^2\)
