[Skip Global Navigation]

Training

Table of Contents — Advanced Techniques: Regression

Introduction to Regression

A Simple Regression Analysis
Fitting Lines to Data
Goodness of Fit
How the Line is Fit
Residuals and Influential Points
What Does Linear Model Mean?
Assumptions of the General Linear Model

Examining the Data

Univariate Exploration — Independent Variables
Measures of Central Tendency
Variability Measures
Shape of Distribution
Extreme Points
Plots
Univariate Exploration — Dependent Variable
Normal Probability Plots
Should Solid Waste Be Normal
Box and Whisker Plot
Relations Between Variables
Relations Between the Independent Variables

Simple Regression: Fit and Assumptions

Running Simple Regression
Simple Regression Output
Information about Residuals
Assumptions of the Analysis

Multiple Regression: Fit and Assumptions

Running Multiple Regression
Descriptive Statistics
Multiple Regression Output
Regression Coefficients
Residual Analysis
Residual Plots
Scatterplot
Partial Regression Plots
Summary of the Analysis
The Need for a Substantive Model of Causation

Stepwise Regression

Methods of Selection
Evaluating Fit
Running Stepwise Regression
Stepwise Output
Should We Use a Four Variable Equation?
Statistical Significance and Practical Importance
Overfitting
More is Not Better: Overfitting and a Selection Strategy
The Next Step

Influential Points and Multicollinearity

Influential Points
Multicollinearity
Why is Multicollinearity a Problem?
How Do You Detect Multicollinearity?
Some Signs and Measures of Multicollinearity
What to Do about Multicollinearity
Requesting the Diagnostics Regression Output
Tolerance and Variance Inflation Factor (VIF)
Further Collinearity Diagnostics
Is Multicollinearity a Problem Here?
Influence Measures
What to Do about Unusual Points
Some Plots of Influence Measures
Using Explore to Locate Extremes
Effects of Large Samples
What If a Cluster of Points is Unusual?

Logistic Regression

Why Not Run Standard Regression?
The Logistic Equation
The Elements of Logistic Regression
Assumptions of Logistic Regression
A First Example of Logistic Regression
Interpreting Logistic Regression Coefficients
Making Predictions
The Accuracy of Prediction
Estimated Probabilities
Checking Classifications
Residual Analysis
Stepwise Logistic Regression
ROC Curves

Dummy Variables

Dummy Variable Coding
Are Dummy Variables Interval in Scale?
A Simple Example with Gender
Error Distribution
Using variables with More Than Two Categories
Dummy Variables and Missing Data
Regression with Race as a Predictor
Choosing a Reference Category
Using Two Categorical Variables
Assessing the Partial Effect of a Categorical Variable
Regression with Interval and Dummy Variables
Graphing the Effect of Dummy Variables
Appendix: Other Dummy Variable Coding Schemes

Interactions and Polynomial Regression

Defining Interactions
Interactions of Dummy Variables
Testing for an Interaction
Further Notes on Interpretation
Adding a Continuous Variable
Graphing Interactions
Interactions Between Categorical and Continuous Variables
Centering Interval Variables
Standardized Regression Coefficients
Adding Additional Variables
Curvilinear Regression
Further Advice on Polynomial Models

Multinomial Logistic Regression

Binary and Multinomial Logistic Regression
The Logit Model
Baseline Logit Model
Running the Multinomial Logistic Regression Analysis
Viewing the Results
Obtaining the Third Pairwise Comparison
Adding an Interval-Scale Predictor (Covariate)
Model with Degree Substituting for Education
Predicted Probabilities
Classification Table

Nonlinear Regression

What Does Nonlinear Mean?
Assumptions of Nonlinear Regression
An Example: Oxygen Concentration Over Time
Setting Up a Nonlinear Regression
Starting Values
Options and Loss Function
Results
Predicted Values

Resources