What is a residual plot?

A residual plot is a graphical method to check how well a model’s predictions match actual data. For example, if you're predicting sales based on price, the residuals are the differences between the predicted and actual sales. In a residual plot, you would plot those differences.

If the residuals are scattered randomly around zero, the model is doing well. But if there’s a visible pattern, the model might be missing something, like a trend or relationship.

Residual plots are key to understanding how well your regression model works. They offer insights beyond just numbers, providing a visual way to assess whether the model captures the underlying patterns in the data.

By examining residual plots, analysts can determine if their model assumptions hold true and whether the predictions are reliable. This helps ensure the model can make accurate predictions, rather than just fitting the data on the surface.

Assessing model accuracy

A major benefit of using residual plots is to evaluate the accuracy of your regression model.

A regression model describes the relationship between an independent variable (like price) and a dependent variable (like sales). Residual plots help identify how well the model fits this relationship.

If you notice patterns in the residuals, it’s a sign the model may be missing key information or making inaccurate predictions, suggesting there’s room for improvement.

Understanding the relationship between observations, predictions, and residuals

Residual plots also help clarify the relationship between observed data, model predictions, and residuals.

Observations are the actual data points we collect, while predictions are the values the model expects. Residuals measure the difference between these two—the errors in the model's predictions.

A good model should have residuals that are small and randomly scattered. If they follow a pattern, the model’s predictions are systematically off, indicating a potential issue with the model.

If you're unfamiliar with residual plots, think of them as a simple graph with a horizontal and vertical axis. Each scattered point on the plot represents the difference between the actual data and the model’s prediction.

By looking at how these points are distributed, you can spot key patterns—such as linearity, non-linearity, or the presence of outliers—that reveal how well the model is performing.

Significance in diagnosing model adequacy

Ideally, residuals will be scattered randomly around zero. This suggests the model’s predictions are unbiased and any errors are random.

If there’s a clear pattern, like a curve or an upward trend, the model may be overestimating or underestimating its predictions.

Residual plots come in many forms, each serving a specific purpose.

Normal Q-Q residual plot

The Normal Quantile-Quantile residual plot compares the distribution of residuals to a normal distribution.

If the residuals are normally distributed, they will align closely with the diagonal on the plot. Deviations from the line suggest an abnormality and could indicate problems with the model's assumptions.

Specialized residual plots

In addition to standard residual plots, several specialized plots are used for specific diagnostics in regression analysis.

For instance, a residuals vs. independent variable plot helps identify relationships that the current model may not be capturing, revealing potential non-linearity or overlooked variables.

A standardized residual plot, on the other hand, allows you to pinpoint large errors and detect outliers by displaying the residuals in relation to their standardized values.

A residuals vs. leverage plot is useful for identifying influential data points that could disproportionately affect the model’s results, guiding you in assessing the stability and reliability of your regression analysis.

You can gain a deeper understanding of model adequacy by analyzing example residual plots. 

A residual plot that shows a funnel shape might indicate the variance of errors changing with predicted values. A curved pattern suggests a non-linear relationship the model did not account for.

Diagnosing model adequacy based on residual plots

Example residual plots allow you to learn how to diagnose potential problems with a regression model. If the residuals show a clear pattern, it typically indicates the model is not adequately capturing the relationship between the variables.

Implications of imperfect models

Despite your best efforts, models can be imperfect. Example residual plots can show you what imperfections look like, such as:

• incorrect functional forms

• missing variables

• outliers

Recognizing these issues allows you to refine the model for reliability and accuracy.

If residual plots reveal issues with a regression model, you can use several methods to improve the model's accuracy and validity.

Transforming variables

One way to address issues like non-linearity is to transform the variables. For example, applying a square root transformation to a variable can linearize relationships and improve the model fit.

Handling missing variables

If a residual plot suggests important factors are missing from the model, adding these variables can enhance the model's accuracy. This step involves identifying and including relevant predictors that may have been overlooked.

Adding new variables

Introducing new variables can sometimes explain the patterns observed in the residuals. By incorporating additional predictors, you can capture more of the variation in the dependent variable, leading to better model performance.

Considering omitted and interaction variables

Sometimes, the model may be missing interaction effects, in which two variables together influence the outcome in a way that isn't captured by their individual effects. Adding these interaction terms can explain patterns seen in residual plots.

A scatter plot illustrates the relationship between two variables, typically showing how a dependent variable changes in response to an independent variable. A residual plot focuses on the errors in a regression model, plotting residuals against predicted values to evaluate model performance. 

While both plots are useful in regression analysis, residual plots specifically assess prediction quality and identify potential issues.