Linear Regression

A linear regression algorithm with optional L1 (LASSO), L2 (ridge) or L1L2 (elastic net) regularization.

Inputs

  • Data: input dataset
  • Preprocessor: preprocessing method(s)

Outputs

  • Learner: linear regression learning algorithm
  • Model: trained model
  • Coefficients: linear regression coefficients

The Linear Regression widget constructs a learner/predictor that learns a linear function from its input data. The model can identify the relationship between a predictor xi and the response variable y. Additionally, Lasso and Ridge regularization parameters can be specified. Lasso regression minimizes a penalized version of the least squares loss function with L1-norm penalty and Ridge regularization with L2-norm penalty.

Linear regression works only on regression tasks.

  1. The learner/predictor name
  2. Choose a model to train:
    • no regularization
    • a Ridge regularization (L2-norm penalty)
    • a Lasso bound (L1-norm penalty)
    • an Elastic net regularization
  3. Produce a report.
  4. Press Apply to commit changes. If Apply Automatically is ticked, changes are committed automatically.

Preprocessing

Linear Regression uses default preprocessing when no other preprocessors are given. It executes them in the following order:

  • removes instances with unknown target values
  • continuizes categorical variables (with one-hot-encoding)
  • removes empty columns
  • imputes missing values with mean values

To remove default preprocessing, connect an empty Preprocess widget to the learner.

Feature Scoring

Linear Regression can be used with Rank for feature scoring. See Learners as Scorers for an example.

Example

Below, is a simple workflow with housing dataset. We trained Linear Regression and Random Forest and evaluated their performance in Test & Score.