polynomial regression torch

Full AI 2025

3 min read 19-02-2025

Polynomial regression is a form of regression analysis in which the relationship between the independent variable x and the dependent variable y is modeled as an nth degree polynomial in x. This allows us to model non-linear relationships between variables, something simple linear regression cannot handle. This guide will walk you through implementing polynomial regression using PyTorch, a powerful deep learning framework.

Understanding Polynomial Regression

Unlike linear regression, which assumes a linear relationship (y = mx + c), polynomial regression models a curved relationship. It fits a polynomial curve to the data points, enabling it to capture more complex patterns. The degree of the polynomial determines the complexity of the curve; a higher degree allows for more complex curves but also increases the risk of overfitting.

Key Advantages:

Handles Non-linear Relationships: Effectively models data where a linear relationship isn't sufficient.
Flexibility: The degree of the polynomial can be adjusted to fit the data's complexity.

Key Disadvantages:

Overfitting: Higher-degree polynomials can overfit the training data, performing poorly on unseen data.
Interpretability: Higher-degree polynomials can be difficult to interpret.

Implementing Polynomial Regression with PyTorch

We'll use a simple example to illustrate the process. Our goal is to fit a polynomial curve to some sample data.

1. Data Preparation

First, we need to generate some sample data. This example uses a sinusoidal function with added noise to create a non-linear relationship:

import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt

# Generate sample data
np.random.seed(0)
X = np.linspace(-1, 1, 100)
y = np.sin(np.pi * X) + np.random.normal(0, 0.1, 100)

# Convert to PyTorch tensors
X = torch.tensor(X, dtype=torch.float32).reshape(-1, 1)
y = torch.tensor(y, dtype=torch.float32).reshape(-1, 1)

This code generates 100 data points and adds some noise to simulate real-world data.

2. Creating the Polynomial Features

Polynomial regression requires transforming the input features. We use torch.polyfit to compute the coefficients of a polynomial that best fits the data. Alternatively, we can manually create polynomial features:

# Manually creating polynomial features (degree 3)
X_poly = torch.cat([X**i for i in range(1, 4)], dim=1) # degree 3 polynomial features

3. Defining the Model

A simple linear model can now be used since we’ve already transformed our input features:

# Define the model
model = nn.Linear(3, 1) # 3 input features (polynomial terms), 1 output

4. Training the Model

Next, we define the loss function (Mean Squared Error) and the optimizer (Stochastic Gradient Descent):

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)

# Training loop
epochs = 1000
for epoch in range(epochs):
    # Forward pass
    y_pred = model(X_poly)
    loss = criterion(y_pred, y)

    # Backward pass and optimization
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch+1) % 100 == 0:
        print(f'Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}')

This loop iteratively updates the model's weights to minimize the loss.

5. Making Predictions and Visualizing Results

Finally, we can make predictions on the training data and visualize the results:

# Make predictions
y_pred = model(X_poly).detach().numpy()

# Plot the results
plt.figure(figsize=(10, 6))
plt.scatter(X.numpy(), y.numpy(), label='Data')
plt.plot(X.numpy(), y_pred, color='red', label='Polynomial Regression (degree 3)')
plt.xlabel('X')
plt.ylabel('y')
plt.title('Polynomial Regression with PyTorch')
plt.legend()
plt.show()

Choosing the Degree of the Polynomial

The choice of polynomial degree is crucial. A low degree might underfit, while a high degree might overfit. Techniques like cross-validation can help determine the optimal degree.

Further Considerations

Regularization: Techniques like L1 or L2 regularization can help prevent overfitting.
Feature Scaling: Scaling the input features can improve model performance.
Other Optimizers: Experiment with different optimizers like Adam or RMSprop.

This comprehensive guide demonstrates how to implement polynomial regression using PyTorch. By understanding the concepts and following the steps, you can build and train your own polynomial regression models for various applications. Remember to always carefully consider the degree of your polynomial to avoid overfitting and ensure the model generalizes well to unseen data.