Nonnegative coefficient estimation
Estimating additive contributions such as mixture proportions or component intensities.
SciPy · Function
optimize.nnls
Solves a least-squares fit with the constraint that every element of the solution is nonnegative. It returns the fitted vector and the residual 2-norm.
How it works
The function solves a convex least-squares problem subject to every element of x being nonnegative. It uses an active-set method to satisfy the KKT conditions, and returns both the solution vector and the 2-norm of its residual.
Syntax
scipy.optimize.nnls(A, b, *, maxiter=None)Parameters
| Name | Type | Description |
|---|---|---|
A | (m, n) ndarray | Coefficient array with shape (m, n). |
b | (m,) ndarray, float | Right-hand-side vector with shape (m,), containing floats. |
maxiter | int, optional | Optional maximum number of iterations; defaults to 3 * n. |
Returns
Returns x, the solution vector, and rnorm, the 2-norm of the residual || Ax-b ||_2.
Key points
- Returns coefficients minimizing squared residual error while constraining every coefficient to be nonnegative.
- The task is a convex optimization problem, so the feasible least-squares solution is globally optimal.
- b supplies the observations or target values that the matrix-product fit should approximate.
- The solver uses an active-set strategy and KKT conditions to manage the nonnegative constraints.
Common mistakes
- If maxiter is omitted, SciPy allows three times the number of columns in A as the iteration limit.
- rnorm measures the remaining fit error, rather than the squared objective value.
Example: Nonnegative least-squares fit
import numpy as np
from scipy.optimize import nnls
A = np.array([[1.0, 0.0], [1.0, 0.0], [0.0, 1.0]])
b = np.array([2.0, 1.0, 1.0])
x, rnorm = nnls(A, b)
print(x) # [1.5 1. ]
print(rnorm) # residual 2-normFinds the nonnegative coefficient vector that best fits b in least-squares sense, then returns the residual norm.
When to use it
Physically constrained regression
Fitting measured observations when negative predictions or weights are not meaningful.
Residual analysis
Obtaining a constrained baseline fit before comparing residual error.
Interview questions
QWhat problem does scipy.optimize.nnls solve?
nnls minimizes the Euclidean residual subject to x >= 0. The inputs are an m-by-n coefficient matrix and an m-element right-hand-side vector; the result contains the feasible solution and residual norm.
Follow-up: Why can this be useful instead of ordinary least squares? The nonnegativity constraint prevents negative coefficients when the modeled quantities cannot physically be negative.
QWhat are the parameters and their default behavior?
A is the coefficient array with shape (m, n), and b is the right-hand-side vector with shape (m,). maxiter optionally limits the number of iterations and defaults to 3 * n.
Follow-up: How would increasing n affect the default iteration budget? It increases the default limit linearly because the limit is defined as three times the column count.
QHow does nnls enforce the nonnegativity constraint?
The implementation uses an active-set method and solves the KKT conditions for the nonnegative least-squares problem. This handles the inequality constraints by determining which variables are active at zero.
Follow-up: What does the returned rnorm represent? It is the 2-norm of Ax - b, so it quantifies the residual magnitude in the original observation space.
Related
scipy.optimize.lsq_linearscipy.optimize.least_squaresscipy.optimize.minimize