Regression module

Regression module contains bare-bones version of the PCM toolbox that can be used to tune ridge/Tikhonov coefficients in the context of tranditional regression models. No assumption are made about independent data partitions.

class regression.RidgeDiag(components, theta0=None, fit_intercept=True, noise_model=<PcmPy.model.IndependentNoise object>)

Class for Linear Regression with Tikhonov (L2) regularization. The regularization matrix for this class is diagnonal, with groups of elements along the diagonal sharing the same Regularisation factor.

Constructor

Parameters:

components (1d-array like): Indicator to which column of design matrix belongs to which group
theta0 (1d np.array): Vector of of starting values for optimization
fit_intercept (Boolean): Should intercept be added to fixed effects (Dafault: true)
noise_model (pcm.model.NoiseModel): Model specifying the full-rank noise effects

fit(Z, Y, X=None)

Estimates the regression parameters, given a specific regularization :param Z: Design matrix for random effects NxQ :type Z: 2d-np.array :param Y: NxP Matrix of data :type Y: 2d-np.array :param X: Fixed effects design matrix - will be accounted for by ReML :type X: np.array

Returns:: self – Model with fitted parameters

optimize_regularization(Z, Y, X=None, optim_param={}, like_fcn='auto')

Optimizes the hyper parameters (regularisation) of the regression mode :param Z: Design matrix for random effects NxQ :type Z: 2d-np.array :param Y: NxP Matrix of data :type Y: 2d-np.array :param X: Fixed effects design matrix - will be accounted for by ReML :type X: np.array :param optim_parameters: parameters for the optimization routine :type optim_parameters: dictionary of parameters

Returns:: self – Model with fitted parameters

predict(Z, X=None)

Predicts new data based on a fitted model :param Z: Design matrix for random effects NxQ :type Z: 2d-np.array :param Y: NxP Matrix of data :type Y: 2d-np.array :param X: Fixed effects design matrix - will be accounted for by ReML :type X: np.array

Returns:: self – Model with fitted parameters

regression.compute_iVr(Z, G, iS, X=None)

Fast inverse of V matrix using the matrix inversion lemma

Parameters:

Z (2d-np.array) – Design matrix for random effects NxQ
G (1d or 2d-np.array) – Q x Q Matrix: variance of random effect
iS (scalar or NxN matrix) – Inverse variance of noise matrix
X (2d-np.array) – Design matrix for random effects

Returns:

iV (2d-np.array) – inv(Z*G*Z’ + S);
iVr (2d-np.array) – iV - iV * X inv(X’ * iV *X) * X’ *iV
ldet (scalar) – log(det(iV))

regression.likelihood_diagYTY_ZZT(theta, Z, Y, comp, X=None, Noise=<PcmPy.model.IndependentNoise object>, return_deriv=0)

Negative Log-Likelihood of the data and derivative in respect to the parameters. This function is faster when N>>P.

Parameters:

theta (np.array) – Vector of (log-)model parameters: These include model, signal and noise parameters
Z (2d-np.array) – Design matrix for random effects NxQ
Y (2d-np.array) – NxP Matrix of data
comp (1d-np.array or list) – Q-length: Indicates for each column of Z, which theta will be used for the weighting
X (np.array) – Fixed effects design matrix - will be accounted for by ReML
Noise (pcm.Noisemodel) – Pcm-noise mode to model block-effects (default: Indentity)
return_deriv (int) – 0: Only return negative loglikelihood 1: Return first derivative 2: Return first and second derivative (default)

Returns:

negloglike – Negative log-likelihood of the data under a model
dLdtheta (1d-np.array) – First derivative of negloglike in respect to the parameters
ddLdtheta2 (2d-np.array) – Second derivative of negloglike in respect to the parameters

regression.likelihood_diagYYT_ZTZ(theta, Z, YY, num_var, comp, X=None, Noise=<PcmPy.model.IndependentNoise object>, return_deriv=0)

Negative Log-Likelihood of the data and derivative in respect to the parameters. This function is faster when P>N

Parameters:

theta (np.array) – Vector of (log-)model parameters: These include model, signal and noise parameters
Z (2d-np.array) – Design matrix for random effects NxQ
YY (2d-np.array) – NxN Matrix: Outer product of the data
num_var (int) – Number of variables in data set (columns of Y)
comp (1d-np.array or list) – Q-length: Indicates for each column of Z, which theta will be used for the weighting
X (np.array) – Fixed effects design matrix - will be accounted for by ReML
Noise (pcm.Noisemodel) – Pcm-noise mode to model block-effects (default: Indentity)
return_deriv (int) – 0: Only return negative loglikelihood 1: Return first derivative 2: Return first and second derivative (default)

Returns:

negloglike – Negative log-likelihood of the data under a model
dLdtheta (1d-np.array) – First derivative of negloglike in respect to the parameters
ddLdtheta2 (2d-np.array) – Second derivative of negloglike in respect to the parameters

regression.likelihood_diagYYT_ZZT(theta, Z, YY, num_var, comp, X=None, Noise=<PcmPy.model.IndependentNoise object>, return_deriv=0)

Negative Log-Likelihood of the data and derivative in respect to the parameters. This function is faster when P>N

Parameters:

theta (np.array) – Vector of (log-)model parameters: These include model, signal and noise parameters
Z (2d-np.array) – Design matrix for random effects NxQ
YY (2d-np.array) – NxN Matrix: Outer product of the data
num_var (int) – Number of variables in data set (columns of Y)
comp (1d-np.array or list) – Q-length: Indicates for each column of Z, which theta will be used for the weighting
X (np.array) – Fixed effects design matrix - will be accounted for by ReML
Noise (pcm.Noisemodel) – Pcm-noise mode to model block-effects (default: Indentity)
return_deriv (int) – 0: Only return negative loglikelihood 1: Return first derivative 2: Return first and second derivative (default)

Returns:

negloglike – Negative log-likelihood of the data under a model
dLdtheta (1d-np.array) – First derivative of negloglike in respect to the parameters
ddLdtheta2 (2d-np.array) – Second derivative of negloglike in respect to the parameters