Preprocessing module

Preprocessing helpers and functions to be applied to EEG data.

This module gathers covariance-based utilities and preprocessing transforms commonly used in EEG/MEG analysis:

Covariance estimators: covariance, covariances, covariances_extended (with wrappers around scikit-learn and NumPy covariance estimators in _check_est).
Filterbank helpers: create_filterbank, apply_filterbank, and get_power.
Whitener: PCA/ZCA-based data whitening (demean, covariance, rotation, transform/inverse).
WaveletTransform: complex Morlet wavelet decomposition of a multi-channel signal.
MultichanWienerFilter: artifact removal with a multi-channel Wiener filter.

Functions applying a filterbank or wavelets accept an n_jobs argument to parallelize the computation with joblib.

Classes

See also the Multiway CCA module page for the multiway CCA (hyperalignment) preprocessing class.

Directly in pyeeg.preprocess one can find also the following classes:

`MultichanWienerFilter`([lags, low_rank, thresh])	This class implements a multichannel Wiener Filter for artifact removal.
`Whitener`([axis, zca, bias])	A data whitener (via either PCA or ZCA).

WaveletTransform(*args, **kwargs)

Summary

`create_filterbank`(freqs, srate[, filtertype])	Creates a filter bank, by default of chebychev type 2 filters.
`apply_filterbank`(data, fbank[, filt_func, ...])	Applies a filterbank to a given multi-channel signal.
`get_power`(signals[, decibels, win, axis, n_jobs])	Compute the (log) power modulation of a signal by taking the smooth moving average of its square values.
`covariance`(X[, estimator])	Estimation of one covariance matrix on the whole dataset.
`covariances`(X[, estimator])	Estimation of covariance matrices from a list of "trials".
`covariances_extended`(X, P[, estimator])	Special form covariance matrix where data are appended with another set.

Listing of all classes and functions

Preprocessing helpers and functions to be applied to EEG data.

This module gathers covariance-based utilities and preprocessing transforms commonly used in EEG/MEG analysis:

Covariance estimators: covariance, covariances, covariances_extended (with wrappers around scikit-learn and NumPy covariance estimators in _check_est).
Filterbank helpers: create_filterbank, apply_filterbank, and get_power.
Whitener: PCA/ZCA-based data whitening (demean, covariance, rotation, transform/inverse).
WaveletTransform: complex Morlet wavelet decomposition of a multi-channel signal.
MultichanWienerFilter: artifact removal with a multi-channel Wiener filter.

Functions applying a filterbank or wavelets accept an n_jobs argument to parallelize the computation with joblib.

class pyeeg.preprocess.MultichanWienerFilter(lags=(0,), low_rank=False, thresh=None)

This class implements a multichannel Wiener Filter for artifact removal. The method is detailed in the reference paper A generic EEG artifact removal algorithm based on the multi-channel Wiener filter from Ben Somers et. al.

To correctly train the model, one must supply portions of contaminated data and clean data. This can be selected visually using the annotation tool from MNE for instance, or automatically by detecting above threshold values and considering this as bad portions. It is ok to have large windows around bad data segments, however the clean segments must be artifact free.

The model expects zero-mean data for both noisy and clean segments.

lags

Lags used for general model (NOT IMPLEMENTED YET)

Type:: list

low_rank

Whether to use low-rank approximation of covariance matrix for the artifactual data

Type:: bool

thresh

If int, this will correspond to the rank prior If float, it will be considered as the percent of variance to be kept

Type:: int or float

W_

Once fitted, contains the filter coefficients

Type:: ndarray

Example

TODO: Add code example

Example of result obtained (cleaning EOG artifact here):

fit(y_clean, y_artifact, cov_data=False)

Fit model to data.

Parameters:

y_clean (ndarray) – Clean segments
y_artifact (ndarray) – Artifact-contaminated segments
cov_data (bool) – Whether the input data are already covariance matrices estimate for each class

fit_transform(y_clean, y_artifact, x, cov_data=False)

Train the model on input and transform directly the data in x.

Parameters:

y_clean (ndarray) – Clean segments used for training.
y_artifact (ndarray) – Artifact-contaminated segments used for training.
x (ndarray) – EEG data to filter.
cov_data (bool) – Whether the input data are already covariance matrices estimated for each class (default False).

Returns:

out – Filtered EEG data (artifacts removed), same shape as x.

Return type:

ndarray

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

cov_data (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for cov_data parameter in fit.
y_artifact (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for y_artifact parameter in fit.
y_clean (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for y_clean parameter in fit.

Returns:

self – The updated object.

Return type:

object

transorm(x)

Filter the data to remove artifact learned by the model.

Parameters:: x (ndarray) – EEG data (samples, channels).
Returns:: out – Filtered EEG data (artifacts removed), same shape as x.
Return type:: ndarray

Notes

The method name transorm is a historical typo of transform but is retained as-is for backwards compatibility.

class pyeeg.preprocess.Whitener(axis=0, zca=False, bias=True)

A data whitener (via either PCA or ZCA).

Whitening linearly transforms the data so that its covariance becomes the identity matrix, decorrelating the channels and rescaling them to unit variance. Two standard whitening transforms are supported:

PCA whitening (zca=False): W = diag(1/sqrt(eigval)) @ eigvec.T.
ZCA whitening (zca=True): W = eigvec @ diag(1/sqrt(eigval)) @ eigvec.T, which additionally rotates the whitened data back to the original channel space (also called Mahalanobis or zero-phase whitening).

The transform is (X - mu) @ W.T; the mean mu is stored during fit() (via demean()) so that transform() and inverse() can be applied consistently to new data.

Parameters:

axis (int) – Axis along which to compute the mean and covariance. Default is 0 (samples first).
zca (bool) – If True, use ZCA whitening, otherwise PCA whitening (default False).
bias (bool) – If True, divide by n when estimating the covariance; if False, use the unbiased estimator (divide by n - 1). Default is True.

mu

Mean of the training data along axis, computed by demean().

Type:: ndarray or None

sigma

Covariance matrix of the (demeaned) training data, computed by cov().

Type:: ndarray or None

W

The whitening matrix such that transform(X) = (X - mu) @ W.T.

Type:: ndarray or None

scale

Diagonal matrix diag(1/sqrt(eigval)) of inverse sqrt eigenvalues.

Type:: ndarray or None

U

Eigenvectors of the covariance matrix.

Type:: ndarray or None

Examples

>>> import numpy as np
>>> from pyeeg.preprocess import Whitener
>>> rng = np.random.default_rng(0)
>>> M = np.array([[2., 0.5, 0.1], [0.5, 1., 0.3], [0.1, 0.3, 1.5]])
>>> X = rng.standard_normal((100, 3)) @ M
>>> wh = Whitener(axis=0, zca=True).fit(X)
>>> Z = wh.transform(X)
>>> np.allclose(np.cov(Z, rowvar=False), np.eye(3), atol=1e-1)
True

compute_rotation(C=None)

Compute the whitening rotation matrix from the (stored) covariance.

The eigenvalues/eigenvectors of the covariance matrix (either self.sigma or the provided C) are used to build the whitening matrix W. PCA whitening uses W = diag(1/sqrt(eigval)) @ eigvec.T and ZCA whitening uses W = eigvec @ diag(1/sqrt(eigval)) @ eigvec.T.

Parameters:: C (ndarray (nchannels, nchannels), optional) – Covariance matrix to diagonalize. If None, self.sigma (as computed by cov()) is used.
Returns:: Stores self.scale, self.U and self.W in place.
Return type:: None
Raises:: AssertionError – If self.sigma has not been computed yet and C is None.

cov(data, axis=None)

Estimate the covariance matrix of data and store it in self.sigma.

The covariance is computed as data.T @ data scaled by the number of samples (optionally debiased by one when self.bias is False).

Parameters:

data (ndarray) – Input data. Typically centered (see demean()).
axis (int, optional) – Axis along which samples run. If None, self.axis is used. The data are transposed so that samples run along axis 0.

Returns:

sigma – The estimated covariance matrix.

Return type:

ndarray (nchannels, nchannels)

demean(data, axis=None)

Subtract the mean of data along axis and store it in self.mu.

Parameters:

data (ndarray) – Input data.
axis (int, optional) – Axis along which to compute the mean. If None, self.axis is used.

Returns:

data_demeaned – data with the mean subtracted along axis.

Return type:

ndarray

fit(X, y=None, axis=None)

Fit the whitener on X: demean, estimate the covariance and compute the whitening rotation.

Parameters:

X (ndarray) – Training data.
y (ignored) – Present for scikit-learn API compatibility.
axis (int, optional) – Axis along which samples run. If None, self.axis is used.

Returns:

self – The fitted estimator.

Return type:

Whitener

fit_transform(X, y=None, axis=None)

Fit the whitener on X and return the whitened data.

Parameters:

X (ndarray) – Training data.
y (ignored) – Present for scikit-learn API compatibility.
axis (int, optional) – Axis along which samples run. If None, self.axis is used.

Returns:

X_white – Whitened data, of the same shape as X.

Return type:

ndarray

inverse(X, axis=None)

Invert the whitening transform, approximately recovering the original (unwhitened) data.

The inverse uses the pseudo-inverse of W.T and adds back the stored mean self.mu.

Parameters:

X (ndarray) – Whitened data.
axis (int, optional) – Unused, kept for API compatibility.

Returns:

X_orig – Data in the original channel space, of the same shape as X.

Return type:

ndarray

transform(X, y=None)

Whiten X using the fitted mean and rotation.

Parameters:

X (ndarray) – Data to whiten. Must have the same number of channels as the training data.
y (ignored) – Present for scikit-learn API compatibility.

Returns:

X_white – Whitened data, of the same shape as X, with approximately identity covariance.

Return type:

ndarray

pyeeg.preprocess.apply_filterbank(data, fbank, filt_func=<function lfilter>, n_jobs=-1, axis=-1)

Applies a filterbank to a given multi-channel signal.

Parameters:

data (ndarray (samples, nchannels))
fb (list) – list of (b,a) tuples, where b and a specify a digital filter

Returns:

y

Return type:

ndarray (nfilters, samples, nchannels)

pyeeg.preprocess.covariance(X, estimator='cov')

Estimation of one covariance matrix on the whole dataset. If X is of shape (trials, samples, channels) Will concatenate all trials together to compute a single covariance matrix across all of them.

Parameters:

X (ndarray (nsamples, nchannels) or (ntrials, nsamples, nchannels)) – Input data. If 3d, all trials are concatenated along the sample dimension before estimating the covariance.
estimator (str or callable) – One of the covariance estimators understood by _check_est() ('cov', 'scm', 'lwf', 'oas', 'mcd', 'corr') or a callable returning a covariance matrix.

Returns:

C – The estimated covariance matrix.

Return type:

ndarray (nchannels, nchannels)

pyeeg.preprocess.covariances(X, estimator='cov')

Estimation of covariance matrices from a list of “trials”.

Parameters:

X (array-like (ntrials, nsamples, nchannels) or list) – If list, each element can also have different number of samples, but the number of channels must be the same for all trials.
estimator (str) – One of covariance estimator from sklearn. See also _check_est().

Returns:

C – The list of covariance matrices for each trial.

Return type:

array-like (ntrials, nchannels, nchannels)

pyeeg.preprocess.covariances_extended(X, P, estimator='cov')

Special form covariance matrix where data are appended with another set. For instance, the data could be EEG data and the other set could be a set of idealised response (e.g. a clean ERP).

Parameters:

X (ndarray (ntrials, nsamples, nchannels) or (nsamples, nchannels)) – Input data. If 2d, a dummy trial dimension is added (a single trial).
P (ndarray (nsamples, nchannels_other)) – The other set appended to the data, e.g. an idealised response from the average across trials.
estimator (str or callable) – One of the covariance estimators understood by _check_est() ('cov', 'scm', 'lwf', 'oas', 'mcd', 'corr') or a callable returning a covariance matrix.

Returns:

C – The extended covariance matrix for each trial. If a single trial was given (2d X), the leading dimension is squeezed out.

Return type:

ndarray (ntrials, nchannels + nchannels_other, nchannels + nchannels_other)

Notes

This assumes that the data are of shape (trials, samples, channels) and that the other set is of shape (samples, channels). The second set is typically an idealised response from the average across trials. The function could however also be called on a single trial, for continuous recordings for instance. In that case, the method used is to extend the data with the a dummy dimmension for the trials and for P convolve the idealise response to singular event with a series of impulses at the times of those events.

pyeeg.preprocess.create_filterbank(freqs, srate, filtertype=<function cheby2>, **kwargs)

Creates a filter bank, by default of chebychev type 2 filters. Parameters of filter are to be defined as name value pair arguments. Frequency bands are defined with boundaries instead of center frequencies.

Parameters:

freqs (list or ndarray of float) – Boundary frequencies of the bands (in Hz). Each value is normalized by the Nyquist frequency (srate / 2) and passed as the Wn argument to the filter design function.
srate (float) – Sampling rate of the signal (in Hz).
filtertype (callable) – Filter design function used to build each filter (e.g. scipy.signal.cheby2, scipy.signal.butter). Must accept Wn and the keyword arguments in **kwargs.
**kwargs – Additional name-value pairs passed to filtertype (e.g. Rs, N, or btype for Chebyshev type II filters).

Returns:

fbank – List of filter coefficients, one (b, a) tuple per frequency band.

Return type:

list

pyeeg.preprocess.get_power(signals, decibels=False, win=125, axis=-1, n_jobs=-1)

Compute the (log) power modulation of a signal by taking the smooth moving average of its square values.

Parameters:

signals (ndarray (nsamples, nchans)) – Input signals
decibels (bool) – If True, will take the log power (default False).
win (int) – Length of smoothing window for moving average (default 125) in samples.
axis (int) – Axis on which to apply the transform
n_jobs (int) – Number of cores to be used (Parrallel job).

Returns:

out

Return type:

ndarray (nsamples, nchans)