osl_ephys.preprocessing.osl_wrappers#

Wrappers for MNE functions to perform preprocessing.

Attributes#

logger

Functions#

`gesd`(x[, alpha, p_out, outlier_side])	Detect outliers using Generalized ESD test
`_find_outliers_in_dims`(X[, axis, metric_func, gesd_args])	Find outliers across specified dimensions of an array
`_find_outliers_in_segments`(X[, axis, segment_len, ...])	Identify outlier segments within an array.
`detect_artefacts`(X[, axis, reject_mode, metric_func, ...])	Detect bad observations or segments in a dataset
`detect_maxfilt_zeros`(raw[, use_maxfilter_log])	This function tries to load the maxfilter log files in order
`bad_segments`(raw, picks[, segment_len, ...])	Set bad segments in an MNE `Raw` object as defined by the Generalized ESD test
`bad_channels`(raw, picks[, ref_meg, significance_level])	Set bad channels in an MNE `Raw` object as defined by the Generalized ESD test in `osl_ephys.preprocessing.osl_wrappers.gesd`.
`drop_bad_epochs`(epochs, picks[, significance_level, ...])	Drop bad epochs in an MNE `Epochs` object as defined by the Generalized ESD test in `osl_ephys.preprocessing.osl_wrappers.gesd`.
`detect_bad_channels_psd`(raw[, fmin, fmax, n_fft, alpha])	Detect bad channels using PSD and GESD outlier detection.
`run_osl_read_dataset`(dataset, userargs)	Reads `fif`/`npy`/`yml` files associated with a dataset.
`run_osl_bad_segments`(dataset, userargs)	osl-ephys Batch wrapper for `bad_segments`.
`run_osl_bad_channels`(dataset, userargs)	osl-ephys Batch wrapper for `bad_channels`.
`run_osl_drop_bad_epochs`(dataset, userargs)	osl-ephys Batch wrapper for `drop_bad_epochs`.
`run_osl_zscore_present_data`(dataset, userargs)	z-scoring parametric regressors, without NaNs
`run_osl_glm_add_regressor`(dataset, userargs)	osl-ephys Batch wrapper for `osl_ephys.preprocessing.osl_glm.add_regressor`.
`run_osl_glm_add_contrast`(dataset, userargs)	osl-ephys Batch wrapper for `osl_ephys.preprocessing.osl_glm.add_regressor`.
`run_osl_glm_fit`(dataset, userargs)	wrapper for the different glm functions in the glm module
`run_osl_glm_permutations`(dataset, userargs)	wrapper for the different permutation options in the glm module

Module Contents#

osl_ephys.preprocessing.osl_wrappers.logger = None[source]#

osl_ephys.preprocessing.osl_wrappers.gesd(x, alpha=0.05, p_out=0.1, outlier_side=0)[source]#

Detect outliers using Generalized ESD test

xvector
Data set containing outliers

alphascalar
Significance level to detect at (default = 0.05)

p_outint
Maximum number of outliers to detect (default = 10% of data set)

outlier_side{-1,0,1}

Specify sidedness of the test

outlier_side = -1 -> outliers are all smaller

outlier_side = 0 -> outliers could be small/negative or large/positive (default)

outlier_side = 1 -> outliers are all larger

Returns:

idx (boolean vector) – Boolean array with TRUE wherever a sample is an outlier
x2 (vector) – Input array with outliers removed

References

B. Rosner (1983). Percentage Points for a Generalized ESD Many-Outlier Procedure. Technometrics 25(2), pp. 165-172. http://www.jstor.org/stable/1268549?seq=1

osl_ephys.preprocessing.osl_wrappers._find_outliers_in_dims(X, axis=-1, metric_func=np.std, gesd_args=None)[source]#: Find outliers across specified dimensions of an array

osl_ephys.preprocessing.osl_wrappers._find_outliers_in_segments(X, axis=-1, segment_len=100, metric_func=np.std, gesd_args=None, channel_wise=False, channel_axis=0, threshold=0.05)[source]#

Identify outlier segments within an array. Parameters: - X: np.ndarray

Input data array with dimensions (channel, time).

axis: int
The axis along which to segment the data (default is -1, the last axis).
channel_axis: int
The axis along which channels are stored (default is 0, the first axis).
segment_len: iant
Length of each segment along the specified axis.
metric_func: callable
Function to compute the metric for each segment (default is np.std).
gesd_args: dict
Additional arguments to pass to the GESD.
channel_wise: bool
If True, the function will treat each channel seperately when detecting bad segments.
channel_axis: int
The axis to treat as the channel axis. Only used when channel_wise=True.
threshold: str or float
Threshold for outlier detection. Only used when channel_wise=True. If ‘any’, a segment is marked as an outlier if any of its channels is an outlier. If a float, a segment is marked as an outlier if the fraction of outlier channels exceeds the threshold.

Returns: - bads: np.ndarray

Boolean array indicating outlier segments.

osl_ephys.preprocessing.osl_wrappers.detect_artefacts(X, axis=None, reject_mode='dim', metric_func=np.std, segment_len=100, gesd_args=None, ret_mode='bad_inds', channel_wise=False, channel_axis=0, channel_threshold=0.05)[source]#

Detect bad observations or segments in a dataset

Parameters:

X (ndarray) – Array to find artefacts in.
axis (int) – Index of the axis to detect artefacts in
reject_mode ({'dim' | 'segments'}) – Flag indicating whether to detect outliers across a dimension (dim; default) or whether to split a dim into segments and detect outliers in the them (segments)
metric_func (function) – Function defining metric to detect outliers on. Defaults to np.std but can be any function taking an array and returning a single number.
segement_len (int > 0) – Integer window length of dummy epochs for bad_segment detection
gesd_args (dict) – Dictionary of arguments to pass to gesd
ret_mode ({'good_inds','bad_inds','zero_bads','nan_bads'}) – Flag indicating whether to return the indices for good observations, indices for bad observations (default), the input data with outliers removed (zero_bads) or the input data with outliers replaced with nans (nan_bads)
channel_wise (bool) – If True, the function will treat each channel seperately when detecting bad segments, only used when reject_mode='segments'.
channel_axis (int) – The axis to treat as the channel axis. Only used when channel_wise=True.
channel_threshold (str or float) – The treshold to use for channel-wise detection. Only used when channel_wise=True.

Returns:

If ret_mode is 'bad_inds' or 'good_inds', this returns a boolean vector of length X.shape[axis] indicating good or bad samples. If ret_mode is 'zero_bads' or 'nan_bads' this returns an array copy of the input data X with bad samples set to zero or np.nan respectively.

Return type:

ndarray

osl_ephys.preprocessing.osl_wrappers.detect_maxfilt_zeros(raw, use_maxfilter_log=True)[source]#

This function tries to load the maxfilter log files in order: to annotate zeroed out data in the mne.io.Raw object. It assumes that the log file is in the same directory as the raw file and has the same name, but with the extension .log. If the log file can’t be found, it will look for zeros in the data.

Parameters:: raw (mne.io.Raw) – MNE raw object.
Returns:: bad_inds – Boolean array indicating which time points are zeroed out.
Return type:: np.array of bool (n_times,) or None

osl_ephys.preprocessing.osl_wrappers.bad_segments(raw, picks, segment_len=1000, significance_level=0.05, metric='std', ref_meg='auto', mode=None, detect_zeros=True, channel_wise=False, channel_axis=0, channel_threshold=0.05)[source]#

Set bad segments in an MNE Raw object as defined by the Generalized ESD test in osl_ephys.preprocessing.osl_wrappers.gesd. This function is typically used by calling run_osl_bad_segments.

Parameters:

raw (mne.io.Raw) – MNE raw object.
picks (str) – Channel types to pick. See Notes for recommendations.
segment_len (int) – Window length to divide the data into (non-overlapping).
significance_level (float) – Significance level for detecting outliers. Must be between 0-1.
metric (str) – Metric to use. Could be 'std', 'var' or 'kurtosis'.
ref_meg (str) – ref_meg argument to pass with mne.pick_types.
mode (str) – Should be None 'diff' or 'maxfilter'. When mode='diff' we calculate a difference time series before detecting bad segments. When mode='maxfilter' we only mark the segments with zeros from MaxFiltering as bad.
detect_zeros (bool) – Should we detect segments of zeros based on the maxfilter files?
channel_wise (bool) – If True, the function will treat each channel seperately.
channel_axis (int) – The axis to treat as the channel axis. Only used when channel_wise=True.
channel_threshold (str or float) – The treshold to use for channel-wise detection. Only used when channel_wise=True.

Returns:

raw – MNE raw object with bad segments annotated.

Return type:

mne.io.Raw

Notes

Note that for Elekta/MEGIN data, we recommend using picks: 'mag' or picks: 'grad' separately (in no particular order).

Note that with CTF data, mne.pick_types will return:: ~274 axial grads (as magnetometers) if {picks: 'mag', ref_meg: False} ~28 reference axial grads if {picks: 'grad'}. Thus, it is recommended to use picks:'mag' in combination with ref_mag: False, and picks:'grad' separately (in no particular order).

osl_ephys.preprocessing.osl_wrappers.bad_channels(raw, picks, ref_meg='auto', significance_level=0.05)[source]#

Set bad channels in an MNE Raw object as defined by the Generalized ESD test in osl_ephys.preprocessing.osl_wrappers.gesd.

Parameters:

raw (mne.io.Raw) – MNE raw object.
picks (str) – Channel types to pick. See Notes for recommendations.
ref_meg (str) – ref_meg argument to pass with mne.pick_types.
significance_level (float) – Significance level for detecting outliers. Must be between 0-1.

Returns:

raw – MNE Raw object with bad channels marked.

Return type:

mne.io.Raw

Notes

Note that for Elekta/MEGIN data, we recommend using picks:'mag' or picks:'grad' separately (in no particular order).

Note that with CTF data, mne.pick_types will return:: ~274 axial grads (as magnetometers) if {picks: 'mag', ref_meg: False} ~28 reference axial grads if {picks: 'grad'}. Thus, it is recommended to use picks:'mag' in combination with ref_mag: False, and picks:'grad' separately (in no particular order).

osl_ephys.preprocessing.osl_wrappers.drop_bad_epochs(epochs, picks, significance_level=0.05, max_percentage=0.1, outlier_side=0, metric='std', ref_meg='auto', mode=None)[source]#

Drop bad epochs in an MNE Epochs object as defined by the Generalized ESD test in osl_ephys.preprocessing.osl_wrappers.gesd.

Parameters:

epochs (mne.Epochs) – MNE Epochs object.
picks (str) – Channel types to pick.
significance_level (float) – Significance level for detecting outliers. Must be between 0-1.
max_percentage (float) – Maximum fraction of the epochs to drop. Should be between 0-1.
outlier_side (int) –
Specify sidedness of the test:
- outlier_side = -1 -> outliers are all smaller
- outlier_side = 0 -> outliers could be small/negative or large/positive (default)
- outlier_side = 1 -> outliers are all larger
metric (str) – Metric to use. Could be 'std', 'var' or 'kurtosis'.
ref_meg (str) – ref_meg argument to pass with mne.pick_types.
mode (str) – Should be 'diff' or None. When mode='diff' we calculate a difference time series before detecting bad segments.

Returns:

epochs – MNE Epochs object with bad epoches marked.

Return type:

mne.Epochs

Notes

Note that with CTF data, mne.pick_types will return:: ~274 axial grads (as magnetometers) if {picks: 'mag', ref_meg: False} ~28 reference axial grads if {picks: 'grad'}.

osl_ephys.preprocessing.osl_wrappers.detect_bad_channels_psd(raw, fmin=2, fmax=80, n_fft=2000, alpha=0.05)[source]#

Detect bad channels using PSD and GESD outlier detection.

Parameters:

raw (mne.io.Raw) – Raw data object.
fmin (float) – Frequency range for PSD computation.
fmax (float) – Frequency range for PSD computation.
n_fft (int) – FFT length for PSD.
alpha (float) – Significance level for GESD outlier detection.

Returns:

Detected bad channel names.

Return type:

list of str

osl_ephys.preprocessing.osl_wrappers.run_osl_read_dataset(dataset, userargs)[source]#

Reads fif/npy/yml files associated with a dataset.

Parameters:

fif (str) – Path to raw fif file (can be preprocessed).
preload (bool) – Should we load the raw fif data?
ftype (str) – Extension for the fif file (will be replaced for e.g. '_events.npy' or '_ica.fif'). If None, we assume the fif file is preprocessed with osl-ephys and has the extension '_preproc-raw'. If this fails, we guess the extension as whatever comes after the last '_'.
extra_keys (str) – Space separated list of extra keys to read in from the same directory as the fif file. If no suffix is provided, it’s assumed to be .pkl. e.g., ‘glm’ will read in ‘…_glm.pkl’ ‘events.npy’ will read in ‘…_events.npy’.

Returns:

dataset – Contains keys: 'raw', 'events', 'event_id', 'epochs', 'ica'.

Return type:

dict

osl_ephys.preprocessing.osl_wrappers.run_osl_bad_segments(dataset, userargs)[source]#

osl-ephys Batch wrapper for bad_segments.

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key raw.
userargs (dict) – Dictionary of additional arguments to be passed to bad_segments.

Returns:

dataset – Input dictionary containing MNE objects that have been modified in place.

Return type:

dict

osl_ephys.preprocessing.osl_wrappers.run_osl_bad_channels(dataset, userargs)[source]#

osl-ephys Batch wrapper for bad_channels.

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key raw.
userargs (dict) – Dictionary of additional arguments to be passed to bad_channels.

Returns:

dataset – Input dictionary containing MNE objects that have been modified in place.

Return type:

dict

Notes

Note that using ‘picks’ with CTF data, mne.pick_types will return:: ~274 axial grads (as magnetometers) if {picks: 'mag', ref_meg: False} ~28 reference axial grads if {picks: 'grad'}.

osl_ephys.preprocessing.osl_wrappers.run_osl_drop_bad_epochs(dataset, userargs)[source]#

osl-ephys Batch wrapper for drop_bad_epochs.

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key raw.
userargs (dict) – Dictionary of additional arguments to be passed to drop_bad_epochs.

Returns:

dataset – Input dictionary containing MNE objects that have been modified in place.

Return type:

dict

osl_ephys.preprocessing.osl_wrappers.run_osl_zscore_present_data(dataset, userargs)[source]#

z-scoring parametric regressors, without NaNs Nans will be zeros in the z-scored version

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key covs.
userargs (dict) – Dictionary of additional arguments containing the keys keys.

osl_ephys.preprocessing.osl_wrappers.run_osl_glm_add_regressor(dataset, userargs)[source]#

osl-ephys Batch wrapper for osl_ephys.preprocessing.osl_glm.add_regressor.

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key covs.
userargs (dict) – Dictionary of additional arguments containing the keys keys.

osl_ephys.preprocessing.osl_wrappers.run_osl_glm_add_contrast(dataset, userargs)[source]#

osl-ephys Batch wrapper for osl_ephys.preprocessing.osl_glm.add_regressor.

Parameters

osl_ephys.preprocessing.osl_wrappers.run_osl_glm_fit(dataset, userargs)[source]#

wrapper for the different glm functions in the glm module

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key covs.
userargs (dict) – Dictionary of additional arguments

Returns:

dataset – Input dictionary containing MNE objects that have been modified in place.

Return type:

dict

osl_ephys.preprocessing.osl_wrappers.run_osl_glm_permutations(dataset, userargs)[source]#

wrapper for the different permutation options in the glm module

Parameters:

dataset (dict) – Dictionary containing at least an MNE object with the key covs.
userargs (dict) – Dictionary of additional arguments

Returns:

dataset – Input dictionary containing MNE objects that have been modified in place.

Return type:

dict