Available DSP functions

____MODULE gicmext.cellmodels____

random

Generate random events at rate (Hz) and place them in an Event Data

element at newpath. if specified, start determines a time when the

events begin, and dur a duration (in seconds) over which the events

continue to be generated. If not specified, start is ds.start() and

dur is the length of the domain of ds. Spikes are generated as

indexes, and sampling rate is ds.fs()

lif

Use the data from select (this should be 1 channel, and if it isn't,

only the first channel is used) as a driver potential for a leaky

integrate and fire model with the indicated parameters. Place the

output spike train in newpath. Refract: The amount to reduce the

driver potential when an event occurs Leak: The "current" which

returns the accumulated potential to 0. Units are in exitation units

per DP per sample Threshold: The potential at which events trigger

rnoise: The stdev of the distribution of Refract (default 0) -

non-zero values will make things slow returnv: if true, the data

element stored in newpath will be a timeseries type element

containing the actual voltages, rather than the resulting events -

currently prevents using the c implementation, and this will make

things slow. SWITCHVALUES(returnv)=[False, True]

____MODULE gicmext.comparators____

clusterTrains1

attempts to caluculate the minimal perturbations required to convert

a single most representative spike train in the labeled event data at

<dpath> into every other spike train. These transformations are then

used to relabel the spikes in each spike train. The data element

written at newpath is a labeledevents set with two label columns

multiDistance

Calculate the Victor distance between all pairs of spike trains in

the labledEvent element at <dpath>. Store the output at newpath.

clusterTrains2

attempts to cluster spikes in the labeledevent instance at dpath.

Cost is the cost per ms to move a spike and is required because spike

correspondance is determined by converting between spike trains using

a Victor algorithm. "avoid" is a fraction of spike trains comparisons

that are not considered. These are the comparisons with the longest

pairwise distances. require is a fraction of the total presentations

in which a spike must exist, and simthresh is the level of group

similarity required to assign spikes to the same event (high values

will result in more unique events, lower values in more permissive

event groups).

testOptimizer

Test function for optimizers. Sets an attribute "Error" to a value

that is abs(number of points in ds > par1 - number of points in ds <

par2). ds must have local data, and should probably be a timeseries.

victorDistance

calculate the Victor/Purpura metric space spike distance between

event data elements in <dpath1> and <dpath2>. The result is stored as

an attribute of ds named <attrib> cost is the cost per millisecond to

shift a spike. Due to issues with float comparison, the calculation

is performed on the integer-valued raw event data. This means that

both event elements must have the same sampling rate. It also means

that there can be some round off error in the cost for coarsly

sampled data. A cost value that results in the cost of shifting a

spike by one sample bin being >=2 is treated as a cost of infinity.

____MODULE gicmext.composition____

power

Calculate the cumulative power of the selection (sqrt(s**2)) and

stores the reluting single channel in dpath

____MODULE gicmext.predictors____

Filter

Apply the data specified by selectFilter as a filter on the data

specified by select. Store the resulting signal in newpath.

histToMoments

corseAlign

align events from an event set in dpath with the signal in select

such that the signal is maximal at the time of the events. Maxshift

is the largest shift (in seconds) allowed. A single shift is applied

to the whole event set

getTestData

Gets the data (and events) stored in "/testdata" and moves them to

the toplevel. Moves toplevel data and events to "/testdata". (for

clairity it wolud be appropriate to rename this to "/trainingdata",

but keeping the name constant allows this function to toggle the

contents of the top level and /testdata.) In general this functino

should only be used after setTestData.

setTestData

Sets all data between the indicated indexes for use as validation

data. These data are extracted from "/" and stored in "/testdata". In

addition, for every "events" or "labeledevents" element at nesting

level 1 the same set of indexes are extracted and stored in

sub-elements of "/testdata". Essentially, this is a macro around 2 or

more calls to subdata.extract. If percent is set to true, the values

of the xindex parameters are treated as percentages of the total

length of the data rather than integer indexes.

SWITCHVALUES(percent)=[True, False]

pcaKernel

Applies a PC-rejection distance measure to the data in select, using

the mean and principal components in dpath (as generated by the pca

function in this module). The result is stored in newpath.

fineAlign

Align events one at a time.

pca

Calculates principal components of a set of vectors stared in dpath

(must be 2D NxM). Stores a 2D array in newpath. This matrix contains,

in the first column, the mean, and in the remaining colums, the

principal eigenvectors. howMuchVar determines how many eigenvectors

to record. If this number is >=1, all eigenvectors are recorded,

otherwise enough components to explain >= this fraction of the total

variance (around the mean) are recorded. If Inverse is true, the

least significant PCs are recorded first, and "howMuchVar" of the

varriance is left unexplained. The labels of the eigenvector channels

contain the corresponding eigenvalue. SWITCHVALUES(mode)=[True,

False]

projectionAnalyzer

construct the de-biased distribution of conditioned projections onto

a filter or set of filters. The filters must have the same length as

the conditioned events in dpathConditional. The distributions are

calculated as histograms in bins of width "step"

naiveProjectionModel

Get a direct measure of projection proability

____MODULE gicspikesort.conditioning____

ensembleStats

Create a new timeseries in newpath that contains the statistics for

each channel of the ensemble in dpath. If order=1, include only the

mean. Higher orders include higher moments (2= std deviation,

3=skewness, etc)

triggeredInsert

Use the events in dpath to insert the waveforms determied by

selectInsert into the data determined by selectTarget. One copy of

selectInsert will be inserted for each event. selectInsert must

specify the same number of channels as selectTarget. Offset

determines the number of samples before the an event to begin each

insert (may be negative). The special value "auto" will attemp to use

the header entry "Lead" (which is set by eventCondition and similar

functions), and defaults to 0 otherwise. Mode determines how to apply

the inserts. Legal values are: "subtract" - subtracts the "insert"

from the signal "add" - adds the insert from the signal "replace" -

sets the signal to equal the insert "smoothreplace" - sets the signal

to equal the insert, using an edge smoothing algorithm. If cumulative

is true, then when several inserts overlap, their effects add

(otherwise the second insert overrides the first). Using cumulative

makes computation slower, so if events don't overlap anyway, set this

to False SWITCHVALUES(mode)=['subtract', 'add', 'cumreplace',

'replace', 'smoothreplace'] SWITCHVALUES(cumulative)=[True, False]

dejitter

Takes ensemble data and temporally aligns (dejitters) the columns

using a gausian model activation distance measure. Max/eftshift and

maxright shift specify hard limits for shifts to the left (earlier

times) and right in sample points. Shiftsd specifies a regularization

constraint that penalizes large shifts. It is applied such that the

standard deviation of the resulting shift indexes tends to this value

(in sample points). A very large value results in the best absolute

alignment. If this is a False value (including 0), it is treated as

infinite (no penalty term). newpathTemplate specifies a path to store

the shift indexes. newpathShifted specifies a path to store the final

dejittered waveforms. If either of these arguments is False, the

corresponding component is not stored (in the case of the waveforms,

they are also not calculated). If dejittered waveforms are

calculated, they will be shorter than the original ensemble events,

since they will be extracted from the middle of those events as

events[maxrightshift:-maxleftshift] such that all the values will be

well specified after shifting.

repeatedWindows

Create an ensemble of samples, similar to eventCondition. Unlike

eventCondition, these waves are not based on an 'event' data set.

Instead, the ensemble is created by selecting sequential windows of

length xcoordPeriod, covering the entire data data set specified in

select. The resulting ensemble data is stored in newpath. Also unlike

eventCondition, xcoordPeriod is _not_ in ms (use normal units for the

x axis - usually seconds)

isolateSpikes

Drops all events that are too close to other events. Events are taken

from dpath, and a new event object is placed in newpath (overwriting

any previous object with that path. The new object contains the same

events as the old object, except that events with another event less

than timeBefore (in ms) before them or timeAfter after them are

removed. If absolute is False, spikes are processed one at a time, so

a later spike may not be dropped if the earlier spike close to it is

dropped first. SWITCHVALUES(cumulative)=[True, False]

eventCondition

Use the events in dpath to condition the data in select. Each event

will trigger a block of data that starts lead ms before the event

(lead may be negative) and is length ms long. These data will be

compiled into an "ensemble" type data instance stored in newpath. See

also mien.datafiles.dataset.alignEvents, which this function calls.

In particular, there is no need to strip "edge" events that do not

generate a full length window - these are filtered out automatically

____MODULE gicspikesort.spikes____

tsToEvents

Creates a set of events from a time series containing only zeros and

ones. If used on a general time series, this will produce an event

for every non-zero sample.

linearShift

Shift data in select such that the channels are offset by ns samples

each

tsFromEvents

Creates a time series of ones and zeros, with ones at the locations

of each event in dpath. If newpath exists, this sequence will be

appended to that timeseries data. Otherwise a new sub data element

will be created

localPeakFind

Smarter (but slower) spike detection. Should not be confused by

drifting baseline and amplitude.

schmitTrigger

Apply a shmidt dual thresholding scheme to the selected data,

generating events. The events are stored in a subdata element with

the indicated dpath. If that element exists and is of an event type,

the events are added as a new labeledevents channel. Otherwise, a new

events element is created. If select specifies multiple channels,

this will be a labeledevents element. "above" determines the

direction of the threshold. If it is False, detect minima Note that

if select specifies a subdata element, and dpath does not begin with

"/" it is interpreted relative to the element specified by select.

SWITCHVALUES(above)=[True, False]

alignMinima

Create a template that would timeshift all the channels in the

selectSearch so that their minima occur at the same time. If

selectApply is not False, shift the channels in this data region

according to the template. If newpathTemplate has a true value, the

norm template will be saved to that dpath. If there is already a data

elemnt there, it will be ammended (it's values will be multiplied by

the new template) (Note that if selectApply and newpathTemplate are

both False, this funcion does nothing other than generate heat and

benchmarking numbers :)

icDetrend

Attempt to condition an intracellular spike waveform for easier

threshholding (by removing drift and amplification changes. Ws is a

time window parameter is seconds. Smaller ws provides finer grained

detrending, but is more likely to damage actual spike data. Select

should specify one channel.

combine

Make a new data element with path "newpath" containing the event

records from the elements in "events" (this is a list of strings

specifying the dpaths of the events). 'newtype' determines what sort

of element is created, and can be histogram, labledevents, or events

(if it is events, the identities of the separate event elements wil

be lost). If 'delete' is true, the selected event elements will be

removed after they are added to the new element.

SWITCHVALUES(newtype)=['events', 'labeledevents', 'histogram']

SWITCHVALUES(delete)=[False, True]

____MODULE mien.dsp.channelops____

sameShape

Makes dpath2 the same length, start, and Fs as dpath1

deleteChannels

qualityBasedMerge

Merge channels in the data element dpath according to a quality

indicator. The function assumes that the channels are dividied into

some number of alternate recordings of the same information. The

recordings have blocksize total channels. Of these, all but one of

the channels are data records, and the single channel with index qci

within each block is a quality indicator. The data are merged such

that there are a total of blocksize final channels, and each one has

the value at each sample point drawn from the block that had the

highest value of the quality channel at that sample point.

ShiftChannels

shifts the listed channels by xcoord1-xcoord2

duplicateChannel

m akes a copy of the channel chanFrom and adds it to the dataset with

name cnameTo

ravel

Convert the selected data object to a single channel by stringing the

existing channels together (inverse of "fold")

reorderChannels

set channel order

crop

Crop the data in dpath, by samples, not times. nsamples is the total

length of the data after the crop. If it is unspecified, it will run

to the end of the data

____MODULE mien.dsp.frequency____

cohere

For each channel in the data specified by select1, calculate the

coherence between this and the coresponding channel in select2. Store

the array of coherences in dpath. If events are selected, they are

converted to timeseries of zeros and ones before the calculation

(this conversion affects only the calculation, not the stored event

data).

PSD

for each channel of data specified in select, calculate the power

spectrum. Store the resulting data in dpath

____MODULE mien.dsp.generators____

randomEvts

newBlankSubData

Generates data as blankTS, but stores it in a new subdata element at

dpath. fs is automatically set to ds.fs().

addSubToMain

get the data contained in the subelement dpath and add it to the data

specified by select. If delete is True, then delete element dpath.

SWITCHVALUES(delete)=[True, False]

GWN

Generates band limited Gaussian white noise in the frequency band

(bandMin, bandMax) with mean 0, and standard deviation std. This

noise is added to all the data specified by select. If rseed is

specified it will be used to seed the random number generator before

generation. This will allow repeated generation of identical "noise".

Rseed should probably be an integer, though it may also be an array

of ints. If invert is True, the noise will be multiplied by -1. This

is usally irelevant, but can be used in combination with rseed to add

and later remove identicle noise samples. Notes: std is specified

before bandpass filtering, so for very narrow bandwidths it will act

as an upper bound rather than an exact measure. bandMin and bandMax

are implicitly limited to the range 0, ds.fs()/2, and no filter will

be applied if they are outside these limits, or if both are False.

bandMax will be automatically set to ds.fs()/2 if it is None but

bandMin is >0.

blankTS

Cause ds to contain data of SampleType timeseries that is all zeros.

The new data will be of type float32, have nchans (int) channels, and

a duration of dur (float) seconds at a sampling rate of fs (float)

Hz. StartTime is set to 0, and channels are labeled c0 c1 c2 etc.

____MODULE mien.dsp.nmpml____

saveData

Saves the data in ds (or the specified subelemnt if dpath is not /)

to the file fname, using the specified format. This will not alter

values in ds. SWITCHVALUES(recurse)=[False, True]

sendData

Sets the data element indicated by upath to have the data and header

values of ds (or specified subdata element, if dpath is not /). upath

is an nmpml unique path in the document containing ds, _not_ a dpath

within ds. upath obeys the behavior of NmpmlObject.getInstance, so if

it begins with a '/' it is absolute in the document, and if it does

not, it is relative to ds, but in either case it is a upath (using

/tag:name components/), not a dpath (which would use only /name/

components) This will not alter the values in ds, or allow side

effects between ds and the new element. The new element will be

created if it does not exist, but if its container does not exist,

this will raise an error. If recurse is true, clones of any

subelements of the specified data will be added as subelemnts of the

target, and also any pre-existing subdata of the target will be

deleted. SWITCHVALUES(recurse)=[False, True]

callMethod

Calls the method "method" of the nmpml element with upath, passing it

args (method defaults to "run" and args to (), so calls to an nmpml

run method need only specify the upath). Args may be a tuple or a

dictionary.

loadData

loads data from fname, and places it in ds (or specified subelement.

If format is None, attempts to guess the format (usually this works,

but sometimes it requires user interaction) If upath is specified,

attempt to find the sub-data element keyed by upath in a complex

document. If this is None (default), and fname reffers to a complex

document, this function will use the first Data instance returned by

that documents getElements('Data') method.

SWITCHVALUES(recurse)=[False, True]

setAttribute

set an attribute on ds or the specified subdata

callMatlab

recieveData

Sets the data and header of ds to be the same as those of the data

element at the specified upath. If dpath is specified (and not / or

None), this function will initialize the specified subdata instead

(creating it if needed). As with sendData, upath is not a dpath. If

recuse is true, subdata will also be imported.

SWITCHVALUES(recurse)=[False, True]

____MODULE mien.dsp.signal____

makeHistogram

Create a histogram across the dependent variable of timeseries data

in "select". The histogram with have the indicated binwidth (in

numerical value, not sample points). Select should specify only one

channel. If it specifies more, only the first one will be used.

multiply

multiply the selected data by factor

sumChannels

sigmoid

Passes the selection through a sigmoid function with the indicated

parameters. If normalize is True, each channel is mapped onto the

range 0 to 1 before applying the sigmoid (this can reduce uncertainty

in the choice of sigmoid midpoint) SWITCHVALUES(normalize)=[True,

False]

normalize

Scale the region selectApply with coefficients that wauld cause the

channels in the region selectSearch to have the same amplitude (==1,

mode determines how the amplitude is measured). selectSearch

specifies the data to determine the normalization template from (for

a typical normalization this is the same as selectApply). Mode

determines what statistic is normalized, and may be max, min, std, or

range (std normalizes the standard deviations). If newpathTemplate

has a true value, the norm template will be saved to that dpath. If

there is already a data elemnt there, it will be ammended (it's

values will be multiplied by the new template)

SWITCHVALUES(mode)=['max', 'min','range', 'std']

mapToRange

Smoothly map the selection so that it has the indicated minimum and

maximum values

bandpass

applies a bandpass filter to each channel of data indexed in chans.

minFreq and maxFreq are in Hz

add

Add factor to the selected data

applyTemplate

Applies a scaling template (generated by functions like "normalize")

stored in dpath to the data specified by select. Mode may be "scale"

(multiply), "offset" (add), or "shift" (move in time). If invert is

True, the template is applied "backwards" (so as to undue the

operation it origionally did - for scaling this means devide instead

of multiply, for adding, it means subtract, and for shifting it means

shift left instead of right) SWITCHVALUES(mode)=['scale',

'offset','shift'] SWITCHVALUES(invert)=[True, False]

resamp

Change the sampling rate of the data element at dpath. If newfs is

None' us the sampling rate of ds.

zeromean

Remove the channel means from each channel in the selected data. The

means are calculated for samples within the selection. If applyToall

is True, the subtraction is performed on the whole channels

(otherwise the subtraction is also performed on the selection).

SWITCHVALUES(applyToAll)=[True, False]

clip

clip the selcted data so that it is never less than ycoordMin and

never greater than ycoordMax. If the values are None, don't clip in

that direction

blank

Set the selectied data to zeros

____MODULE mien.dsp.subdata____

delete

delete the data element at dpath

copy

Like move, but generates a clone of the origional to place at the new

location

extract

Move the selected data to a new data element at newpath. If delete is

False, leave the data in the origional location as well (copy rather

than move). For all types other than events, delete is only possible

if select specifes either all samples, all channels, or the entire

element. If select specifies an entire element (it is of the form

("foo", None, None)), this function acts as move or copy (depending

on the value of "delete"). SWITCHVALUES(delete)=[False, True]

combine

Combine the two data elements. For timeseries, this adds channels in

dpath2 to dpath1. If copy is False, dpath2 is then removed.

SWITCHVALUES(delete)=[False, True]

colapseAllToTop

fold all subdata into the top level element if possible

moveToTop

Use the data specified in dpath as the contents of the toplevel data

element. If swap is True, also copy the data from the toplevel

element into dpath. Otherwise, delete these data, and the subelement

dpath SWITCHVALUES(swap)=[False, True]

move

Moves a data element from dpath to newpath. Newpath should be the

path of the new container (not the new path for the element itself).

If newpath doesn't exist, a new element of SampleType "group" will be

created with that path.

swap

Exchange the data element at dpath1 with the one at dpath2.

rename

Change the name of the element at dpath

appendSamples

Add the samples in "select" to the existing data in element dpath

(this must be sampled data with the same number of channels). If

delete is true, then remove the selected adat after the copy.

SWITCHVALUES(delete)=[False, True]

mien.dsp.generators.newBlankSubData

Generates data as blankTS, but stores it in a new subdata element at

dpath. fs is automatically set to ds.fs().

mien.dsp.nmpml.sendData