Summary for implemented descriptors

Descriptors sumary

This section list some examples of the kind o features/descriptors that are obtained in current projects. It is not an extensive list, its purpose is simply to take some use cases.

Stats (Both nominal and absolute)

All those descriptors can be extracted from any raw TData arrays whatever the meaning.

They can also be aplied to objects having defined some arithmethic operators (*/+-). It is expected that those objects are structs containing TData fields so that the result of applying such arithmethic operators is an object of the same type applying the same operator over those TData fields one by one.

Frame Representations

The frame can have several representations:

Their relations are 1 to 1 with an AudioFrame.

Audio Descriptors

Those descriptors are calculated from the subject audio. Most only depend on the raw data and its size but some of them also depends on the audio SampleRate.

Some attributes have sense only when the audio represents a bigger audio than a frame, so when the audio is a frame audio, they are not suposed to be used/aggregated. Have a lot of sense on audio that only represents a single note/sample.

They need SampleRate: (Temporal)Centroid, (Log)AttackTime, Decrease,

Spectral Descriptors

Spectral descriptors are computed mainly from stats over the Spectrum magnitude buffer. Some of them use the spectral range and the spectrum size to normalize bin position into frequencies.

SpectralPeak descriptors

Frame Descriptors

Are calculated for each kind of subdescriptor if an analog object is present on the related frame.

Segment Descriptors:

Contains Frames Descriptors that contains the statistical agregations of every descriptor on the contained FrameDescriptors. There is no subsegment aggregation.

Note

Melody

Distribution Descriptors

They take the MelodySegmentDescriptors and then calculates the following statistics:

On the following Note attributes along the Melody Segment children

Envelope Descriptors

Fundamental Frequency Histograms descriptors

Those descriptors are calculated, firstly by computing two histograms on the fundamental frequency of every frame on a given segment. And then, computing some values on those histograms.

Histograms can be computated progressively from a frame fundamental pipeline. Also both maximums can be computed progressively since you only have to compare with the last updated bin. It has the property that the progresive calculation is true for the already computed frames.

Histogram extraction and histogran descriptors computation may be generalized. The folded version is not

Appendix: Aggregating numerical descriptors

Primitives

U: (R -> R):  pow, exp, log, abs, inv, neg
B: (R x R -> R): sum, mult, min, max, dif, div
N: (Rn -> R): size
S: (? -> ?): getFieldArray

vectorizeUnary: U -> (Rn -> Rn)
        vectorizeUnary(U)(X) = [U(x1),U(x2),...]

vectorizeBinary: B -> (Rn x Rn -> Rn)
        vectorizeBinary(B)(X,Y) = [B(x1,y1), B(x2,y2)...]

accumulate: B x R -> (Rn -> R)
        accumulate(B,a)(X) = B(B(...B(B(a,x1),x2)...),xn)

accumulate: B -> (R x Rn -> R)
        accumulate(B)(a,X) = B(B(...B(B(a,x1),x2)...),xn)

accumulate: B -> (Rn -> R)
        accumulate(B)(X)   = B(B(...B(B(x1,x2),x3)...),xn)

partial_accumulate: B x Rn -> Rn
        partial_accumulate(B,X) = [ x1, B(x1,x2), B(B(x1,x2),x3), ]

adjacent: B x Rn -> Rn
        adjacent(B,X) = [ x1, B(x1,x2), B(x2,x3), B(x3,x4),...]

invertArguments: B -> B
        invertArguments(B)(x,y) = B(y,x)

bindFirst: B x R -> U
        bindFirst(B,a)(x) -> B(a,x)

bindSecond: B x R -> U
        bindSecond(B,a)(x) -> B(x,a)