Semantical Analysis

---

Detailed Description

This module explains how to use CLAM to handle audio description extraction using the DescriptionScheme object and its relatives.

They will allow you to do the extraction in a modular and incremental way taking from other projects the parts you are interested in and adding your own descriptors in a incremental way.

This module intends to implement the system described on http://www.iua.upf.es/mtg/clam/devel/doc/descriptors/Descriptors.html but there is still some way to achieve the full functionality described in there.

Defining and instanciating descriptors

The central object for description extraction is the DescriptionScheme. The description scheme (CLAM::DescriptionScheme) defines which are the attributes (CLAM::Attribute) we want to compute. You can relate attributes to a name and a type and you can organize attributes in different ''scopes''. You can understand a scope (CLAM::DescriptionScope) as the kind of target for a given set of attributes. For example, we normaly talk about note scope, sample scope, frame scope, phrase scope... that means that a given attribute will have a value for every single note, sample, frame, phrase...

 CLAM::DescriptionScheme scheme;
 scheme.AddAttribute <CLAM::TData>    ("AudioSample", "SignalLevel");
 scheme.AddAttribute <CLAM::TData>    ("AudioSample", "FilteredSignal");
 scheme.AddAttribute <SamplePosition> ("Frame", "Center");
 scheme.AddAttribute <CLAM::TData>    ("Frame", "Energy");
 scheme.AddAttribute <CLAM::TData>    ("Frame", "RMS");
 scheme.AddAttribute <CLAM::Spectrum> ("Frame", "SpectralDistribution");
 scheme.AddAttribute <FramePosition>  ("Note", "Onset");
 scheme.AddAttribute <CLAM::Pitch>    ("Note", "Pitch");

The description scheme only specifies the attribute organization. The real values are hold into the data pool (CLAM::DescriptionDataPool). An instance of a DescriptionDataPool will hold the attributes extracted from a single description source (ie, an audio). It take the structure defined by the description scheme.

 CLAM::DescriptionDataPool pool(scheme);

So, summarizing:

• A description scheme defines attributes to be computed
• Every attribute has its name and type and is related to a given scope
• A scope specifies the kind of attribute target
• A description data pool is the real container for the values computed by extraction.
• A description data pool matches the structure specified by a given description scheme.

Accessing the pool by hand

Accessing the pool by hand is not the ideal way of doing it but, currently, extractor binding is not so complete so, by now, it is the only way to do certain things.

The scope provides interface to:

• Populate a given scope with a size. That is for the scope Note, telling how many notes there are.

     pool.SetNumberOfContexts("Note",60);
  

• Obtaining the attribute pool for writting, so you will get a vector of pitches one for every note.

     CLAM::Pitch * pitches = pool.GetWritePool<CLAM::Pitch>("Note","Pitch");
  

• Obtaining the attribute pool for reading

     const CLAM::Pitch * pitches = pool.GetReadPool<CLAM::Pitch>("Note","Pitch");
  

Warning:

A write access is required before any read access. The pool will create the attribute pool memory only when somebody requires to write in it.

The access is templatized by the attribute type. The pool user need not to handle generic types (void*, casts...) and her code keeps typesafe. Some checking between the usage and the real type for the attribute is done on run-time. So if you use a different value type an assertion will fail.

XML

Description data pools can be loaded or stored in XML, as any other CLAM::Component, by using an CLAM::XmlStorage.

 // Storing a description in XML
 CLAM::XmlStorage::Dump(pool, "DescriptionPool", "mysong.xml");

 // Recovering an XML description
 CLAM::XmlStorage::Restore(pool, "mysong.xml");

Binding extractors

While you can use the pool simply as a container, like it has been explained above, the aim of this system is to be able to deploy the extraction system from an XML file that describes the Description Scheme and the Extraction Scheme. This would be done by encapsulating algorithms that compute attributes on an abstraction called CLAM::Extractors. They should be something very close to what a CLAM::Processing is and, in fact, they will likely converge as the iterations go on. An extractor has hooks for input and output data that are fetched from the data pool. The way data is fetched and droped is determined by the binding.

So, this part of the module is work on progress but there are some parts already implemented and usable. By now, what we have is some kinds of hook binding. Current implemented binding operations are bindings on the same context, and indirection, that is, using an attribute to point another one even on a different scope.

By now, there is no such abstract CLAM::Extractor but you can take a look to some Extractors CLAMTest::CharCopierExtractor and CLAMTest::CharJoinExtractor. Also you can see how binding is done by looking at the ExtractorTest.cxx file.

What is left to implement

• An abstract CLAM::Extractor to derive from
• An special kind of extractor for scope population (how many items in a scope?)
• Bindings extension: relative position
• Solving Range and Relative bindings when outside the scope space
• The type system
  • Defining units friendly types for using them in attributes
  • Solving the creation of concrete Attribute specification from the type name
• XML Serialization for schemes
• Exploring new hook binding functionalities driven by real cases

Classes
class	CLAM::AttributePool
	A container for the values for a single attribute along the scope. More...
class	CLAM::AbstractAttribute
	Defines the interface for an Attribute definition in a DescriptionScheme. More...
class	CLAM::Attribute< AttributeType >
	This class is the concrete implementation for AbstractAttribute for a given type of attributes. More...
class	CLAM::DescriptionScheme
	The description scheme defines the set of attributes (Attribute) to be used for feature extraction systems. More...
class	CLAM::SchemaError
	A description scope defines a set of attributes which have the same ocurrences. More...
class	CLAM::Hook< AttributeType >
class	CLAM::ReadHook< AttributeType >
class	CLAM::ReadRangedHook< AttributeType >
class	CLAM::WriteHook< AttributeType >
class	CLAM::DescriptionDataPool
	Contains the extracted data for a given description process. More...
class	CLAM::ScopePool
	A container for the attributes values along the differents contexts of a single scope. More...