Spectral Synthesis

In this part of the tutorial we will have to work on the synthesizing our analysis data back into something audible. Look at the following diagram:

This is just the basic scheme of the SMS synthesis but remember that we have to undo the effect of the circular shift and the zero padding. Afterwards we can multiply by the inverse of the analysis window and apply the overlap-add process.

For our application we would like to resynthesize the sinusoidal and residual components of the signal seperately aswell as together. It might be wise to encapsulate the spectral synthesis part of the process as a seperate ProcessingComposite or a sub function of our main Do() method. But to start with, we will only work on the synthesis of the residual component.

Add an IFFT to your application

1. Add also an audio plot to show the output of the IFFT and comment the shape of the synthesized audio frame using the sine.wav file.
2. To undo the effect of the analysis CircularShift we can use a CircularShift processing object configured to -AnalysisShift amount. Implement the necessary process to undo the analysis circular shift. You can use the Do method in the SpectralSynthesis class as a reference. What are the most important steps you had to take? A!
3. Implement the necessary process to undo the zero-padding (this is the zero-padding used by the residual branch of our analysis). What are the most important steps you had to take? A!

To undo the effect of the analysis window, we have to multiply the audio frame by the inverse of the analysis window. To do so, you will have to use a WindowGenerator (configured to generate the inverse of the analysis window used in the residual component) and an AudioMultiplier.

1. Add this part to your application and comment the changes that take place in the synthesized audio frame. Insert a screenshot of the audio frame at this point and comment the result.

We are now ready to perform the overlap-add. The OverlapAdd object in CLAM will take two hop sizes worth of data and will output one hop size worth of overlap-added data. If you append all the output chunks in a series, you end up with the signal you want. This is a bit more tricky than it sounds. If for the residual branch of your analysis you'd chosen a window size of more than twice the hop size + 1, you will soon have a problem. If you've payed close attention you'd notice that in this case we've ended up with more data than two hop sizes which are needed for the OverlapAdd. The thing to do is to 'cut out' two hop sizes worth of data centered around the center of your (window size) audio chunk. Now we have the correct amound of data, but if you were to perform the overlap-add, you'd notice another problem; the overlapping regions aren't 'crossfaded' but simply added. The solution to this problem is to multiply your two hop sizes worth of data with a window to 'smooth out' the transitions between the different chunks. The most common window for this is the triangular window (without normalizing) although there are other windows which can be used aswell (but not every window!).

5. Multiply your synthesized audio frame by a triangular window. Insert a screenshot of the audio frame at this point and comment the result.

Finally, we can perform the actual overlap-add. The OverlapAdd Processing in CLAM uses a circular buffer to implement it's overlap-add efficiently. Everytime a new frame (two hop sizes of data) arrives, the first half is added (ie. 'mixed-in') to the data which was already in the circular buffer and the second half just overwrites the space that follows.

6. Add an OverlapAdd to your synthesis class so you can correctly synthesize the residual component. Insert a screenshot of the audio frame at this point and comment the result.