"IVDisplayControl" best starting point for spectrum display or?

Made with iPlug2
1

I’ve been looking through the various Vector controls to see which one can be modified most readily to produce a basic spectrum display/graph. It looks like IVDisplayControl might work but maybe also IVScopeControl? There is also IVPlotControl - which seems like what I want - but looks like it only produces static images using fixed functions.

I realize most displays of this type (spectrum analyzers, EQ graphs, etc.) are highly customized but I’m curious what existing IPlug control might be a good starting point for this.

Any tips appreciated. Thank you.

2

IVPlotControl is a good starting point.

The code that runs this analyser (Surreal Machines) uses very similar code (but adds the ability to draw the data in a filled manner). Ontop of this I have a class that generates the set of points to plot from the magnitude spectrum.

I see what you mean by “static images using fixed functions” but your assumption is that the plot functions calculate directly - as IPlotFunc is a std::function it can be a stateful type (such as a functor), and therefore it is free to return plotted data, rather than calculate a function programmatically. In perhaps clearer words Plot::func can be an object that stores data (a struct or class with a suitable call operator).

If IVPlotControl doesn’t suit your purposes it’s worth noting that the core drawing function is IGraphics::DrawData() and if it’s easier I’d suggest rolling your own control around this (this is essentially the core of the code that I seem to have adapted for the display in Impact).

In my case I have two further important classes - one that does the DSP side of the analysis. The other is the bit that takes a power spectrum and does the correct log-log transforms to plot the spectrum (which transforms some data into the sort of format required by DrawData. I feel like @olilarkin and I might have spoken before about whether I’d share this for iplug2 (the graphics part at least), but I’m not 100% sure.

I’d be happy for something like this to go into Iplug2, but I don’t have time to reformat my code to use the IPlug2 vector style system (surreal machines uses our own system). There’s nothing particularly private in there though. This code is based on code for a Max object called spectrumdraw~ that I wrote a while back and is fully BSD licensed but that original code is truly horrible to work with directly. It might be ideal simply provide it as a helper class that takes spectral data and provides it to use with IVPlot control.

I’d be happy to provide the two plotting classes off list if someone wanted to either just look at them, or ideally modify them in a way that they might fit back into iplug2 as a spectral display is a pretty common thing to do…

1 Like
3

Actually - I can share it here with a few extras and people can do with it what they will. It could easily be turned into a control suitable to use with DrawData(). Some hacking will be required to disentangle from the underlying Plotter class, but probably not much.

I’m not sharing the plotter class, simply because it is pretty much is a close copy of DrawData() simply with a method for filling rather than stroking the data.I can post it if desired.

In header:

//////////////////////////////////////
//         Spectral Plotter         //
//////////////////////////////////////

class SpectralPlotter : protected Plotter
{
    struct Point
    {
        Point(float xIn, float yIn) : x(xIn), y(yIn) {}
        
        float x;
        float y;
    };
    
public:
    
    SpectralPlotter(const IRECT& bounds);
    
    void SetBounds(const IRECT& bounds) { Plotter::SetBounds(bounds); }
    
    void SetFreqRange(float freqLo, float freqHi, float sampleRate);
    void SetDBRange(float dbLo, float dbHi);
    
    // N.B. size is the number of points, the last being the nyquist (and not the FFT size)
    
    void Data(const float* powerSpectrum, unsigned long size);
    void Plot(IGraphics& g, const Plotter::Style& style);
    
protected:
    
    void AddPoint(const Point& p);
    void ConvertAndAddPoints(Point& p1, Point& p2);
    
    int NumPoints() { return static_cast<int>(mXPoints.size()); }
    
    float CalcXNorm(float x) { return (logf(x) - mLogXLo) / (mLogXHi - mLogXLo); }
    float CalcYNorm(float y) { return (logf(y) - mLogYLo) / (mLogYHi - mLogYLo); }
    
    float mOptimiseX;
    float mSmoothX;

    float mLogXLo;
    float mLogXHi;
    float mLogYLo;
    float mLogYHi;
    
    std::vector<float> mXPoints;
    std::vector<float> mYPoints;
};

In Source:
(this runs on fhe graphics thread and thus I don’t mind about allocating there, but if you wished the vector reserve stuff could be in a constructor (rather that Data() which is where you feed new data in). However, as vector never sizes down unless you explicitly make it happen this is going to be a one-off cost.

//////////////////////////////////////
//         Spectral Plotter         //
//////////////////////////////////////

SpectralPlotter::SpectralPlotter(const IRECT& bounds)
: Plotter(bounds), mOptimiseX(1.f), mSmoothX(1.f)
{
    SetFreqRange(20.f, 20000.f, 44100.f);
    SetDBRange(-100.f, 0.f);
}

void SpectralPlotter::SetFreqRange(float freqLo, float freqHi, float sampleRate)
{
    mLogXLo = logf(freqLo / (sampleRate / 2.f));
    mLogXHi = logf(freqHi / (sampleRate / 2.f));
}

void SpectralPlotter::SetDBRange(float dbLo, float dbHi)
{
    mLogYLo = logf(powf(10.f, dbLo / 10.0));
    mLogYHi = logf(powf(10.f, dbHi / 10.0));
}

// N.B. size is the number of points, the last being the nyquist (and not the FFT size)

void SpectralPlotter::Data(const float* powerSpectrum, unsigned long size)
{
    mXPoints.reserve(size);
    mYPoints.reserve(size);
    mXPoints.clear();
    mYPoints.clear();
    
    float xRecip = 1.f / static_cast<float>(size);
    float xAdvance = mOptimiseX / mBounds.W();
    float xPrev = 0.f;
    
    float ym2 = CalcYNorm(powerSpectrum[1]);
    float ym1 = CalcYNorm(powerSpectrum[0]);
    float yp0 = CalcYNorm(powerSpectrum[1]);
    float yp1 = 0.0;
    
    // Don't use the DC bin
    
    unsigned long i = 1;
    
    // Calculate Curve
    
    for (; i < size; i++)
    {
        int N = NumPoints();
        float x = CalcXNorm(i * xRecip);
        
        // Add cubic smoothing if desired
        
        if (i + 1 < size)
        {
            yp1 = CalcYNorm(powerSpectrum[i+1]);
         
            if (mSmoothX)
            {
                float xInterp = 1.0 / (x - xPrev);
            
                for (float xS = xPrev + xAdvance; xS < x - xAdvance; xS += xAdvance)
                    AddPoint(Point(xS, interpolateCubic((xS - xPrev) * xInterp, ym2, ym1, yp0, yp1)));
            }
        }
       
        AddPoint(Point(x, yp0));
        
        ym2 = ym1;
        ym1 = yp0;
        yp0 = yp1;
        
        if (N && (XCalc(mXPoints[N]) - XCalc(mXPoints[N - 1])) < mOptimiseX)
            break;
        
        xPrev = x;
    }
    
    while (i < size)
    {
        Point minPoint(CalcXNorm(i * xRecip), powerSpectrum[i]);
        Point maxPoint = minPoint;
        
        float x = XCalc(minPoint.x);
        float xNorm = CalcXNorm(++i * xRecip);
        
        while (((XCalc(xNorm) - x) < mOptimiseX) && i < size)
        {
            if (powerSpectrum[i] < minPoint.y)
                minPoint = Point(xNorm, powerSpectrum[i]);
            if (powerSpectrum[i] > maxPoint.y)
                maxPoint = Point(xNorm, powerSpectrum[i]);
            xNorm = CalcXNorm(++i * xRecip);
        }
        
        if (minPoint.x < maxPoint.x)
            ConvertAndAddPoints(minPoint, maxPoint);
        else
            ConvertAndAddPoints(maxPoint, minPoint);
    }
}

void SpectralPlotter::Plot(IGraphics& g, const Plotter::Style& style)
{
    Plotter::Plot(g, mXPoints.data(), mYPoints.data(), NumPoints(), style);
}

void SpectralPlotter::AddPoint(const Point& p)
{
    mXPoints.push_back(p.x);
    mYPoints.push_back(p.y);
}

void SpectralPlotter::ConvertAndAddPoints(Point& p1, Point& p2)
{
    p1.y = CalcYNorm(p1.y);
    p2.y = CalcYNorm(p2.y);
    
    AddPoint(p1);
    AddPoint(p2);
}

You’d also need the code for cubic interpolation which makes the low frequencies look nice…

This is a template because we use it for a mix of scalar and SIMD types, but it doesn’t need to be one for this usage (that’s also why all the inputs are passed by const reference - if I knew that the inputs were double/float I wouldn’t bother with that).

template <typename T>
inline T interpolateCubic(const T& x, const T& y0, const T& y1, const T& y2, const T& y3)
{
    // N.B. - this is currently a high-quality cubic hermite

    T c0 = y1;
    T c1 = 0.5 * (y2 - y0);
    T c2 = y0 - 2.5 * y1 + y2 + y2 - 0.5 * y3;
    T c3 = 0.5 * (y3 - y0) + 1.5 * (y1 - y2);

    return (((c3 * x + c2) * x + c1) * x + c0);
}
2 Likes
4

To fully explain - it looks like this class is a helper for creating spectral displays, so it doesn’t have most of the control stuff - the idea is that in your control class somewhere you call Data() to update the data, and Plot() when you are drawing.

As I said - making a version of this that would work with IVControl would perhaps be the easiest way to make something that could integrate into iPlug2.

1 Like
5

@AlexHarker

Thank you for all the tips, suggestions and code samples. Yes, I think this would be a valuable addition to IPlug’s current library of controls. If I get a basic class put together (as an IPlug control) that is usable for everyone I will post it up. I have much studying to do here first as I haven’t yet worked with vector drawing in IPlug but accept the challenge!

Thank you. :+1:

6

I’ve wanted to add an IVSpectrumControl and IVSpectrogramControl for ages but haven’t got round to it. I think @stw sent me some code once which I seem to have lost. It would be cool to get it working with the WDL fft, with an option to use apple accelerate

1 Like
7

Sorry @olilarkin, that wasn’t me… at least i don’t remember :thinking:

8

oops i think it was mttlvc

9

I haven’t dug in too deeply yet but I noticed that the IVDisplayControl and IVScopeControl in the IPlugControls example run very smoothly and continuously - like they are running independently/uninterrupted by ProcessBlock(). How is that happening? I thought graphics only updated during idle time between Process calls (usually making them appear somewhat “lurchy” in response time). In the case of an FFT driving a spectrum display I would expect the display to update only once per ProcessBlock and, therefore, appear a little “freeze frame-ish” - but yet the IPlugControls scope example doesn’t do that (it’s smooth). Do I misunderstand how IPlug graphics work or is there something special about the IVScope and Display controls?

10

I’m not sure of that specific control, but one technique is to add each block’s meter data as an element to a queue, and then pull entries from that queue to draw during an idle method. It decouples the two threads, allowing each to run at its own pace with minimal locking and blocking.

1 Like
11

What controls the pace at which the meter data is pulled from that queue? Does it have it’s own clock based on the audio thread sample rate or ?

Will study the examples some more. Thank you for the info.

12

I’ve used my own control inheriting from IBitmapMeterControl, the important part is that I use an IBufferSender in my ProcessBlock method to call it’s ProcessBlock method to capture the data, and then call that Sender’s TransmitData method in my plug-in’s OnIdle method. The IBufferSender’s ProcessBlock method identifies the control it’s going to eventually update by tag ID, so you need to make sure your meter/display elements each have a unique tag defined. Just remember to queue up what you expect your display to use. In my case it’s the entire input or output block. I think OnIdle gets called more frequently than visual persistence of vision, so it appears pretty smooth to me. Faster than the eye can follow at any rate.

I made a screen recording of what the meter movement looks like this way, but .mov isn’t an allowed extension to attach here. Sorry.

There IS an example of this in the samples, I just don’t recall which one.

1 Like
13

I’ve started building a SpectrumGraphControl using IVPlotControl as a template. It has turned out to be easier than I expected with most of the code needed simply to format the data to create the desired plot (thank you @AlexHarker for the helpful code posted above).

One thing I don’t understand is how vector “Draw” items are removed from the UI, i.e., so that it doesn’t draw new curves on top of previous ones in an ever growing pile. It appears from reviewing other controls that the control is wiped clean by simply redrawing the control’s background as part of the Draw() function - however the default color for backgrounds is “transparent” - so how does that work? Any clarification on that greatly appreciated!

So I still have much more to learn and do here. Thank you for the info.

14

@rexmartin - Whenever your control gets called to draw then you need to redraw everything.

Controls behind your control will have already been called to do their drawing (and those on top will get called after).

Therefore at the time you are called to draw any plugin background will be there, and so you can either draw a specific bounding box/background for your control (as might be common for a graph type control) or just draw onto of whatever is already there (e.g. a curve onto of the plug background and controls deeper in the stack.

1 Like
15

OK, so I did understand it. We “erase” by drawing over with new stuff (control background, etc.). Now that you’ve explained that I recognize that this is how the “Paint” program in Windows works. The “eraser” in Paint draws over whatever you are “erasing” with the background color (it doesn’t actually “erase” anything).

Great. Thank you for clarifying. :+1:

16

I thought graphics only updated during idle time between Process calls (usually making them appear somewhat “lurchy” in response time). In the case of an FFT driving a spectrum display I would expect the display to update only once per ProcessBlock and, therefore, appear a little “freeze frame-ish”

Let’s look at this a bit…and choose a worst-case scenario. If we use a 1024-sample block size at a low sample rate of 44100 - each block is occurring every 23 milliseconds and change (1024/44100). That’s a pretty short period of time for the human eye to gather and process visual info, and approaches 44 frames per second. Certainly reaction time is much longer than that…nearly an order of magnitude longer. If something is looking “freeze frame-ish” I don’t think it’s because of the length of a processed block. 32-sample block size at 96kHz is occurring every 0.3 milliseconds (over 3300 frames per second), to look at something on the other end of the scale. 24 frames per second is used in film for seamless motion, and I don’t think people complain that Gone With The Wind looked freeze-framed.

We got 99 problems, but the process block ain’t one. :grin:

1 Like
17

Yep - you’re right. The problem I was having was due to a bad buffer I used between the ProcessBlock and the FFT. I was waiting for the large FFT (8192 at 44.1kHz) to fill before running and updating the UI - and creating discontinuities and time lag in the process. The right way is to update a ring buffer with the new Process data, run the FFT and update the UI on every OnIdle() call, overlapping where it may. It updates so fast now that I was able to add a frame-to-frame smoothing filter which smooths it down nicely. Next step spline interpolation between bins…

18

I would not suggest taking an FFT every OnIdle();

The way I’d do this would be to use a ring buffer and push to a queue once every N samples (the hop size). This might typically be once every (FFT_size / 2), but it can be whatever you want. That allows you to explicitly control the speed of the update.

I put my time smoothing in the DSP side code (allowing you customise it separately from the UI).

If you’re using the code I gave that includes cubic interpolation for the low bins. It’s not worth doing it for higher bins as you’ll actually end up with more than one bin per pixel. That is likely to happen pretty quickly unless your FFT size is quite low. The code I posted also tries to optimise the drawing of that to prevent complex stroke paths that aren’t worthwhile.

19

Agree and now, after learning by doing, I am not. I’m computing the display FFT once per ProcessBlock - and only the first time OnIdle() is called. If OnIdle() is called repeatedly between ProcessBlocks the FFT only runs the first time. I’m loading the FFT from a ring buffer (of size FFT) that’s updated in ProcessBlock. So no matter when the display FFT runs it’s always working on a snapshot of the latest audio data. It’s working well and with very low CPU load.

Does that mean you are running the display FFT in the ProcessBlock (outside the for loop) - or are you transferring the FFT data back and forth between the UI side and the DSP side?

I’m running everything but the ring buffers on the UI side. As you point out, that does not provide a good place for frame-to-frame smoothing because the time periods are random. However I’m using a very slow filter with some scaling to nFrames and samplerate which is good enough for my needs.

I had a similar interpolation code I was using for audio that I used here. And yes, I only run it on the low bins. I’m running a large FFT so I’m only interpolating 3 points in between actual data points then using a simple DrawLine routine to “connect the dots”. Looks smooth enough for my needs.

Curious how some plugins smooth their spectrum display by “1/2, 1/6, etc. Octave”. It’s a spline filter of some sort that is variable and may be separate from the display interpolation, IDK. Will save that for another day!

20

Hi @AlexHarker,
i’m fiddling with your SpectrumPlotter class going to implement my own analyzer. Thanks again for sharing the code!
However there’s an undeclared function XCalc(), maybe part of your Plotter class? Could you tell what it’s actually calculating, or share that code as well?