How to do DFT for convolution of Audiosignals

If you have a simple question and just want an answer.
DSPguy
Junior Member
Posts: 7
Joined: Sat Mar 04, 2017 3:47 pm

How to do DFT for convolution of Audiosignals

Post by DSPguy »

Hello everybody!
I'm a totaly new to the XCore Processors in der terms of writing firmware for them. At the moment I got a StartKit with the AudioSlice to learn how to write code for it. My goal is to write a programm which convolutes an audiostream with a given impulse response with a lenght of 2048 to 9600 samples lenght at a samplerate of 48000Hz.

To get started I downloaded the AN00201 Audio Effects demo for the StartKit and tried to understand it. I now have removed all the unneccesary parts (e.g. the wavegen, biquad filters etc.) so that it is basically just passes the incomming Samples from the input samplebuffer to the outgoing samplebuffer without doing anything with the samples.
Now I need some help on how to do an FFT on the input samples to come a step closer to a convolution with a given impulse response. I simply cannot find a simple(!) example for an FFT with non-complex data as I would need. It would be really nice if someone could give me some hints on that topic!

Thanks in advance!


User avatar
andrew
Experienced Member
Posts: 114
Joined: Fri Dec 11, 2009 10:22 am

Post by andrew »

I recommend using the over-lap-and-add algorithm: https://en.wikipedia.org/wiki/Overlap%E ... add_method

Do you want to process mono or stereo? If it's stereo then and FFT can be used efficiently by packing one channel into the real inputs and the other channel into the imaginary inputs. after a dsp_fft_bit_reverse() and a dsp_fft_forward(); you can reconstruct the two independent frequency domain representations with: dsp_fft_split_spectrum().

Hope that helps
User avatar
Thomas
Experienced Member
Posts: 66
Joined: Fri Feb 05, 2010 12:34 pm

Post by Thomas »

Hello
You find an application example for FFT and inverse FFT in the xCORE-200 DSP Elements Library [AN00209]
This is accessible directly from xTIMEcomposer. Just click on Examples and then enter "DSP" in the search field.

But the DSP library is mainly supported for xCORE-200.
For audio development we recommend the xCORE-200 Multichannel Audio Platform:
https://www.xmos.com/products/audio/xco ... o-platform

Best Regards, Thomas
DSPguy
Junior Member
Posts: 7
Joined: Sat Mar 04, 2017 3:47 pm

Post by DSPguy »

Hey Guys,
thanks for your help! I've now taken a look at the over-lap-and-add algorithm because it seems to be the right choice for my project.
I've now started to implement it but therefor I first need some basic stuff. As far as I understand it I need to collect a small ammount of samples to work with them so I tried to write a small programm which simply collects 20 Samples in a buffer array and then if this array is filled up passes these samples to a second buffer which is read by the I2S routine when it needs to send out a sample. So far so good but it seems like this does not work properly when I go up with the number of buffered samples (30 samples is already to much) and it sound terrible (some kind of timing problem I think). I think that may is a result of the needed time for copying the buffers so that the i2s routine already needs a new sample when they are not ready.
But if it takes already to much time to buffer 30 samples then how can the over-lap-add algorithm made possible on the processor?
Do you have any suggestions what I am doing wrong or what I could optimize in my code? My first thought is to do more stuff in parallel but I don't know how exactly I should start with this idea.
It would be great if you could give me some advice how to go on from here! Thanks!


This is my code based on the AN00201

Code: Select all

#include <xs1.h>
#include <platform.h>
#include <startkit_gpio.h>
#include <i2s.h>
#include <i2c.h>
#include <gpio.h>
#include <cs4270.h>

/* Ports and clocks used by the application */
startkit_gpio_ports gpio_ports =
on tile[0] : {XS1_PORT_32A, XS1_PORT_4A, XS1_PORT_4B, XS1_CLKBLK_3};

out buffered port:32 p_dout[1] = on tile[0]: {XS1_PORT_1G};

in buffered port:32 p_din[1]   = on tile[0]: {XS1_PORT_1K};

in port p_mclk                 = on tile[0]: XS1_PORT_1E;
out buffered port:32 p_bclk    = on tile[0]: XS1_PORT_1F;
out buffered port:32 p_lrclk   = on tile[0]: XS1_PORT_1I;

port p_i2c                     = on tile[0]: XS1_PORT_4D;

port p_gpio                    = on tile[0]: XS1_PORT_4C;

clock mclk                     = on tile[0]: XS1_CLKBLK_1;
clock bclk                     = on tile[0]: XS1_CLKBLK_2;

// Pin map for GPIO: clock select, codec reset are on pins 1 and 2 of 4C
static char gpio_pin_map[2] = {1, 2};

#define SAMPLE_FREQUENCY 48000
#define MASTER_CLOCK_FREQUENCY 24576000
#define CODEC_I2C_DEVICE_ADDR 0x48


[[distributable]]
void i2s_handler(server i2s_callback_if i2s,
                 client i2c_master_if i2c,
                 client output_gpio_if clock_select,
                 client output_gpio_if codec_reset
                 )
{
  int32_t in_samps[2] = {0,0};
  // Zwischenpuffer
  int32_t puffer[20] = {0};
  //Zwischenpuffer 2
  int32_t puffer2[20] = {0};
  //Zaehler
  size_t j  = 0;
  //Zaehler2
  size_t y  = 0;
  size_t flag  = 0;
  while (1) {
    select {
    case i2s.init(i2s_config_t &?i2s_config, tdm_config_t &?tdm_config):
      /* Set CODEC in reset */
      codec_reset.output(0);

      /* Set master clock select appropriately */
      if ((SAMPLE_FREQUENCY % 22050) == 0) {
        clock_select.output(0);
      }else {
        clock_select.output(1);
      }

      /* Hold in reset for 2ms while waiting for MCLK to stabilise */
      delay_milliseconds(2);
      /* CODEC out of reset */
      codec_reset.output(1);

      cs4270_configure(i2c, CODEC_I2C_DEVICE_ADDR,
                       SAMPLE_FREQUENCY, MASTER_CLOCK_FREQUENCY,
                       CODEC_IS_I2S_SLAVE);

      /* Configure the I2S bus */
      i2s_config.mode = I2S_MODE_I2S;
      i2s_config.mclk_bclk_ratio = (MASTER_CLOCK_FREQUENCY/SAMPLE_FREQUENCY)/64;
      break;

    case i2s.restart_check() -> i2s_restart_t restart:
      // This application never restarts the I2S bus
      restart = I2S_NO_RESTART;
      break;

    case i2s.receive(size_t index, int32_t sample):
      //Collect 20 samples if Channel 0, If 20 Samples are collected pass them to a second buffer,set Ready Flag and empty first buffer
      if (j < 20)
      {

          if(index==0)
          {
              puffer[j] = sample;
              j++;
          }
      }
      else
      {
           flag = 1;
           for (size_t y = 0; y < 20; y++)
           {
               puffer2[y] = puffer[y];
               puffer[y] = 0;
           }
           j = 0;
      }
      break;


    case i2s.send(size_t index) -> int32_t sample:
            //Check if Sample for channel 0 is requested and Ready Flag is set. Then send sample from buffer2.
            if(index==0 && flag == 1)
            {
                sample = puffer2[y];
                y++;
                if(y==20)
                {
                    y=0;
                }
            }
            else
            {
                sample = 0;
            }
      break; // end of select
    }
  }
};


int main(void)
{
  startkit_led_if i_led;
  startkit_button_if i_button;
  slider_if i_slider_x, i_slider_y;
  i2s_callback_if i_i2s;
  i2c_master_if i_i2c[1];
  output_gpio_if i_gpio[2];
  par {

    on tile[0]: i2c_master_single_port(i_i2c, 1, p_i2c, 10, 0, 1, 0);
    on tile[0]: output_gpio(i_gpio, 2, p_gpio, gpio_pin_map);
    on tile[0]: startkit_gpio_driver(i_led, i_button,
                                     i_slider_x,
                                     i_slider_y,
                                     gpio_ports);
    on tile[0]: {  configure_clock_src(mclk, p_mclk);
                   start_clock(mclk);
                   i2s_master(i_i2s, p_dout, 1, p_din, 1,
                              p_bclk, p_lrclk, bclk, mclk);
                }
    on tile[0]: i2s_handler(i_i2s, i_i2c[0], i_gpio[0], i_gpio[1]);

  }
  return 0;
}
User avatar
johned
XCore Addict
Posts: 185
Joined: Tue Mar 26, 2013 12:10 pm
Contact:

Post by johned »

Doing the buffering in i2s_handler you have the potential for breaking the i2s timing, which you can verify with a scope on the i2s signals.
I would do the buffering/unbuffering in a separate core.
Best regards,
John
DSPguy
Junior Member
Posts: 7
Joined: Sat Mar 04, 2017 3:47 pm

Post by DSPguy »

Ok then I will try to put the Samplebuffer out of the I2S Handler routine.
My solution to this would be to create a streaming channel between the buffering task and the I2S Handler Task. I will find out if this is a solution to my problem and get back to you.

Thanks for your help guys!
DSPguy
Junior Member
Posts: 7
Joined: Sat Mar 04, 2017 3:47 pm

Post by DSPguy »

Hey Guys!
I've now tried my best to seperate the function of "delaying" by 10 Samples on another thread.

Therefor I started again with AN00201 and removed the unnessacary stuff and replaced it with my routine. The application now only runs the I²C Task and communicates over a streaming channel with the audio_effects task. The incomming samples are treated exactly the same as in the previous example (collect 10 samples in array "puffer", when "puffer" is filled copy the array to "puffer2" and reset "puffer").

It seems like this solution is even worse because now I can already hear failures in the Audiostream by just delaying by 2 samples. I have no Idea what I am doing wrong here? Any ideas?

main function:

Code: Select all

[[distributable]]
void i2s_handler(server i2s_callback_if i2s,
                 client i2c_master_if i2c,
                 client output_gpio_if clock_select,
                 client output_gpio_if codec_reset,
                 streaming chanend c_dsp)
{
  int32_t in_samps[2] = {0};
  int32_t out_samps[2] = {0};
  while (1) {
    select {
    case i2s.init(i2s_config_t &?i2s_config, tdm_config_t &?tdm_config):
      /* Set CODEC in reset */
      codec_reset.output(0);

      /* Set master clock select appropriately */
      if ((SAMPLE_FREQUENCY % 22050) == 0) {
        clock_select.output(0);
      }else {
        clock_select.output(1);
      }

      /* Hold in reset for 2ms while waiting for MCLK to stabilise */
      delay_milliseconds(2);

      /* CODEC out of reset */
      codec_reset.output(1);

      cs4270_configure(i2c, CODEC_I2C_DEVICE_ADDR,
                       SAMPLE_FREQUENCY, MASTER_CLOCK_FREQUENCY,
                       CODEC_IS_I2S_SLAVE);

      /* Configure the I2S bus */
      i2s_config.mode = I2S_MODE_I2S;
      i2s_config.mclk_bclk_ratio = (MASTER_CLOCK_FREQUENCY/SAMPLE_FREQUENCY)/64;
      break;

    case i2s.restart_check() -> i2s_restart_t restart:
      // This application never restarts the I2S bus
      restart = I2S_NO_RESTART;
      break;

    case i2s.receive(size_t index, int32_t sample):
      if (index == 0) {
        for (size_t i = 0; i < 2; i++) {
          c_dsp <: in_samps[i];
          c_dsp :> out_samps[i];
        }
      }
      in_samps[index] = sample;
      break;

    case i2s.send(size_t index) -> int32_t sample:
      sample = out_samps[index];
      break; // end of select
    }
  }
};


int main(void)
{
  streaming chan c_aud_dsp;
  chan c_gain;
  startkit_led_if i_led;
  startkit_button_if i_button;
  slider_if i_slider_x, i_slider_y;
  i2s_callback_if i_i2s;
  i2c_master_if i_i2c[1];
  output_gpio_if i_gpio[2];
  par {
    /* These four tasks are library functions that drive the hardware
       peripherals */
    on tile[0]: i2c_master_single_port(i_i2c, 1, p_i2c, 10, 0, 1, 0);
    on tile[0]: output_gpio(i_gpio, 2, p_gpio, gpio_pin_map);
    on tile[0]: startkit_gpio_driver(i_led, i_button,
                                     i_slider_x,
                                     i_slider_y,
                                     gpio_ports);
    on tile[0]: {  configure_clock_src(mclk, p_mclk);
                   start_clock(mclk);
                   i2s_master(i_i2s, p_dout, 1, p_din, 1,
                              p_bclk, p_lrclk, bclk, mclk);
                }

    /* These three tasks are defined in this application */
    on tile[0]: i2s_handler(i_i2s, i_i2c[0], i_gpio[0], i_gpio[1],
                            c_aud_dsp);
    on tile[0]: audio_effects(c_aud_dsp, i_led, i_button, c_gain, 2);
  }
  return 0;
}

Audio Effects Function:

Code: Select all

void audio_effects(streaming chanend c_dsp,
                   client startkit_led_if i_led,
                   client startkit_button_if i_button,
                   chanend c_gain,
                   static const size_t num_chans)
{
  // Unprocessed input audio sample buffer
  int32_t in_samps[num_chans] = {0};

  // Sample buffer to hold samples after processing
  int32_t out_samps[num_chans] = {0};

  // Zwischenpuffer
    int32_t puffer[10] = {0};
    //Zwischenpuffer 2
    int32_t puffer2[10] = {0};
    //Zaehler
    size_t j  = 0;
    //Zaehler2
    size_t y  = 0;
    size_t flag  = 0;

  while(1) {
    // Send/Receive samples
    for (size_t i = 0; i < num_chans; i++) {
      c_dsp :> in_samps[i];
      c_dsp <: out_samps[i];
    }


        if (j < 9)
        {
        puffer[j] = in_samps[0];
        j++;
        }
        else
        {
            j=0;
            flag = 1;
            for (size_t y = 0; y < 9; y++)
            {
                puffer2[y] = puffer[y];
                puffer[y] = 0;
            }
        }

        if (flag == 1)
        {
            out_samps[0] = puffer2[y];
            out_samps[1] = puffer2[y];
            y++;
            if(y==10)
            {
              y=0;
            }
        }

  }
}
DSPguy
Junior Member
Posts: 7
Joined: Sat Mar 04, 2017 3:47 pm

Post by DSPguy »

Can nobody help me with this problem? I can't find the issue...
User avatar
andrew
Experienced Member
Posts: 114
Joined: Fri Dec 11, 2009 10:22 am

Post by andrew »

If you simply want to delay the sample stream by 10 samples then here is a drop in replacement for the audio_effects() function you have written.

Code: Select all

void audio_effects(streaming chanend c_dsp,
                   client startkit_led_if i_led,
                   client startkit_button_if i_button,
                   chanend c_gain,
                   static const size_t num_chans)
{
    //Zaehler
    unsigned index = 0;
    #define PUFFER_LENGTH 10
    //puffer
    int puffer[PUFFER_LENGTH] = {0};
    
    while(1){
        int in_sample;
        c_dsp :> in_sample;
        int out_sample = puffer[index]
        puffer[index] = in_sample;
        index++;
        if(index == PUFFER_LENGTH)
            index = 0;
        c_dsp <: out_sample;
    }
}
Let me know if that doesn't do what you expect.
User avatar
andrew
Experienced Member
Posts: 114
Joined: Fri Dec 11, 2009 10:22 am

Post by andrew »

Sorry, i can see how this is not the same. If you describe what you want the audio effect to do then I can probably help some more.
Post Reply