No simultaneous in & out USB audio with lib_xua

[Jash]

Hello,

I have a custom board design using a XU316-1024-QF60B-C32, connected to a DAC for one speaker and 8 PDM microphones placed on two distinct squares (so I remap them in user_pdm_process() to expose only 4 of them in USB endpoints). I use XUA library for UAC2.0 with 48kHz sample rate, 32 bits resolution, no USB DFU.

With a firmware using only microphones, no problem getting it functional, with NUM_USB_CHAN_IN=4, NUM_USB_CHAN_OUT=0, XUA_NUM_PDM_MICS=8 and I2S_CHANS_DAC=0.

With a firmware using only the DAC, no problem either, with NUM_USB_CHAN_IN=0 and NUM_USB_CHAN_OUT=1, XUA_NUM_PDM_MICS=0 and I2S_CHANS_DAC=2.

When I merge these two firmwares to use simultaneously microphones and DAC, that's where I am struggling: I get nothing when I try to record what the microphones hear (time cursor is stuck to 0 in Audacity), and I cannot manage to output something to the speaker either. I can record the output loopback without problem though.
Same result when changing core priority or thread speed.
With an oscilloscope, I can see the output signal of the microphones, master clock and BCK are still present too, but no LRCK signal to the DAC.
Connecting my board with this firmware to a Linux computer gives me error code -71, Windows does not output any error.

Using the microphones-only firmware with I2S_CHANS_DAC=2 instead of 0 (with still NUM_USB_CHAN_OUT=0), it remains functional.

Using the DAC-only firmware with XUA_NUM_PDM_MICS=8 instead of 0 (with still NUM_USB_CHAN_IN=0), I get the same behavior as with the merged firmware.

I precise that each time I change the XUA defines, I adapt the code to maintain the overall coherence (USB endpoints, arguments of XUA_AudioHub(), etc).

I tested and got the same results with lib_xua version 5.0.0 (branch master) and version 5.0.1 (branch develop).
I also tested with branch feature/mic_array_v5_sr_change (with additions of #ifndef - #endif structures around macros in lib_xua/api/xua_pdm_mics.h to avoid conflicts with my custom design defines), pointed out by issue #454, no change either.

Do you think of any advice that could help me get the firmware with simultaneous in & out working?

EDIT: After debugging with xgdb, I noticed that the core running pdm_rx_isr() locks up while the core running ma_frame_rx() in the AudioHub loop also locks up at the same time. This behavior seems to be related to issue #230 of Mic Array library, however I don't really understand why this happens only when I have simultaneous in & out audio channels and not when I have in channels only. I also precise that these two cores run on two distinct tiles.
By any chance, do you have some tips to get around this problem?

Many thanks in advance

[andrewxcav]

The issue could be on the host. See here for some troubleshooting steps:

https://www.xmos.com/documentation/XM-0 ... io-signals (basically your computer is seeing the same VID/PID come up over USB with different profiles and gets confused).

Note that the stalling while gdb debugging *might* have been caused by GDB breaking some timing with the host, but I wouldn't guarantee that is what is happening.

[Jash]

Thank you for your answer.

I cleared the cached device data with USBDeview as said in the troubleshooting link you sent, but the problem still remains.

By the way, everytime I change the firmware of my board, I go to device manager and uninstall the device, then unplug-replug it to make Windows consider it as a new device.
I assume this amounts to the same thing as removing the cached device data with USBDeview.

For the code debugging, do you know a solution to determine what is happening without breaking the timings?

[andrewxcav]

One method of "lightweight" debug would be to use xscope to maybe monitor one channel in each direction:

https://www.xmos.com/documentation/XM-0 ... index.html

[Jash]

Thank you again for your reply, I will test further with xscope probes.

In the meantime, I tested with a simple xrun of my app to avoid potential xgdb timing issues, and it terminates with:

Code: Select all

Thread 1.1 hit Catchpoint -1 (XCore Exception ET_ILLEGAL_RESOURCE), 0x00080aec in __xcore_chanend_out_ct (__c=2147614978, __ct=1 '\001') at C:\Program Files\XMOS\XTC\15.3.0\target/include\xcore/_support\xcore_chanend_impl.h:91
91          asm volatile("outct res[%0], %1" :: "r" (__c), "r" (__ct));
[Switching to 2.1 (tile[1] core[0])]

Thread 1.1 is the one running mic_array_task() (which calls pdm_rx_isr()), thread 2.1 is the one running XUA_AudioHub() (which calls ma_frame_rx()).
This again lets me think that it could be related to issue #230 of Mic Array library, does that sound coherent to you?

By the way, I noticed that mics.PdmRx.AssertOnDroppedBlock() was set to false in ma_task() (in lib_xua/lib_xua/src/core/pdm_mics/mic_array.cpp), but when I set it to true, xrun terminates with:

Code: Select all

Thread 1.1 hit Catchpoint -1 (XCore Exception ET_ECALL), 0x00080a52 in pdm_rx_isr () at ../../lib_mic_array/lib_mic_array/src/pdm_rx_isr.S:81
81          not D, A  // if the missed blocks counter is set to -1 (default)
[Switching to 2.1 (tile[1] core[0])]

Does it give you any idea of what is happening?
Anyway, I will come soon with some more data using xscope.

[infiniteimprobability]

Hi, this could be related to the following PR - https://github.com/xmos/lib_xua/pull/460

The standard board that this is tested on only has 2 mics so XUA does work out of the box. 8 mics is 4x the workload, and at this level, it makes sense to separate the port operations from the DSP operations of the decimator. A typical maximum of a single-threaded mic_array instance is 4 mics, which we do on some of our voice products.

FYI, we have had 16 channels running https://github.com/xmos/sln_voice/tree/ ... aggregator running but actually needed a total of 4 threads to do that reliably - see https://github.com/xmos/sln_voice/blob/ ... nfig.h#L16. But it shows that 8 should be well within the capability of the chip.

Hope this helps get you up and running.

[Jash]

Hi, I discovered this PR just a few hours ago, great timing ahah
I tested with it: no more lock up with xgdb, no more interruption with xrun either, but same behavior for audio inputs with Audacity as I mentioned in my first message (time cursor stays at zero). However, now I can output sound to speaker!

With some printf in user_pdm_process(), I see that mic datas are non zero and non constant, so I assume the PDM processes run fine, but maybe the AudioHub doesn't transfer the data to the USB stacks... I need to further investigate on AudioHub then

Thanks for the mic aggregator example, I will take a look at how it is done here.

Thank you both again for your time