Getting started with Audio: USB <-> DSP <-> many-channel analog

[AaronD]

After several attempts to use a Raspberry Pi Pico with their own SDK, and then a PJRC Teensy 4.1 with the Arduino framework, both of which failed at the USB Audio stage for different reasons in addition to the Pi Pico not having enough "juice" anyway, I ended up with an XK-AUDIO-316-MC-AB demo board as depicted here:
https://www.xmos.com/develop/usb-multichannel-audio/

I downloaded all of the resources I could find for it, and found a wonderful getting started tutorial in 2. INSTALL TOOLS -> Linux 64-bit 15.2.1 -> XMOS/XTC/15.2.1/doc/html/index.html. That got me to make some really simple programs to prove that everything works in the first place (it does), and show how multithreaded communications work. And it appears from that, that XMOS has done an amazing job of making a non-standard architecture look and feel as close to standard as possible while still keeping the special features that necessarily make it non-standard, and a wonderful job as well, of documenting all of that to make it easy to understand.

Great! Now to look at using it for audio...and that's where things fall apart for me.

There's an app note - AN01008: Extending USB Audio with Digital Signal Processing - linked from the page above, that makes it seem like there's a complete, working, configurable project that connects everything on the dev board to a many-channel USB sound card, plus a "put your code here" section that can be left alone for no processing at all, or filled up with DSP work. Again, great! But all I can find is the USB Audio Software project and the corresponding USB Audio User Guide, also linked from the same page. That is much harder to understand than the Getting Started examples and explanation! Page 2 of AN01008 has a link to http://github.com/xmos/sw_usb_audio.git, which appears to be that same project again, which still doesn't match what I understand from the rest of the app note.

The USB Audio User Guide says on page 40 that I may have to disable some physical I/O to free up a core for DSP, which is fine because I'm only interested in the analog converters anyway. But it's not quite what AN01008 seems to say, about an entirely single-threaded (Figure 2 on page 6) and entirely functional starting project, with the "your code here" section for DSP as an overridden void UserBufferManagement(unsigned[], unsigned[]); Being entirely single-threaded and functional to start with, would of course leave all of the other cores and tiles free for my own functionality, which would be awesome!

Did I misunderstand something?

[Ross]

that makes it seem like there's a complete, working, configurable project that connects everything on the dev board to a many-channel USB sound card, plus a "put your code here" section that can be left alone for no processing at all

That sounds about right :)

The USB Audio project uses many threads (configurable based on what features you enable). In the xcore world "everything" is software - so, for example, the S/PDIF transmit interface is implemented in software running in a (hardware) thread. So, by disabling hardware interfaces you are essentially freeing up threads and MIPs for more DSP processing.

You can put some (very) simple DSP in the UserBufferManagement(unsigned[], unsigned[]); but most projects offload samples to other processing threads at this point - typically buffering before before performing some sort of block-based processing.

It's becoming increasingly popular to integrate DSP into the design so we are considering adding this blocking into the design proper.

You might like to look at Integrating Audio Weaver (AWE) Core into USB Audio for an example of how we integrated some substantial DSP (and control of said DSP) into the design.

Hope that's helpful

[Ross]

From AN01008:

As XCORE is a concurrent multi-threaded multi-core processor, there are other threads and cores available for DSP. It depends on the precise configuration of the USB stack (whether you use special interfaces such as S/PDIF, ADAT, MIDI) but in a simple case with just I2S, USB Audio uses around 30% of the compute, with one tile being completely empty.

This could be made more clear I guess! By "compute" it means threads.

[AaronD]

The USB Audio project uses many threads (configurable based on what features you enable). In the xcore world "everything" is software - so, for example, the S/PDIF transmit interface is implemented in software running in a (hardware) thread. So, by disabling hardware interfaces you are essentially freeing up threads and MIPs for more DSP processing.

From AN01008:

As XCORE is a concurrent multi-threaded multi-core processor, there are other threads and cores available for DSP. It depends on the precise configuration of the USB stack (whether you use special interfaces such as S/PDIF, ADAT, MIDI) but in a simple case with just I2S, USB Audio uses around 30% of the compute, with one tile being completely empty.

This could be made more clear I guess! By "compute" it means threads.

So Figure 2 on page 6 of AN01008 is *not* the entire project, but only the USB part? And every I/O format is its own (hardware) thread that runs in parallel with that, and is never shown or mentioned at all? That could have been clearer too.
.

You can put some (very) simple DSP in the UserBufferManagement(unsigned[], unsigned[]); but most projects offload samples to other processing threads at this point - typically buffering before before performing some sort of block-based processing.

I was indeed planning to offload the DSP work to other threads. Keep the USB thread as-is, and keep the I2S/TDM thread(s) and related chip management. Delete everything else. Eventually, I'm going to spin my own board that only has that, once I figure out what I'm doing. :-)
Reserve one thread for communication with an external "system coordinator" (DSP coefficients, level meters, etc.), and the rest are all available for DSP.

A big part of why I ended up with XMOS at all - instead of, say, an Analog Devices dedicated DSP chip with their graphical signal-flow IDE - was the specific *absence* of buffering or block processing. I have a project in mind that essentially amounts to a NON-isolating hearing aid (among lots of other things while I'm at it), and I don't want to introduce an acoustic comb filter if I can avoid it. So, I'm looking at a higher-than-usual sample rate, just so the converters can use a less-aggressive filter with fewer samples of latency *in addition to* less time between samples, and I want the entire DSP chain to only have one sample in it at a time, per channel. One sample all the way through, then next sample all the way through, etc.

Yes, it's less efficient that way, so I'll have to see how much I can do, but that's the idea.

I'm almost resigned to writing my own DSP functions as well, which will likely reduce the efficiency even more, since every library I've seen so far has been either restrictively licensed or hard-coded for block processing. If you know of one that satisfies those requirements (or if XMOS actually *has* one that I haven't seen yet), I'd be interested in that too.

Thanks!

[AaronD]

I'm almost resigned to writing my own DSP functions as well, which will likely reduce the efficiency even more, since every library I've seen so far has been either restrictively licensed or hard-coded for block processing. If you know of one that satisfies those requirements (or if XMOS actually *has* one that I haven't seen yet), I'd be interested in that too.

Okay, I've looked through the DSP module now. The license looks good, and it does indeed process one sample at a time, not a buffer. And it appears to be optimized already for that job on this chip. So that's good!
I'll still need to make a compressor/limiter, and one instance of it (for hearing protection) might need a "1/2-order" or "inverse pink noise" filter (+3dB/oct, +10dB/dec) in its side-chain to make it respond better to high frequencies, but at least the general EQ and HP/LP are already there.

And I've successfully reduced the example project to just a USB <-> 8x8 Analog sound card with nothing else...or at least I *think* I have. The executable is smaller, anyway, and it still works. The compiler complained too, when I missed a reference to remove that still wanted something that I'd deleted. Is there a good way to tell for sure that I've found everything to remove? Maybe a tool somewhere to see what the #defines actually are, parse the #ifdefs, and remove them along with the dead code that they create? I only need the one configuration.

Also, is there a way to know how many cores a project actually uses, and what each one is doing? I have this at the end of the build log:

Code: Select all

Creating app_usb_aud_xk_316_mc.xe
Constraint check for tile[0]:
  Memory available:       524288,   used:      63032 .  OKAY
    (Stack: 3228, Code: 34168, Data: 25636)
Constraints checks PASSED.
Constraint check for tile[1]:
  Memory available:       524288,   used:      13676 .  OKAY
    (Stack: 828, Code: 11256, Data: 1592)
Constraints checks PASSED.
Build Complete

which is good as far as it goes, but I also want to know how many cores I'm using within each tile and what they're doing, to compare with how many I have and thus see what's available for DSP and system management.

Maybe I just need to pare it down some more so I can read directly what's going on, but I still have a long way to go if that's the case. A tool to analyze it automatically and tell me that I'm already there (or not as the case may be) would be greatly appreciated!
Or a different example project that comes as a series of disconnected modules that are built up, instead of an integrated one that does everything and needs to be untangled. I could go for that too. With documentation of course, for what each module requires or can share with others.

Thanks!

[Ross]

The define based "code-less" model is is for people that don't really want to make serious modifications - there was some desire to slap a GUI in front of the define generation at one point.

You can make your own program using the component parts rather than starting from the complex main(). There's are example here: https://github.com/xmos/lib_xua/tree/de ... ua_example.
See also the documentation.

The incomplete report is likely due to including lib_xcore as one of the dependencies, the tools guys are still working on sorting this out. In the meantime, if you're not using the dependency, you could remove it.

[AaronD]

I finally got back to this! And will likely have more delays as other things keep getting in the way. Anyway:

The define based "code-less" model is is for people that don't really want to make serious modifications - there was some desire to slap a GUI in front of the define generation at one point.

That would be nice! But I suspect I'm going to be code-heavy enough that the "code-less" part is moot. Though it would still be a wonderful place to start for those that, say, came from a different build system, or have always used an IDE with a Build button that "just works", or always called the compiler directly with no build system at all, or whatever the case may be, and want to see that it's well-designed enough to not have to fight a foreign system (to them) just to verify the functionality in the first place. I've had way too many of those experiences already, with other projects. Much better to have it "just work", right out of the box, and *then* dig into it to see how. It seems to me that you've done a LOT of effort in that direction, and that you're *almost* there, but you're still just a little bit short.

The complex example is maybe just one step short - Makefiles exist to just blindly `make` with and watch it succeed and run with no understanding whatsoever of the project or of `make` itself, but it does kinda need a GUI like you mentioned - but the component one seems farther away.

You can make your own program using the component parts rather than starting from the complex main(). There's are example here: https://github.com/xmos/lib_xua/tree/de ... ua_example.
See also the documentation.

I'm not entirely sure which documentation you're referring to there. There are docs scattered all around the projects themselves, and links to relevant app notes in the various download sections of the website. Is it one of those? Something else?

And like I alluded to above, the component project does not "just work" for me. AN00246 (the one you directly linked to) appears to be a complete project that only does 2-channel from USB host to I2S DAC and nothing else by default, but still has #defines to do a lot more, while keeping the main() short and readable. Great! But it doesn't build.

No Makefile, and cmake fails too. From the tutorial that came from here:
https://www.xmos.com/develop/usb-multichannel-audio/ -> (2. INSTALL TOOLS) -> Tools-15---Linux-64_15_2_1/XMOS/XTC/15.2.1/doc/html/index.html -> Tools Guide -> Installation and configuration -> Configuring the command-line environment
(which leads to .../Tools-15---Linux-64_15_2_1/XMOS/XTC/15.2.1/doc/html/tools-guide/install-configure/getting-started.html)
I figured out to put a script in the project directory that does this:

Code: Select all

#!/bin/bash



# usage: source <this file>

# don't just call it; the return wipes out its effect



PREVPATH=`pwd`
cd /home/aaron/Downloads/XMOS/2024-06-04/1_EVALUATION/2_INSTALL_TOOLS/Tools-15---Linux-64_15_2_1/XMOS/XTC/15.2.1
source SetEnv

cd $PREVPATH

So then I can do everything from the project directory...or at least that's the idea.

After `cd`ing to the project directory and `source`ing that script, the complex project `make`s with a couple of warnings, no errors, and runs just fine. But when I do that with the component project, `make` fails because there's no Makefile, and `cmake` does this:

Code: Select all

aaron@aaron-ubuntustudio-m6800:~$ cd /home/aaron/Downloads/XMOS/Simple-Audio-Project/1_Build-Original/lib_xua-develop/examples/AN00246_xua_example/
aaron@aaron-ubuntustudio-m6800:~/Downloads/XMOS/Simple-Audio-Project/1_Build-Original/lib_xua-develop/examples/AN00246_xua_example$ source ./0_setup.sh 
aaron@aaron-ubuntustudio-m6800:~/Downloads/XMOS/Simple-Audio-Project/1_Build-Original/lib_xua-develop/examples/AN00246_xua_example$ cmake .
CMake Error at CMakeLists.txt:2 (include):
  include could not find requested file:

    /xcommon.cmake


-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:23 (XMOS_REGISTER_APP):
  Unknown CMake command "XMOS_REGISTER_APP".


-- Configuring incomplete, errors occurred!
See also "/home/aaron/Downloads/XMOS/Simple-Audio-Project/1_Build-Original/lib_xua-develop/examples/AN00246_xua_example/CMakeFiles/CMakeOutput.log".
aaron@aaron-ubuntustudio-m6800:~/Downloads/XMOS/Simple-Audio-Project/1_Build-Original/lib_xua-develop/examples/AN00246_xua_example$ 

CMakeOutput.log is 47 KiB! Not posting all of that here!

[Ross]

Heya,

I linked you to the head of develop branch, my bad. We're just in the middle of moving examples to the "new" cmake based build system and deleting the old makefiles since this thread started.

The new system requires XTC 15.3 (it includes the xcommon.cmake file cmake is complaining about).

you can use the lib_xua v4.2.0 at the minute, heres a better link to that example with the old Makefile in tact: https://github.com/xmos/lib_xua/tree/56 ... ua_example

Or you can use the head of the repo and use XTC 15.3 with the new build system

We're in a bit or a transitionary period at the moment. The new release will be available imminently (days).

The documentation I was referencing was https://github.com/xmos/lib_xua/release ... ib_xua.pdf, specifically section 6. Again, a new release of this doc will be up on GitHub and xmos.com very soon.

[AaronD]

That would explain things! Thanks for clearing it up. Probably best to wait until it settles, catch up on the other things, and then see what it does.

Thank you!