Getting started is needlessly painful (rant)

[andy-aic]

Hi,

I've spent a good part of a week trying to understand the xcore.ai platform and get a project up and running. This write-up is from the point of view of someone that wants to evaluate the platform to run a ML model for an audio application.

As a newcomer, my first point of contact was your website. I have an xcore.ai evaluation kit, so I first looked at the xcore.ai page. I see some high-level features and I scroll down to find a list of random part numbers with links to their respective datasheets. At this point I do not know what part number is in the evaluation kit, so I try to find that out next. I'm interested in building an audio application, and I see "Audio Processing" so I click on View more. It takes me to a page of another audio evaluation kit which I don't have, so I go back to the previous page. I see "Build your own solution" and it takes me to this page where I can see the dev kit I have. Cool. So I'm looking for a getting started guide, a datasheet or something. The listed documentation has links to the XS3 Architecture ISA, some port map and an xcore-iot programming guide. Not what I'm looking for. I scroll down and I see more software documentation but not what I'm looking for.

So I decide to Google xk-evk-xu316, which is the part number shown in the box of my eval kit. The first link Google gives me looks promising, so I click on it: https://www.xmos.com/download/xcore.ai- ... ual(5).pdf
I find out that this is some other revision of the eval kit, because it doesn't look like the one I have. So I go back to your website. I find this page and I see my dev kit. I click on Quick Start and I see a very short PDF that tells me to go download the tools and shows me how to compile a single C file using the compiler directly. Okay... I close the PDF and continue looking for a reference manual or something to at least get to know the hardware I'm working with.

I find the right website for my dev kit and I download the hardware manual. This looks like what I'm looking for. In the introduction, the PDF says there's a quick start guide in xmos.ai/getting-started, but that page brings me here for some reason. Anyway, I continue reading and I find out that the processor on the board is the XU316-1024-FB265-C32. Cool, so now I can look it up on the table and find the right datasheet.

After getting to know the hardware, I was finally ready to get started with the SDK. I go to check the repository, read the README and I click on the https://www.xmos.com/documentation/XM-014660-PC-2/html/ link. I go through it and waste a bunch of time before I realize that I was reading some outdated docs all these time after I found the right docs on the website. The first thing I read is that I need to install the tools, so I download the tools for Linux (my OS), and I get the archive. I go to the documentation and start reading. I install everything and go through the first setup.

I'm now on the quick start, which again shows the same one-file hello-world example that gets compiled using xcc directly. I had already noticed in the SDK that it used CMake on their examples, so I just skip that since compiling by directly calling the compiler is rarely what you do. I continue reading and I see here that you need a wrapper xc file to run code on multiple tiles. At this point I had already read the hardware docs, so I'm familiar with the multiple tiles per device, multiple cores per tile architecture.

I finished reading the quick start of the tools, so I go back to the quick start of the SDK. Pretty much it tells you to go back to the tools' documentation to continue reading the architecture and programming guides, so I do. I see another quick start so I keep reading. There are some code examples that again show you some basics of multicore programming on xcore, and it shows an example of how to compile code with xcc directly.

I finish reading and I go back to the SDK docs. At this point I'm in the "advanced tutorials" section, checking the audio mux example for FreeRTOS. It basically tells you to run two commands to compile the example and that's it. At this point I've spent two days reading documentation. All the tutorials are the same, so I continue with the RTOS programming guide. Again, I spend the whole day reading more documentation, only to realize that there were no tutorials at all in the tutorials section.

I continue reading the build system user guide and I learn not much from it.

At this point I'm fed up of reading so I turn to the xmos github and I see a bunch of repos. I found ai tools and continue reading docs. I find some examples, but no cmake files. Only makefiles. Weird.
I check the lib_xua repo and find more examples. No cmake files either.

So at this point, I haven't found a way to get started creating my own application using your build system. I found this community and asked for a template. So far I've spent days trying to figure out how to get started on a project that uses tflite micro. There seems to be a dependency management system, since there are a bunch of libs in your repo, but for the life of me I can't figure it out.

I'm here again to ask if there's an actual getting started guide that tells me how to create a CMake based application with FreeRTOS that uses lib_tflite_micro to run a model.

Any help will be appreciated.

[michaelpoluektov]

Hi, thanks for your feedback! I'm not 100% qualified to address everything, but here's what I can answer:

> I'm here again to ask if there's an actual getting started guide that tells me how to create a CMake based application with FreeRTOS that uses lib_tflite_micro to run a model.

I don't think we've got one that's up to date, but it's being worked on. The examples in `ai_tools` use xmake, which is still the preferred way to compile stuff in the current version of XTC tools.

To clarify: I wouldn't recommend you use `lib_tflite_micro` directly: if you plan on compiling existing TFLite models (recommended), use `ai_tools`. Here's a getting started guide: https://github.com/xmos/ai_tools/blob/d ... t/flow.rst. The lower level library (for inserting common neural net operators, which is used by `lib_tflite_micro`) is `lib_nn`.

`ai_tools` (and `lib_tflite_micro` for that matter) should probably run on bare metal, since it handles parallelisation by itself. Otherwise I'm not qualified to answer anything about FreeRTOS since I've never used it.

If you want to use CMake, here's a script that will allow you to link the files you generated with `ai_tools`. Make sure `ai_tools` is installed and `XMOS_AITOOLSLIB_PATH` is set.

Code: Select all

set(XMOS_AITOOLSLIB_PATH_CMAKE "${XMOS_AITOOLSLIB_PATH}/buildfiles/aitoolslib.cmake")

if(XMOS_AITOOLSLIB_PATH STREQUAL "")
    message(FATAL_ERROR "Path to XMOS AI tools NOT found")
elseif(NOT EXISTS  ${XMOS_AITOOLSLIB_PATH_CMAKE})
    message(FATAL_ERROR "Cmake file 'aitoolslib.cmake' NOT found in this path")
else()
    message(STATUS "Found python package xmos-ai-tools at: ${XMOS_AITOOLSLIB_PATH}")
    include(${XMOS_AITOOLSLIB_PATH_CMAKE})
    set(LIB_NAME tflite_micro)
    add_library(${LIB_NAME} STATIC IMPORTED GLOBAL)
    target_compile_definitions(${LIB_NAME} INTERFACE ${XMOS_AITOOLSLIB_DEFINITIONS})
    set_target_properties(${LIB_NAME}  PROPERTIES
        LINKER_LANGUAGE CXX
        IMPORTED_LOCATION ${XMOS_AITOOLSLIB_LIBRARIES}
        INTERFACE_INCLUDE_DIRECTORIES ${XMOS_AITOOLSLIB_INCLUDES})
endif()

[andy-aic]

Hi Michael,

Thanks for your answer.

Just for anyone else reading this, I've found reading the docs from https://github.com/xmos/sw_usb_audio with the ones from https://www.xmos.com/documentation/XM-0 ... index.html to be much more useful and helpful to get started rather than going through the SDK docs.

I don't understand why something as basic as "the preferred way to compile stuff" is hidden all the way down in a reference section of the Tools docs. This stuff should be front and center in the getting started sections.

It's much more useful to understand the folder structure required to develop for the xcore platform than learning how to call xcc to compile a single c file.

[MichaelB]

Hi Andy,

Thanks for pointing out the problems you've had with the documentation. I agree that the arrangement is not ideal at the moment.

In your original post, you said that you're interested in running an ML model for an audio application. The documentation best suited to help you will depend on the nature of that audio application. Do you want to process voice signals, music signals, or something else? Any information you can share about your application will help.

[andy-aic]

We'll be evaluating running a speech enhancement model on the xcore.ai evk. FWIW I've already read the docs on the ai_tools repo, but I found it to be more focused on the ML side of things (python, tflite, etc) than on actually explaining how to run the examples. I think the docs on most repos expect the reader to already have certain knowledge about the XMOS way of doing things, and that way of doing things is not particularly well explained in one single place.

Regarding running models, I've had a few questions whose answers I'm not even sure where to even start looking for.
- how much work is done for me in terms of taking advantage all the cores of a given tile? Is this automatic? Inherent to the tfmicro port?
- how to best use the LPDDR1 memory? I feel that you can take advantage of it, but this is somehow limited by the size of flash? Or am I missing something? If I have 128 MB of RAM, but only 4 MB of flash how can I take advantage of more than 4 MB of RAM if I need to transfer the weights from flash in the first place?

[andy-aic]

Another question. I built and ran this example, and wanted to check if I could run it with XSIM, but it doesn't seem to work. It looks like it hangs? I did try it with a simple hello world and that worked fine. Does XSIM have limitations if the application needs flash to run?

[albertoisorna]

Hi Andy,

Thanks for your post and comment.
If your application uses a model stored in Flash, it is not straightforward to use xsim to emulate your example, the easiest way will be to run it on board.
We will discuss about how we could use xsim and flash and provide more info.
Also, for debugging reasons, you can use the Python interpreter: https://github.com/xmos/ai_tools?tab=re ... nterpreter.

[andy-aic]

Hi Alberto,

Thank you for your answer.

My most immediate goal would be to have a simple way to test as many models as possible (hundreds or thousands), relatively quickly and easily. I basically need to run inference using a given model architecture and get an estimate of how fast does it run on the xcore.ai. Since we want to test many different model architectures, sizes, etc, I would like to be able to write a script that compiles a program using a given model, and runs it in the simulator. I guess ultimately we can also just have a dev board connected and use xflash and xrun instead.

I'd still appreciate answers to the other two questions, if possible :)

[MichaelB]

Hi Andy,

If your project only includes running a NN model, then the AI tools are probably your best source of information. If you want something more complex and you're interested in speech, then xcore-voice may be a better starting place that xcore-iot. It already includes examples that use a Sensory or Cyberon ASR engine, although none of them involve generating the model itself. Whether you use one of those engines or one from another source, you will see how include one. Xcore-voice assumes use of a different board than the one you have, specifically one with built-in microphones and a Digital to Analogue Converter. It also includes a classical voice DSP pipeline. With a bit of work, you should be able to either modify the code to ignore the microphone inputs or wire in some microphones to the Explorer board.