Use external LPDDR memory on tile #1

[alex4d1]

Hi all,

I have a custom board with a XU316 and external memory, identical setup as on the xcore.ai Explorer board.
My XN file has an entry for the external memory just like the XN file of the XCORE examples provided by XMOS.

I was able to use the external memory on tile #0 simply by doing this:

Code: Select all

#if ON_TILE(0)
__attribute__ ((section(".ExtMem.bss")))
#endif
int32_t mic_data[4096]; 

Note that this array is used in my code further down, so it won't be discarded by the linker.


Based on the documentation I can use the external memory either on tile #0 or on tile #1, this is decided at compile time.
So I changed my code to this:

Code: Select all

#if ON_TILE(1)
__attribute__ ((section(".ExtMem.bss")))
#endif
int32_t mic_data[4096]; 

Now my application compiles correctly, but it doesn't run. XRUN does not run correctly, I don't see any output and it doesn't return.
Only if I reduce the array size to 64 or less it works.
I believe this lines up with the L1 cache size, so does this mean the caching works but the cache is not able to write the ext memory?

If I use the following attribute

Code: Select all

__attribute__ ((section(".ExtMem.data")))

instead and run the application with XRUN I get:

xrun: Binary not correctly downloaded to target device

and XRUN returns.

I also tried manually enabling the external memory on tile one in main.c:

Code: Select all

    // Enable external memory access
    unsigned int ctrl0 = getps(XS1_PS_XCORE_CTRL0);
    ctrl0 |= XS1_XCORE_CTRL0_EXTMEM_ENABLE_MASK;
    setps(XS1_PS_XCORE_CTRL0, ctrl0);

But I believe this would be too late anyways, since the array is in the BSS section which is initialized by the startup code.

Does anybody have successfully accessed the external memory from tile #1? Any help would be greatly appreciated.

Cheers
Alex

[alex4d1]

Just some follow up information:

I confirmed the issue with the "getting_started" example and the xcore.ai Explorer board.
I modified the "getting_started" example as follows:

#if ON_TILE(1)
__attribute__ ((section(".ExtMem.bss")))
uint32_t mic[4096];
#else
uint32_t mic[4096];
#endif

void tile0_hello_task(void *arg) {
rtos_printf("Hello task running from tile %d on core %d\n", THIS_XCORE_TILE,
portGET_CORE_ID());

for (;;) {
rtos_printf("Hello from tile %d: %d\n", THIS_XCORE_TILE, mic[0]);
vTaskDelay(pdMS_TO_TICKS(1000));
mic[0]++;
}
}

void tile1_hello_task(void *arg) {
rtos_printf("Hello task running from tile %d on core %d\n", THIS_XCORE_TILE,
portGET_CORE_ID());

for (;;) {
rtos_printf("Hello from tile %d: %d\n", THIS_XCORE_TILE, mic[0]);
vTaskDelay(pdMS_TO_TICKS(1000));
mic[0]++;
}
}

If the mic array of tile #0 is defined on the external memory then everything works fine. If the mic array of tile #1 is defined on the external memory then tile #1 is getting stuck somewhere in the __start_core function.
The address pointer is pretty high so I assume it is stuck in the hard fault handler or similar.

I verified that the XCORE_CTRL0 register is set correctly (by setting the array size to <= 64, so I was able to get the tile running to read back the register). I can see that the correct tile has the external memory interface enabled and the size is configured correctly.

I am clearly missing something here. Any ideas what it could be?

Cheers
Alex

[CiaranW]

Hi Alex,

This should work. I wonder if somewhere else in your application or libraries, something is trying to use ExtMem on tile[0]? That might cause the LPDDR to get routed to tile[0] instead of tile[1].

Here is a minimal example of using ExtMem on tile[1], which should hopefully help you find the issue. I have tested this on the XCORE-AI-EXPLORER. 3 files:

MultiTileMain.xc

Code: Select all

#include <stdio.h>
#include <platform.h>
#include <xs1.h>

extern "C" {
  int tile0_main(void);
  int tile1_main(void);
}

int main(void) {
  par {
    on tile[0]: tile0_main();
    on tile[1]: tile1_main();
  }
  return 0;
}

PatternData.c

Code: Select all

// Initialise a DDR array to a pattern.
// Check it was initialised.

#include <stdio.h>
#include <stdlib.h>
#include <print.h>

__attribute__((section(".ExtMem_data"))) int x[] __attribute__((aligned(4))) = {
  0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
  20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
  40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
  60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
  80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99,
  100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119,
};

int tile1_main(void) {
  for (int i = 0; i < (sizeof(x)/sizeof(x[0])); i++) {
    if (x[i] != i) {
      fprintf(stderr, "Array index %d was not initialised in EXTMEM!\n", i);
      exit(EXIT_FAILURE);
    }
    printf("x[%d]==%d\n", i, x[i]);
  }

  return 0;
}

int tile0_main(void) {
  return 0;
}

Makefile

Code: Select all

run.xe: PatternData.c MultiTileMain.xc
	xcc $^ -o $@ -O2 -target=XCORE-AI-EXPLORER -mcmodel=large -g

If you put these 3 in a folder, then 'xmake' and 'xrun --io run.xe', you should see the LPDDR being accessed on tile[1]. If you wish, you can run it in xgdb instead and see that the memory accesses are actually going to LPDDR from tile[1].

Hope this helps,
Ciaran

[alex4d1]

Thanks Ciaran.

Some more insights:
I compiled and ran your example on my Explorer board successfully.

Then I integrated your code into the bare-metal example “ explorer_board”. It also ran fine.

So this indicates that this issue seems to only appear in an RTOS application.
I wonder whether this is something inside the rtos bsp or in the RTOS port itself?

Thanks
Alex

[alex4d1]

Ok finally some progress:

It seems to me that the ON_TILE macro is causing this issue. I refactored the getting_started example and removed all the ON_TILE macros and now it works fine.
My application is heavily using this macros so I'll have to see how I can work around this.
Anyways, here is the modified example for anyone who is interested:

Code: Select all

// Copyright 2021 XMOS LIMITED.
// This Software is subject to the terms of the XMOS Public Licence: Version 1.

#include <platform.h>

#include "FreeRTOS.h"
#include "task.h"

#include "rtos_printf.h"

#include "platform/platform_init.h"
#include "platform/driver_instances.h"

__attribute__((section(".ExtMem_data"))) int x[] __attribute__((aligned(4))) = {
  0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
  20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
  40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
  60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79,
  80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99,
  100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119,
};

void tile0_blinky_task(void *arg) {
  rtos_printf("Blinky task running from tile %d on core %d\n", THIS_XCORE_TILE,
              portGET_CORE_ID());

  uint32_t gpio_port = rtos_gpio_port(XS1_PORT_4C);

  rtos_gpio_port_enable(gpio_ctx_t0, gpio_port);

  for (;;) {
    rtos_gpio_port_out(gpio_ctx_t0, gpio_port, 0x000F);
    vTaskDelay(pdMS_TO_TICKS(1000));
    rtos_gpio_port_out(gpio_ctx_t0, gpio_port, 0x0000);
    vTaskDelay(pdMS_TO_TICKS(1000));
  }
}

void tile0_hello_task(void *arg) {
  rtos_printf("Hello task running from tile %d on core %d\n", THIS_XCORE_TILE,
              portGET_CORE_ID());
  
  for (;;) {
    rtos_printf("Hello from tile %d\n", THIS_XCORE_TILE);
    vTaskDelay(pdMS_TO_TICKS(1000));
  }
}

void tile1_hello_task(void *arg) {
  rtos_printf("Hello task running from tile %d on core %d\n", THIS_XCORE_TILE,
              portGET_CORE_ID());
  
  for (;;) {
    rtos_printf("Hello from tile %d\n", THIS_XCORE_TILE);
    vTaskDelay(pdMS_TO_TICKS(1000));

    for (int i = 0; i < (sizeof(x)/sizeof(x[0])); i++) {
      if (x[i] != i) {
        fprintf(stderr, "Array index %d was not initialised in EXTMEM!\n", i);
        exit(EXIT_FAILURE);
      }
      printf("x[%d]==%d\n", i, x[i]);
    }
  }
}

// static void tile_common_init(chanend_t c) {
//   platform_init(c);

// #if ON_TILE(0)
//   xTaskCreate((TaskFunction_t)tile0_hello_task, "tile0_hello_task",
//               RTOS_THREAD_STACK_SIZE(tile0_hello_task), NULL,
//               configMAX_PRIORITIES - 1, NULL);
//   xTaskCreate((TaskFunction_t)tile0_blinky_task, "tile0_blinky_task",
//               RTOS_THREAD_STACK_SIZE(tile0_blinky_task), NULL,
//               configMAX_PRIORITIES - 1, NULL);
// #endif

// #if ON_TILE(1)
//   xTaskCreate((TaskFunction_t)tile1_hello_task, "tile1_hello_task",
//               RTOS_THREAD_STACK_SIZE(tile1_hello_task), NULL,
//               configMAX_PRIORITIES - 1, NULL);

// #endif

//   rtos_printf("Start scheduler on tile %d\n", THIS_XCORE_TILE);
//   vTaskStartScheduler();
// }

static void tile_common_init_a(chanend_t c) {
  platform_init(c);

  xTaskCreate((TaskFunction_t)tile0_hello_task, "tile0_hello_task",
              RTOS_THREAD_STACK_SIZE(tile0_hello_task), NULL,
              configMAX_PRIORITIES - 1, NULL);
  xTaskCreate((TaskFunction_t)tile0_blinky_task, "tile0_blinky_task",
              RTOS_THREAD_STACK_SIZE(tile0_blinky_task), NULL,
              configMAX_PRIORITIES - 1, NULL);

  rtos_printf("Start scheduler on tile %d\n", THIS_XCORE_TILE);
  vTaskStartScheduler();
}

static void tile_common_init_b(chanend_t c) {
  platform_init(c);

  xTaskCreate((TaskFunction_t)tile1_hello_task, "tile1_hello_task",
              RTOS_THREAD_STACK_SIZE(tile1_hello_task), NULL,
              configMAX_PRIORITIES - 1, NULL);

  rtos_printf("Start scheduler on tile %d\n", THIS_XCORE_TILE);
  vTaskStartScheduler();
}

//#if ON_TILE(0)
void main_tile0(chanend_t c0, chanend_t c1, chanend_t c2, chanend_t c3) {

  (void)c0;
  (void)c2;
  (void)c3;

  tile_common_init_a(c1);
}
//#endif

//#if ON_TILE(1)
void main_tile1(chanend_t c0, chanend_t c1, chanend_t c2, chanend_t c3) {
  (void)c1;
  (void)c2;
  (void)c3;

  tile_common_init_b(c0);
}
//#endif