lib_ethernet - Packet Filtering Topic is solved

[DemoniacMilk]

The ethernet library allows multiple client threads to be connected, each of them setting up custom filters. Any received packet is analyzed and if it passes the filtering for a (or multiple?) thread(s) it is forwarded.

Can I

• set up multiple threads, so both get the same packet?
• send all packets to a specific thread, that were not wanted by any other thread (including packets that were not meant for the devices MAC adress)?

EDIT:
found a counter in the librarys code that should be holding the number of threads that want a package, so the first question is answered.

I think i also found the location where unwanted (as in: wrong mac address) packages are dropped:

Code: Select all

          unsigned key0 = buf->data[0];
          unsigned key1 = buf->data[1] & 0xffff;
          unsigned filter_result = mii_macaddr_hash_lookup(table, key0, key1, &buf->filter_data);
          if (filter_result) {
            buf->filter_result = filter_result;

            if (ethernet_filter_result_is_hp(filter_result))
              buffers_used_add(used_buffers_rx_hp, (mii_packet_t *)buffer, RGMII_MAC_BUFFER_COUNT_RX, 1);
            else
              buffers_used_add(used_buffers_rx_lp, (mii_packet_t *)buffer, RGMII_MAC_BUFFER_COUNT_RX, 1);
          }
          else {
            // > > Drop the packet  <-- WHAT IF I WANT TO GIVE THE PACKET TO A CERTAIN THREAD INSTEAD? < <
            buffers_free_add(free_buffers, (mii_packet_t *)buffer, 1);
          }

but I dont see how I could get them out of there. I could try to add the package to a used buffer, but it still did not meet the filter requirements to be sent to a thread, so it would stay in the buffer forever?

[Thomas]

What version of the Ethernet Software are you using?
We recommend lib_ethernet for new Designs.
This library has functions add_ethertype_filter and add_macaddr_filter which can be called by clients to define which packets should be forwarded to them by the filter.

That way you should be a able to achieve this:
- set up multiple threads, so both get the same packet?

To achieve this:
- send all packets to a specific thread, that were not wanted by any other thread (including packets that were not meant for the devices MAC address)?

you will have to modify lib_ethernet at a low level:
ethernet_do_filtering: Change the function to assign result also when the mac address doesn't match
send_to_clients: Change it to forward packets to a specific client that are not wanted by any other client.

[henk]

Hi,

I presume the problem is filter_result written into the buf structure. Could you assign that a value, or alternatively hack mii_macaddr_hash_lookup() to return the right value for anything else?

Cheers,
Henk

[DemoniacMilk]

Some days late, but I was working on another problem first.

On examining lib_ethernet (v3.1.2), data/packets seem to go through the following steps:

1. received by PHY
2. sent through RGMII Interface
3. received and CRC checked by low level driver
4. passed to buffer manager
   • drop packet if buffer full
5. MAC address hash check
   • hash check == 0: drop the packet
   • hash check != 0: add buffer to used buffers for further processing

At this point, data with valid target MAC address is stored in the input buffer and we know that at least one client wants the packet.
I think the ethernet_rx_server task takes care of the packets from here, but do not understand the meaning of the different client and link states. The task does

1. check for periodic and interface events
2. check link states
   • this also notifies the client if a packet is ready, but i dont understand this section
3. Process all high priority packets
4. Handle incoming low priority packets
   • Loop through all connected clients and:
     • Check result of MAC hash to see if the current client may want the packet
     • If client set a filter for ether types, compare ether type and filter values
     • Client wants packet -> add to client qeue if not full -> notify client

So what I think I need to adjust is the MAC address hash check.
Only packets passing through this step will be available to the rx server and the hash result contains what client would like a packet (except if additional ethertype filters are set). I don't understand how the hashing works tho. To me it looks like a simple comparison is done that magically reveals if the target mac address is what a client would like to have.
In ethernet_add_filter_table_entry it seems like the hash result is used as an index for an array, and the array elements contain some kind of a bit mask, where a 1 as bit n signals that client number n wants the packet? What if I go through the table (maybe as a reaction to some additional config_interface function) and just set all the bits for the calling client to 1? The MAC filter size is limited to 256, so this would probably not forward all packets?

[DemoniacMilk]

Actually, writing threads is a pretty helpful. I had an idea that seemed rather simple in the end.

In macaddr_filter_hash, two variables were added wantsAll and wantsUnwanted. They are meant to store the clients who either want every packet or just packets no one else wanted (kind of a bit mask).
Interface functions were added to the ethernet config interface, allowing a client to (un)register for wanting all/unwanted packets and the mii_macaddr_hash_lookup()-function was altered to

Code: Select all

[...]
  if (key0 == table->entries[x].id[0] &&
      key1 == table->entries[x].id[1]) {
    *appdata = table->entries[x].appdata;
    return (table->entries[x].result | wantsAll); // instead of just .result
  }
  return (wantsUnwanted | wantsAll); // instead of 0
}

I am running into parallel usage violations now, cause the variables are used at multiple locations. Integrating the variables into the hash table should fix this?
There is not only the hash filter, there is also another version of the filter in macaddr_filter.xc. This one is used for 10/100Mbit ethernet?

[DemoniacMilk]

Okay so the filters were adjusted to

Code: Select all

typedef struct mii_macaddr_hash_table_t
{
  unsigned polys[2];
  unsigned num_entries;
  mii_macaddr_hash_table_entry_t entries[MII_MACADDR_HASH_TABLE_SIZE];
  unsigned wantsAll;
  unsigned wantsUnwanted;
} mii_macaddr_hash_table_t;

and

Code: Select all

unsigned mii_macaddr_hash_lookup(mii_macaddr_hash_table_t *table,
                                 unsigned key0,
                                 unsigned key1,
                                 unsigned *appdata)
{
  if (key0 == 0 && key1 == 0)
    return 0;

  // Always perform both lookups to ensure lookup time remains
  // relatively constant
  unsigned int x = hash(key0, key1, table->polys[0]);
  unsigned int y = hash(key0, key1, table->polys[1]);
  
  if (key0 == table->entries[y].id[0] &&
      key1 == table->entries[y].id[1]) {
    *appdata = table->entries[y].appdata;
    return (table->entries[y].result | table->wantsAll);
  }
  
  if (key0 == table->entries[x].id[0] &&
      key1 == table->entries[x].id[1]) {
    *appdata = table->entries[x].appdata;
    return (table->entries[x].result | table->wantsAll);
  }
  printhexln(count++);
  return (table->wantsAll | table->wantsUnwanted);
}

both to be found in macaddr_filter_hash.c. Something equivalent was done in macaddr_filter.c.


With additional interface functions for the Config-interface

Code: Select all

        case i_cfg[int i].give_all_packets(size_t ifnum):
            rgmii_add_client_wants_all(ifnum);
            break;
        case i_cfg[int i].not_give_all_packets(size_t ifnum):
            rgmii_remove_client_wants_all(ifnum);
            break;
        case i_cfg[int i].give_unwanted_packets(size_t ifnum):
            rgmii_add_client_wants_unwanted(ifnum);
            break;
        case i_cfg[int i].not_give_unwanted_packets(size_t ifnum):
            rgmii_remove_client_wants_unwanted(ifnum);
            break;

it is possible to register clients for wanting all/unwantend packets:

Code: Select all

    // get the tasks RX client index and let CFG know that we want aaaaaaall packets
    unsigned int index = ethRx_if.get_index();
    ethCfg_if.give_all_packets(index);

by setting a bit in wantsAll or wantsUnwantend

Code: Select all

void rgmii_add_client_wants_all(unsigned ifnum){
    backup_table->wantsAll |= (1 << ifnum);
    swap_tables(0);
    backup_table->wantsAll |= (1 << ifnum);
}

void rgmii_remove_client_wants_all(unsigned ifnum){
    backup_table->wantsAll &= ~(1 << ifnum);
    swap_tables(0);
    backup_table->wantsAll &= ~(1 << ifnum);
}
(and the same for 'unwanted' instead of 'all')

It seems like the changes do not achieve what I wanted. I registered a thread for "wantsAll", but it does never get any packets.
EDIT: It works well, I just had a flaw in my bridge thread that sends ethernet data to a serial interface: On finishing a serial transmit, the serial server notifys the bridge that it is ready to send another packet. On receiving the notification, the bridge needs to acknowledge and then get another ethernet packet. I simply forgot to acknowledge, so no additional packet was sent.