Ingress filter

From Allegro Network Multimeter Manual
Jump to navigation Jump to search


The ingress filter page, allows setting allow/deny filters for live traffic preprocessing. Filtered out/denied traffic, will NOT become available throughout the dashboard nor the packet ring buffer. Filtered out/denied traffic will be irretrievable for (post) analysis.

Functionality

Filters can be applied for:

  • IP addresses (with possible subnet mask).
  • pairs of IP addresses (with possible subnet mask).
  • MAC addresses.
  • VLAN tags (or none for no VLAN tag).
  • specific TCP/UDP ports.
  • physical interface IDs (as listed in Interface statistics).
  • duplicate packets.
  • a set of filter via the expert-filter

They can all be set to either deny list or allow list mode. Filtering will be evaluated for every packet in tab order. The more restrictive filter will be applied. For instance if no IP address is denied but a specific MAC address is on the deny list, no traffic for that MAC address will be processed.

Important Notes

The ingress (NIC) filter is applied to live traffic only, i.e. the traffic sent to the monitoring interfaces of an Allegro Network Multimeter. When replaying data from the ring buffer, loading a pcap or using the remote traffic capture feature, filtering is not used and/or applied.

Empty allow lists will discard all traffic so no traffic will be available for traffic processing.

The data recorded to/stored in the Packet Ring buffer, is of course also affected by the Ingress filter. Additional ring buffer capture rules may be configured under "Generic - Packet Ring Buffer", further explained in our wiki here https://allegro-packets.com/wiki/Packet_ring_buffer#Packet_ring_buffer_snapshot_length_filter

NIC filter.png

IP filters

In the IP filter/IP pair filter, IP addresses and IP ranges can be entered with their respective subnet mask.

  • /32 subnet mask: 192.168.1.1/32 means the filter will deny/allow IP 192.168.1.1
  • /24 subnet mask: 192.168.1.0/24 means the filter will deny/allow IP range 192.168.1.0 - 192.168.1.255
  • /16 subnet mask: 192.168.0.0/16 means the filter will deny/allow IP range 192.168.0.0 - 192.168.255.255

The IP filter/IP pair filter allows for importing an IP list in the following format:

#A line with a comment
1.2.3.1
1.2.3.2
1.2.3.3

Each entry can be annotated with a comment by clicking on the pencil icon in the table row.

By clicking on Import list a dialogue box will be opened where you can choose to download such a list from a given URL or specify a file from your system. The IP addresses are added to the existing ones up to a maximum of 10000 addresses.

The Export list button allows for exporting the IP filter list in the same format as the import.

The Delete all button allows for deleting all IP addresses from the filter list.

Expert Filter

You are able to filter the packets from live traffic via the expert or complex filter which you can also use when capturing traffic. The filter is defined in a C-style syntax and supports combination of AND/OR operators, precedence order with parentheses and equal/not equal comparisons.

You are also able to select if packets matching the filter should be monitored or excluded from monitoring. More information are on the Capture module and Complex filter sites in the wiki.

Packet deduplication

Packet deduplication provides the ability to filter packets from live traffic which have already been seen. This feature creates a hash from significant parts of the packet and stores the hash for a certain amount of time and within the configured memory limit. If for a second packet (or possibly further packets) the same hash value is calculated this packet is discarded and will not be analyzed by the system. The feature provides several options for configuring which parts of a packets are regarded as significant for duplicate detection.

It is also possible to capture packets which have been detected as duplicates but since these packets are excluded from further processing as well as the packet ring buffer it is only possible to create a live capture. Also, since only hash values are stored, the first packet of a series of duplicates will not be part of the duplicate capture, but it can be captured with the regular capture feature as it is part of the packet processing.

Packet deduplication works on a per Virtual Link Group basis. Duplicates are detected within each VLG so duplicates from one link groups do not interfere with packets having the same hash in other link groups.

To utilize the packet deduplication feature, follow the following steps:

  1. Configure and enable the packet deduplication engine in the Ingress filter configuration section as described below.
  2. Configure each VLG to enable or disable packet deduplication for that group. It is on by default.

Statistics

The top graph and counter show how many packets have been discarded as duplicates.

The Memory used graph shows how much of the memory which has been configured for use by the packet deduplication is actually consumed. If the value is very high it is possible that the configured amount of memory is not sufficient for the actual traffic. Insufficient memory results in reduction of the packet timeout.

The Oldest packet age graph shows how old the oldest packet known to the packet deduplication is. If this value is significantly lower than the configured Packet timeout value the configured amount of memory may not be sufficient for the actual traffic.

Settings

Option Description
Enabled Turns the packet deduplication filter on and off.
Reserved memory (MB) Controls how much memory in megabytes is reserved for packet deduplication. This memory then cannot be used for other statistics. Changes to this value will need a restart of the processing to take effect.
Packet timeout (ms) The time in milliseconds after which a packet hash is removed from the packet deduplication. If the time is between identical packets is longer than this value the packets will not be detected as duplicates.
Compare starting at layer Here it is possible to choose where the packet deduplication will start to analyze the packet. If e.g. 'Layer 3' is chosen it is possible for two packets to have different MAC addresses and still be detected as duplicates.
Layer 7 length limit for compare (bytes) This value controls how many bytes of the application payload are actually used for the hash calculation. A very high value may affect the performance while a very low value may increase the risk of false positives.
Ignore VLAN The VLAN tag will not be used by the packet deduplication so that two packets from different VLANs can still be detected as duplicates.
Ignore IP TOS and traffic class The IP 'type of service' and 'traffic class' fields will not be used by the packet deduplication so that two packets with different values in these fields can still be detected as duplicates.
Ignore IP TTL and HOP The IP 'time to live' and 'hop counter' fields will not be used by the packet deduplication so that two packets with different values in these fields can still be detected as duplicates.
Ignore TCP SEQ and ACK numbers The TCP sequence and acknowledgement numbers will not be used by the packet deduplication so that two packets with different TCP sequence and acknowledgement numbers can still be detected as duplicates.
Ignore TCP options Any TCP options will not be used by the packet deduplication so that two packets with different TCP options can still be detected as duplicates.

Limitations

  1. In some circumstances, real duplicates cannot be distinguished from retransmissions. For example, for TCP in IPv6 traffic a retransmitted ACK packet might be byte-wise identical to the original ACK packet. The IPv6 header does not have an IP-ID field by default so it is identical and the TCP header is identical too if both the sequence and acknowledge number are the same and no timestamp option header is used. In this case it might help to decrease the packet timeout in the deduplication configuration since real duplicates in a network setup usually appear much faster than actual retransmissions.