Capture module: Difference between revisions

From Allegro Network Multimeter Manual
Jump to navigation Jump to search
Access restrictions were established for this page. If you see this message, you have no access to this page.
No edit summary
 
(123 intermediate revisions by 12 users not shown)
Line 1: Line 1:
==''' Packet ring buffer'''==
==Capture module ==  
The ring buffer feature allows to create a buffer of fixed size on an external storage device to which all processed packets will be recorded. If the fixed size buffer is full then the oldest packets in the buffer will be replaced with new packets in a round robin fashion. If the feature is not enabled a button titled ‘Create ring buffer’ is visible. Upon clicking on it a dialog will be displayed and allows to specify the size of the ring buffer. It must be ensured that enough space is available on the external storage device. As soon as the ring buffer has been created statistics about the ring buffer will be displayed instead of the button:
The Allegro Network Multimeter allows direct capturing of network traffic as a HTTP download to your computer. No packet data is stored on the device itself. Traffic can be directly filtered for specific packets, only the relevant packets will be captured. In addition, it is also possible to capture network traffic to an attached storage device, see the settings section below for details. Capturing network traffic is usually started by clicking on a PCAP button in a certain module. These buttons allow capturing specific traffic, for example for a certain IP address or a network protocol. The capture module allows to configure filter for traffic that has not even started right now, for example for an IP address that is not in use at the moment but might be used later. The capture module page displays all currently running captures and allows starting new captures with specific filters.


{| class="wikitable sortable"
|-
|[[File:Generic modules.png|800px|none|right]]
|}
=== Current captures ===
The first part of the page displays all downloads running for the current user session, and all downloads running for other user sessions (like when a download has been started outside the browser by directly using command line tools such as wget or curl).
The list contains the client IP and port of the user running the download.
The next three counters describe:
* the number of packets captured for the corresponding filter.
* the number of packets dropped by the capturing module.  Packet drops can appear when doing live capture and more packets are captured than can be transferred via HTTP to the client, or the storage is not capable of storing all matching packets. During live capture, the drop counter is the exact number of packets matching the filter but were dropped because of the reasons mentioned previously.  Packet drops are also accounted when capturing from the past out of the ring buffer. It happens when the ring buffer dropped packets during the capture interval due to insufficient bandwidth available on the storage devices. In retrospective capturing, the drop counter only indicates that some packets may have been missed. The counter is the total amount of packets not available in the ring buffer, which are not necessarily part of the capture filter.
* the number of ignored packets. Ignored packets do not match the given capture filter.
The following columns list the applied filter criteria. The last column contains a button to stop the corresponding download. Downloads can also be stopped by clicking the same capture button that started the capture in the corresponding module. If multiple devices have been configured, the list also contains all captures from all multi-devices which can be stopped individually.
==== Finished captures ====
This list shows the last captures that have finished. It shows up to 100 entries while the oldest entries are discarded when the list is full. For each entry the following information is displayed:
* the packet capturing mode
* the type of capture
* the start- and end-time of the capture (the actual real-time when the capture was initiated and ended)
* the number of captured/dropped/ignored packets during the capture
* the amount of data generated for the capture (along with the average throughput of e.g. the capture download)
* the capture filter which was used for the capture
* the finishing reason of the capture job (e.g. completed normally, stopped by user or aborted due to no space left on storage)
The button '''Delete list of finished captures''' will delete all entries from this list.
==== Recent filters ====
This list shows the most recently used capture filters for the current user. The most recent capture filter is displayed on the top. Next to each capture filter there is a button to permanently save this capture filter as a favorite as well as a button to simply start a capture with this filter again. The '''Use as expert filter''' button will copy the capture filter into the expert filter input field and allows for customizing the capture. The button '''Delete list of recent filters''' will delete all entries from this list.
==== Favorites ====
This list shows favorite capture expressions. A capture can be marked as a favorite either in the capture dialog by clicking on the star button in the top right corner or by marking it as a favorite in the '''Recently captured''' list. A description can be given and will be displayed in this list. For each favorite capture a PCAP button is available to simply start this capture again. The '''Remove favorites''' button allows for cleaning the list. The '''Use as expert filter''' button will copy the capture filter into the expert filter input field and allows for customizing the capture.
==== Planned captures ====
On this tab captures to a storage device can be planned ahead of time and these captures can even be set to repeat automatically. A click on the '''Add planned capture''' button opens a dialog where the planned capture can be configured. This includes settings like the storage device to use, the start date and time of the capture, the duration of the capture, if a capture should repeat and the filter expression used during the capture. It is important to know that planned captures will not function if the chosen storage device is not active.
==== Capture profiles ====
[[File:Capture profiles config.png|thumb|600x600px|Capture profile configuration]]
[[File:Capture profiles edit profile.png|thumb|600x600px|Edit capture profile]]
Capture profiles can be used in the capture dialog for custom packet truncation rules. Such profiles defines custom rules for packet snapshot length similar to ring buffer filters. This allows to use a different snapshot length for specific layer 7 protocols, if for example traffic of one protocol shall be captured completely while for another protocol only the IP header shall be captured. Such a capture profile can be selected in the capture dialog in the "Truncate packet length" select box.
Up to 10 different profiles can be configured by clicking at the "Add profile" button. The add/edit dialog allows to enter a profile name and up to 10 rules for snapshot length. Similar to [[Packet ring buffer#Packet ring buffer snapshot length filter|ring buffer filters]], the first rule that matches for the selected type is used to decide for the snapshot length of the actual packet.
To restrict the capture possibilities of an user it is possible to choose an 'restricting profile'. This allows the administrator to stop the user from capturing other sensitive data like SIP- and RTP-Packages.
==== PCAP anonymization profile ====
Anonymization profiles can be used in capture dialog to allow for quickly switch between rules to anonymize aspects of the generated PCAPs.


* Timestamp of oldest packet: The timestamp of the oldest packet in the ring buffer.
When enabled, following options are available (more than one option is possible):
* MAC addresses on L2
:MAC addresses on L2 will be replaced by random addresses octet-wise. Multicast/broadcast addresses will not be randomized.
* IP addresses on L3
:IP addresses on L3 will be replaced by random addresses octet-wise for IPv4 and hextet-wise for IPv6. Multicast/broadcast addresses will not be randomized. The octets of the IP address will have the same length in textual representation (e.g. 100.20.3.40 -> 105.31.6.41). For IPv6 address short notation will be considered and the randomized result will also have the same textual length.
* IP addresses on L7
:IPv4 and IPv6 addresses in textual representation in L7 payload will be randomized similar to L3.
* Mapped IP addresses in STUN packets on L7
:STUN payload IP addresses will be randomized similar to L3.
* Phone numbers, name and Call ID in SIP packets on L7
: SIP payload data is masked with 'xxx' values for the names and phone numbers in the fields "From", "To", "Contact", "P-Asserted-Identity", "P-Preferred-Identity". Call IDs, usernames and sip.instance values are also replaced. IP addresses are not touched, if they shall be anonymized, please use option "IP addresses on L7".
* URLs and HTTP hostnames on L7
: URLs and HTTP hostnames in L7 payload are masked with 'xxx' values. The length of the masked name/URL will stay the same and line feeds won't be touched.
:: Examples:
:: 'GET /website.html?param1=value HTTP/1.1\r\n' will be changed to 'GET xxxxxxxxxxxxxxxxxxxxxxxxxx HTTP/1.1\r\n'
:: 'Host: allegro-packets.com\r\n' will be changed to 'Host: xxxxxxxxxxxxxxxxxxx\r\n'
:: https://www.allegro-packets.com/en/ will be completely masked


* Total size: The total size of the ring buffer on the external storage device. If the cluster packet ring buffer feature is active and the Write redundancy level is set to a different value than no replication an adjusted value is displayed to reflect the redundant copies of packet data. The raw on-disk value will be displayed next to it in parentheses.
Within an anonymization profile it is also possible to define MAC and IP lists with entries that should be anonymized or that should be excluded from anonymization. The proper L2/L3 anonymization option '''must be turned on''' in order to have an effect!


* Used size: The currently used amount of memory in the capture buffer. If the cluster packet ring buffer feature is active and the Write redundancy level is set to a different value than no replication an adjusted value is displayed to reflect the redundant copies of packet data. The raw on-disk value will be displayed next to it in parentheses.
The amount of SIP phone numbers last hidden digits can be configured, e.g. with a setting of 4 last hidden digits a phone number +49123456789 becomes +4912345xxxx.
* Overall bytes captured since start: The amount of captured bytes since system start. This may be smaller than the used size if the system has been restarted. And it may be larger than the used size in case the ring buffer is full. The history graph shows the captured traffic of the last minute or in the selected interval (if set).
* Bytes dropped since start: The traffic which was processed but could not be written to the ring buffer since the start of processing. This is usually an indicator that writes to the external storage device were not fast enough. The history graph shows the drops over time.
* Bytes discarded due to snapshot length rules since start: The traffic which matched the snapshot length rules criteria and was not written to the ring buffer. The history graph shows discarding over time.
* Data in flight: The amount of data which is currently stored in the queue that holds processed packets before they are written to the packet ring buffer. If larger bursts of traffic need to be stored in this queue the size can be modified in the capture module settings.


Address anonymization is stable for the whole PCAP, i.e. the same addresses will be replaced by the same random addresses. As an example, if both randomization of IP addresses on L3 and L7 is active and a SIP call with RTP is captured, both IP addresses in L3 and SIP SDP payload are replaced by the same values so that the correlation of the RTP stream is still intact.


<p> When the ring buffer is full and old packets are deleted, the graphs will show the time range with no data in darkgrey background color. The time range before start of the ring buffer will be visualized in the same way. When the ring buffer is running, the behavior of the PCAP capture buttons throughout the system changes: if the user interface is in live mode and a capture is started, a dialog will appear asking to specify from how far back in time the capture should start. This way it is possible to e.g. capture the traffic of an IP address starting from an hour ago. The capture will also continue with live traffic. If the user interface is in “back-in-time” mode (a timespan from the past is selected) starting a capture will produce a dialog asking to confirm that the capture will cover exactly the timespan selected. The capture will automatically stop after the selected timespan has been processed. </p>
Checksums of the changed packets are '''not''' being recalculated.


A privileged user may enforce an anonymization profile that is being used for all users that do not have the unrestrictive capture permission in a role. This profile will always be taken into account and the unprivileged user may only add additional anonymizations on top of it for captures (but the user can't overwrite it with less restrictive settings).


==== Cluster ring buffer ====  
==== SFTP server ====
The cluster ring buffer feature allows to use multiple whole disks in parallel for a single packet ring buffer. It also allows to optionally write redundant copies of packets to multiple disks to provide fault tolerance in case of a disk failure.
Credentials and upload directories for a SFTP server can be configured here. They can be selected in the capture dialog later when choosing "Capture to SFTP server" as capture type. It is possible to either configure a user and password or use authentication with a public key. In the latter case, the displayed SSH public key needs to be inserted in the authorized hosts list on the SFTP server.
When clicking the ‘Create cluster ring buffer’ button an empty cluster ring buffer will be created and the ‘Cluster configuration’ tab on the now visible packet ring buffer statistics page becomes available. In the ‘Cluster configuration’ tab you can configure the ‘Write redundancy level’ at the very top. This level controls
how many redundant copies of each packet are written. no replication means, that only a single copy of each packet is written and provides no redundancy. This level gives the highest write bandwidth for a given number of disks. single replication means that one additional copy of each packet is written to some other disk and thus reduces the total write performance for a given number of disk to half the performance of no replication. double replication and triple replication write two and three additional copies of each packet respectively. Note that for each level to work there must be at least the number of replications + 1 disks available in the cluster.


=== Simple capture ===
The second section of the capture page allow to select some fields to filter network traffic for. By default, only the IP field is visible, the other fields can be enabled by clicking on the corresponding toggle switch. Each line allows to enter a filter criterion for the corresponding network traffic element. To start the capture with the entered filter criteria just click at the '''Start capture''' button. For reference, the expert filter expression is shown at the end of the section so it can be used to copy and paste
the string into the expert filter section.


Below the ‘Write redundancy level’ setting is the list of all disks available for use in the cluster. Following columns
=== Using expert filters to start captures ===
are displayed in the list:
The third part of the page allows for starting a download for any criterion combination using complex filter expressions. A capture filter is defined in a C-style syntax and supports combination of AND/OR operators, precedence order with parentheses and equal/not equal comparisons. If the filter exp Session can be evaluated to true, the packet is
* Disk: A description of the disk and its capacity.
captured.
* Enclosure: If the disk is part of a multi-disk enclosure this column will show the enclosure number along with the slot number.
If a value contains a space, the whole value must be quoted with “”.
* Status: If the disk has been added to the cluster this column will display the current status as ‘ok’ or ‘failed’.
Following operators are supported:
* Locator: For disks in a multi-disk enclosure the button displayed in this column allows to turn the slot locator LED on and off.  
* '''and''', '''&&''' : AND operator. The filter expression will match if all operands could be evaluated to true.
* '''or''', '''||''': OR operator. The filter expression will match if any operand can be evaluated to true.


'''Following comparison operators are supported:'''
* '''==''': Will evaluate expression to true if left and right operand are equal.
* '''!=''': Will evaluate expression to true if left and right operand are not equal.


In the last unlabeled column there are three buttons displayed which have the following functionality:
'''Following operands are supported:'''
* Add to cluster: Add a fresh disk to the cluster. The disk will be formatted and added as empty storage to the cluster. All previous data on the disk is lost.
* '''ip''': An IP address. The packet is captured if either source or destination IP address of the packet match. A netmask and a port can also be specified. For IPv6 addresses with a specific port, the address must be written in brackets.  
* Resume in cluster: If the disk was previously part of a cluster it can be resumed. The data on that disk is now part of the packet ring buffer.
:Example:
* Remove from cluster: Remove the disk from the ring buffer. The data stored on that disk is not part of the packet ring buffer anymore but the data is not removed from the disk. It can be resumed in the cluster at a later time.
:{| class="wikitable sortable" 
|-         
| 
ip == 10.0.0.1


If a disk is missing because it was e.g. removed from the enclosure it will be displayed in a separate list with much of the information as in the list described above but only one button with the option to remove it from the cluster packet ring buffer.
ip == ff02::1:3


ip == 10.0/16


==== Packet ring buffer snapshot length filter ====
ip == 10.0.0.1:1234
Rules can be configured that control the snapshot length of each packet which shall be stored in the packet ring buffer. These rules can also be used to prevent certain packets from being stored in the packet ring buffer. This allows to fine tune how much packet data needs to be written to the packet ring buffer. The information about the original length of a packet will still be available in captures except when the packet was not written to the packet ring buffer at all (e.g. due to a ‘discard’ rule).  


These rules can be created, edited, deleted, moved up and moved down in the rules list by using the respective buttons.
ip == [2a02:810a:1340:1292:1c6b:e58d:6ebc:6cd2]:123
|-
|}


Evaluation of the rules takes place in the order of the rules as displayed in the rules list from top to bottom. The first rule that matches for a given packet will be applied and no further rules will be evaluated for that packet. This means that the most generic rule should be at the bottom of the list (like e.g. ‘all packets will be discarded’) and more specific rules should be higher up in the list (like e.g ‘packets with an IP matching 192.168.1.0/24 will be fully captured’).
* '''ip.src, ip.dst''': The source or destination IP address.
:Example:
:{| class="wikitable sortable" 
|-         
| 
ip.src == 10.0.0.1/8 and ip.dst == 10.0.0.1/8
|-
|}
: This will match traffic only within 10.0.0.1/8 subnet. If src/dst is omitted, also in or outbound traffic from or to other subnets is captured!


*'''mac''': A MAC address. The packet is captured if either source or destination MAC address of the packet match.
:Example:
:{| class="wikitable sortable" 
|-         
|  mac == 12:34: :56:78:90:ab
|-
|}


When creating a snapshot length filter rule, a dialog is displayed and allows following options:
* '''port''': A TCP or UDP port. The packet is captured if either source or destination port match.
* Rule condition: Match all packets or a certain MAC or IP address, TCP/UDP port, a layer 7 protocol a VLAN tag or an interface. The input field below allows entering the corresponding value.
:Example:
* Negate: Controls comparison of the rule condition to the value. If this is off, the value must match. If this is on, the value must not match.
:{| class="wikitable sortable" 
* Action: What shall be done with the matching packets.
|-       
| port == 80
|-
|}
 
* '''portrange''': A TCP or UDP port range. The range can be a single number or a comma separated list of values or value ranges.  
:Example:
:{| class="wikitable sortable" 
|-                   
| portrange == 80,100-120,-10,65000-
|-
|}
 
* '''serverport''': A TCP or UDP port of a server. The packet is captured if the given port is a port of the server and not of a client. The range can be a single number or a comma separated list of values or value ranges.
:Example:
:{| class="wikitable sortable" 
|-         
| serverport == 80,100-120,-10,65000-
|}


– Snapshot length: The packet is captured with a max length as specified in the input field below. If the packet is larger, the remaining bytes will be discarded.
* '''macProtocol''': A MAC protocol such as IPv4 or IPv6. For all seen MAC protocols, please consult the MAC Protocol Statistics module.  
:Example:
:{| class="wikitable sortable" 
|-         
| macProtocol == IPv4
macProtocol == "Non IP"
|-
|}


– Discard: Discard the whole packet.
* '''l4Protocol''': A layer 4 protocol such as TCP or UDP or any IP protocol number. Protocols can also be OR combined as a comma seperated list.
:Example:
:{| class="wikitable sortable" 
|-         
| l4Protocol == ICMP,ICMPv6
|-
|}


– Full: The whole packet is captured.
* '''l7Protocol''' or '''dpiProtocol''': A layer 7 protocol. Protocols can also be OR combined as a comma seperated list. For all seen protocols please consult the Layer 7 protocols module.
* '''countryCode''': A country code such as US. For all seen country codes please consult the Geolocation module.
* '''arpip''': An IP address within an ARP request or response.
* '''vlan''': A VLAN tag of an outer or inner VLAN. May be a number or none or any.
* '''outervlan''': A VLAN tag of an outer VLAN. May be a number or none or any.
* '''innervlan''': A VLAN tag of an inner VLAN. May be a number or none or any.
* '''multicastGroup''': A multicast IP address or any. The filter will match all IGMP or MLD negotiation packets related to that multicast IP address.
* '''rtpPayloadType''': The RTP payload type such as PCMU or MP2T. This filter will match all RTP packets with the given payload type.
* '''interface.name''' or '''namedInterface''': The physical interface. This can be the name or interface number (as printed on the device). For interface ids please consult the Interface stats page.
:Example:
:{| class="wikitable sortable" 
|-         
| <nowiki>interface.name == uplink || interface.name == 3</nowiki>
|-
|}


– Header + data: Capture just certain parts of the packet. When selecting “L3 header”, layer 2 and layer 3 headers are stored. When selecting “L3 + L4 header”, layer 2, 3 and 4 headers are stored. When selecting “L3 + L4 + L7 data”, an input field is shown where the length of layer 7 data can be configured. In this case layer 2, 3 and 4 are stored together with the specified amount of layer 7 data.
* '''interface.rawid''': This is the raw internal ID of interfaces. It starts from 0. The interface stats page also shows the raw ID.
* '''link''': The link pair of two interfaces as stated in Interface stats. A single link number can be given.
* '''pppoeProtocol''': A specific PPPoE protocol (by name or by ID), or "none"/"any"
* '''pppoeAuthentication''': An authentication state of a PPPoE session
* '''pppoeLinkConfiguration''': A link configuration state of a PPPoE session
* '''ptpMsgType''': A specific PTP message type number or any for the whole PTP traffic.
* '''profinetFrameId''': A specific Profinet frame ID.
* '''profinetCmOpnum''': A specific operation number for Profinet CM (Context Manager) requests or responses. Can also be any for every operation number. Following values are used:
:#connect
:#release
:#read
:#write
:#control
:#read implicit


* '''mpls''': A label of an outer or inner MPLS. May be a number or none or any.
* '''outerMpls''': A label of an outer MPLS. May be a number or none or any.
* '''innerMpls''': A label of an inner MPLS. May be a number or none or any.
* '''qosIpDscp''': The DSCP value in the IP header. May be a number.
* '''qosMplsTc''': The traffic class value in the outermost MPLS label stack entry.
* '''qosVlanPcp''': The priority code point value in the outermost VLAN tag.
* '''group''': The name of a configured group or ‘default’. If the name contains whitespaces, the name must be enclosed in quotes.
* '''badCRC''': The value of this operand will be 1 for packets with a CRC error and will be 0 for good packets. Capturing packets with bad CRC is currently only supported on 1Gb interfaces.
* '''icmpType''': The value of a certain ICMP type (e.g. Echo request 8, Echo reply 0).
* '''dhcpMessageType''': The value of a certain DHCP message type (e.g. Request 3, ACK 5).
* '''tcpFlags''': A single TCP flag or a list of TCP flags joined by the ‘+’ sign. If a list of flags is given, all flags must be present in the packet. Supported TCP flags are syn, ack, fin, rst, psh and urg.
* '''callId''': The string value of a SIP call ID or similar identifier (e.g. P-Palladion-ID)
* '''ipFragment''': If set to 1 all IPv4 fragments will be captured (i.e. packets having the 'More fragments' flag and 'Fragment offset' set). If set to 0 all packets without IPv4 fragmentation will be captured.
* '''regexp''': The packet payload matches the quoted regular expression (RegEx) to the other side of the == operator or does not match the regular expression to the other side of the != operator. In case of IP packets the matching will be performed on the L7 payload of the packet. In case of non-IP packets the matching will be performed on the whole packet except the Ethernet header. Regular expressions largely support the pattern syntax used by the PCRE library with the exception of certain constructs. An invalid pattern will produce a descriptive error message and prevent the capture from being started.
* '''ipDoNotFragment''': The value of this operand will be 1 for IPv4 packets that are marked as to not be fragmented (packets which have the 'do not fragment' flag set).
* '''mactype''': The type of the packets destination MAC address. Allowed values are 'unicast, 'broadcast' and 'multicast'.
* '''slowSubtype''': The subtype of packets with the macProtocol == SLOW (e.g. 1 for LACP).
* '''wifiFrequency''': The frequency (channel) of a WiFi packet in MHz.
* '''wifiBssid''': The BSS ID participating in a WiFi packet. This is similar to a MAC address.
* '''callNumber''': SIP caller (SIP from tag) or callee (SIP to tag) number (firmware >= 4.5)
* '''callerNumber''': SIP caller (SIP from tag) number (firmware >= 4.5)
* '''calleeNumber''': SIP callee (SIP to tag) number (firmware >= 4.5)


For a specific precedence you may use parentheses '''('''/''')'''.


==== Analyzing the packet ring buffer ====
Examples:
When the packet ring buffer is activated it is possible to restart the packet processing core and analyze all packets contained in the packet ring buffer. When the Analyze packet ring buffer button is pressed a dialog will appear which allows to choose the time range of the packet ring buffer which is to be replayed. After confirming this dialog the Network Multimeter will reset all statistics and start analyzing the contents of the packet ring buffer. Progress, statistics and the option to resume normal operation will appear on the Packet ring buffer page.
* The expression
:{| class="wikitable sortable" 
|-         
| ip == 10.0.0.1:1234 and ip == 10.1.0.1:9876
|-
|}
:will match a connection from 10.0.0.1 to 10.1.0.1 or vice versa with the ports 1234 and 9876 involved.


* The expression
:{| class="wikitable sortable" 
|-         
| ip == 10.0.0.1 and ip != 10.0.0.2
|-
|}
:will match packets having 10.0.0.1 either as source or destination. If a communication peer of 10.0.0.1 is 10.0.0.2 the packets will not be captured.


* The expression
:{| class="wikitable sortable" 
|-         
| l4Protocol == ICMP,ICMPv6
|-
|}
:will match packets with ICMP or ICMPv6 layer 4 protocols.


==== Extracting the packet ring buffer ====
* The expression
When the packet ring buffer is active the complete contents of it can be extracted by capturing the complete timespan that is contained within. For convenience a button labeled Extract packet ring buffer is available that opens the capture dialog with the start time and end time set to the appropriate values.
:{| class="wikitable sortable" 
|-         
| portrange == 80,443
|-
|}
:will match packets to or from port 80 or 443.


* The expression
:{| class="wikitable sortable"
|-
| regexp == "allegr[o,a]'''<nowiki>|</nowiki>'''HTTP"
|-
|}
:will case sensitive match packets that contain the string(s) 'allegro' and/or 'allegra' and/or 'HTTP' anywhere in the payload.
:NOTE: The use of regexp is CASE sensitive. You must use the (?i) modifier to enable case insensitive filtering.


* The expression
:{| class="wikitable sortable"
|-
| regexp == "(?i)allegro'''<nowiki>|</nowiki>'''http"
|-
|}
:will case insensitive match packets that contain the string(s) 'allegro' and/or 'http' anywhere in the payload.
:NOTE: The use of regexp is CASE sensitive. You must use the (?i) modifier to enable case insensitive filtering.


Of course the Allegro Network Multimeter regular expression (RegEx) capture filter, can also be used in combination with our other capture expressions.


==''' Pcap analysis module'''==
* The expression
The pcap analysis module allows analyzing pcap files by sending them to the device. After analyzing the pcap, the web interface shows all the metadata as if the packets are live traffic at the time of the pcap recording.
:{| class="wikitable sortable"
|-
| regexp == “allegro'''<nowiki>|</nowiki>'''analyzer” and l7protocol == "dns"
|-
|}
:Will case sensitive match and capture <u>only DNS packets</u> containing the string(s) “allegro” and/or “analyzer.


* The expression
:{| class="wikitable sortable"
|-
| regexp == “allegro'''<nowiki>|</nowiki>'''analyzer” and l7protocol != "dns"
|-
|}
:Will case sensitive match and capture all (except DNS) packets containing the string(s) “allegro” and/or “analyzer.


<i>Whenever you are unsure about the outcome of RegEx based packet capturing, you can pre-test the outcome of your expressions on https://pythex.org/.
While pre-testing on https://pythex.org/, avoid using the “IGNORECASE” button. Instead use the (?i) modifier for constructing case insensitive expressions, as mentioned above.
PCRE/Python based expression examples and explanations you'll find on https://www.programiz.com/python-programming/regex</i>


====Notes====
Starting pcap analyze will stop the network ports and thus the normal packet processing and forwarding is disabled. The network connections of the devices connected to the Multimeter will stop working.


All captures can be limited to any amount of time or bytes, for example to capture only one minute or one megabyte of traffic.


Below the list of filter criteria, you will find the button to actually start (or stop) the capture. In case the filter expression is invalid, the button is disabled.


==== Start new Upload====
====Layer 7 protocol capture====
To select a file to analyze, simply drag a file from your file manager to the drop zone. The second option is to click into the drop zone. After a click, a file selection dialog will open.
Layer 7 protocol detection engine may need several packets to recognize the currently used protocol. For these captures all not yet recognized packets will be skipped. As soon as the protocol recognition is finished, all packets matching the protocol filter will be captured.
After selecting a file, the name and the size of the pcap will be displayed in the drop zone box.  


To proceed, press the “Upload and analyze pcap” button. A modal dialog will open.
=== Configuration settings ===
By clicking on the gear button on the top right of the Capture web page, you can access the configuration section.


* A warning will be shown if the device is in bridge mode, since no more packets will be forwarded when startin pcap analyze mode.
*The maximum number of concurrent capture sessions
* If a packet ring buffer is configured, it is possible to write packets to it. This allows simple extraction of packets as in live packet processing.
:Per default out-of-box setting (factory default), all Allegro Network Multimeter models allow up to 4 concurrent captures. As of FW ≥ V3.4, the number of allowed concurrent captures, can be increased up to 64, depending on model and available RAM. Note, that Increasing the number of allowed parallel/concurrent captures will decrease memory capacity for the in-memory DB, therewith decreasing the maximum timeframe of statistics held and presented in the web-interface. If the memory usage is too high, the number of parallel captures might be lower/limited by the Allegro itself.


The pcap file itself will not be stored on the storage of the Multimeter (except in the packet ring buffer, if activated in the upload modal dialog).
*Split PCAP file after this size
: This option can be used to limit the size of the PCAP file when storing to an attached device. Once the captured traffic would exceed this threshold, a new PCAP file with the current time stamp is created and the traffic is written to the new file. If the time stamp is still the same, an index is attached to the filename.
: When set to 0, no splitting will be done.


*Split PCAP file after this duration
:This option can be used to limit the duration of the PCAP file when storing to an attached device. The duration starts counting with the start of the capture. Once the captured traffic would exceed the duration, a new PCAP file with the current time stamp is created and the traffic is written to the new file.
:When set to 0, no splitting will be done.
:Both split parameters can be combined. The PCAP file will be split as soon as one threshold has been reached.


*The maximum number of concurrent packet ring buffers
:This option is used to specify how many cluster packet ring buffers can run in parallel.
:Be aware that each cluster will have it's own queue in memory and therefore the memory required is the number of cluster packet ring buffers multiplied by the setting of '''The size in MB for the queue of the packet ring buffer'''.
:A reboot of the device or a restart of the processing is needed for a change to this option to take effect. The maximum number of concurrent packet ring buffers is 8.


==== PCAP analysis statistics====
*The size in MB for the queue of the packet ring buffer
After the upload started, a progress section will be displayed. This includes a progress bar and the time of the last
: This option allows to configure the size of the queue that holds processed packets before they are written to the packet ring buffer. Increasing the size of this queue may help if the disk used for the packet ring buffer cannot keep up with bursts of traffic so that packet drops occur in the packet ring buffer.
processed packet. When viewing the progress bar on a different tab or on a different browser, the progress bar
:Be aware that memory allocated to this queue is not available for storing statistics and metadata so that choosing a large value for this queue reduces the overall data storage time.
will not show the correct value.
:Most users will not need to change this value from the default value.
:A reboot of the device or a restart of the processing is needed for a change to this option to take effect.


*The maximum size in MB for the packet reorder buffer when capturing from the packet ring buffer
:This setting allows to choose the maximum size that the packet reorder buffer may grow to. For performance reasons the packet ring buffer does not ensure a total order of packets when storing them on disk. The packet reorder buffer is used to restore the correct order of packets in a capture when capturing from the packet ring buffer. A larger packet reorder buffer makes it more likely that the packet order can be restored for all packets. The actual amount of memory used for the packet reorder buffer depends on this setting but also on the amount of free memory in the system so that the effectively used amount of memory may be less than this setting indicates.


=== Capture settings dialog ===
[[File:Choose capture settings.png|none|thumb|600x600px]]
This dialog appears after a capture button has been clicked. Following settings are possible:
*Start time and end time
:By clicking on the input field or on the calendar icon you can choose the start and end time of the capture. The input field is also editable with keyboard and allows entering a time on a second basis.
: If the start time is in the past, the complete capture is performed on the stored data of the capture ring buffer. When the capture reaches the newest packets it still continues to read from the capture ring buffer. The dialog will limit the start time input to the earliest data of the capture ring buffer. Be aware, that a possible capture ring buffer filter was applied on the past data and is also applied on future data in this mode.
: The start time may also be in the future. The capture is scheduled and starts as soon as a packet is received with a time later than the start time.
:If the whole time input field is marked and deleted, the start or end time will reset back to the default value. The default value for start time is '''now''', the capture will start with pushing the '''Start capturing''' button. The default value of the end time is '''unlimited''', the capture will not stop unless stopped manually by clicking on the stop button.
:Eight buttons offer quick selection of often used time settings.


*Packet ring buffer
:If multiple packet ring buffer clusters are active this dropdown menu allows to choose from which cluster the packets should be captured.


==== Viewing the pcap metadata====
*Capture type
During and after the upload of the file, all modules will show the metadata produced by analyzing the packets in the pcap file.
: This drop down menu allows to choose the method how packets are captured. The last successful setting is persistently stored per user. Following methods are available:


:*HTTP download
::This is the default method. The capture will start a HTTP download of a PCAP file directly in the browser.
:: Available HTTP download options:
::*Set file name: allow to configure a custom file name for the capture file
::*Download as zip archive: Download the capture file as a compressed zip archive
:*Disk
::This method is only visible if at least one storage device is active and has some amount of free storage space. The capture will create a PCAP file on the selected storage device.
::If PCAP export via SFTP is enabled, an additional checkbox is displayed to store the capture file in the export directory, slated for upload according the SFTP export settings.


:*Interface
::This mode will transmit the captured packets on a physical network interface. It is not available when the system is analyzing a PCAP file or is analyzing the packet ring buffer. The speed of the replay can be chosen below to real-time or specific speed or unlimited speed. The actual achievable speed depends on factors like the speed of the storage device. The maximum speed is typically <= 5 Gbps/400 kpps (also depending on the interface speed).


:* ERSPAN
::This mode will transmit the captured packets encapsulated in a GRE + ERSPAN header on the management interface to a given target IP address. On the target system the  traffic can be  selectively captured using the filter '''ip proto 0x2f''' when using an application like Wireshark or tcpdump.


==== Resuming normal operation====
:* Capture to SFTP server
After finishing the analysis, the processing can be set back to live mode by clicking the “Resume normal operation” button at the bottom of the page.
::This mode will stream the PCAP to the selected SFTP server using the given credentials. SFTP servers can be configured on the Capture page


*Storage device
:If the 'disk' capture type is chosen, this drop-down menu allows to choose one of the active storage devices. The capture will be stored on the selected device.


==''' Incidents module'''==
*File name
The Incidents module allows for notifications to be created when certain network incidents are detected. These notifications can be viewed in the web GUI and may also be delivered by email. Repeating incidents are counted as such and the time of the first and last occurrence of an incident is remembered. What makes an incident unique depends on the type of incident.
:If browser download or disk storage has been chosen and the "Set file name" checkbox has been selected, the file name of the created capture file can be set in this field. The default value shows the filename with date wildcards and the capture filter. The date wildcards are then replaced by the start time of the capture.  
Incidents can be configured with three levels of severity: low, medium and high. The first occurrence of a medium or high severity incident will trigger a status notification which is visible at the top right of the web GUI. Up to 1000 incidents will be remembered by the system and if this limit is exceeded the oldest incidents will be discarded.
:Date format specifier can be used as supported by strftime() function call. Some common specifier:
:*%Y for year
:*%m for month
:* %d for day
:*%H for hours
:*%M for minutes
:*%S for seconds
:Filenames are remembered when the dialog is closed and reopened. Clicking the circular arrow resets the filename to its default.


{|
*Storage directory
|-
:If disk storage has been chosen, the target directory on the storage device can be set in this field. Sub-directories on the storage device can be created and inspected on the [[Storage#Working with storage contents|Storage]] page.
| [[File:Incidents module.png|1000px|thumb|right]]
|}


==== Types of incidents ====
* Zip options
The following list shows which types of incidents can currently be detected and how they are triggered.
:If browser download has been chosen and the zip download option is selected, the file size can be configured after which the pcap files within the archive is spit into additional files
[[File:Incidents module1.png|800px|frame|right]]
:The number is entered in megabytes. 0 means no splitting.


==== MAC incidents ====
*Interface to transmit on
* '''new MAC''' : report an incident when a unicast Ethernet MAC address is seen for the first time.
:This dropdown menu is only shown when Capture type is Interface. Here the physical interface on which to transmit captured packets can be selected.
* '''new DPI protocol for MAC''': report an incident when a layer 7 protocol is first detected for a unicast Ethernet MAC address.
* '''broadcast packet rate exceeded threshold''': report an incident when the number of broadcast packets within  the duration of one second exceeds the configurable threshold


* ERSPAN target address
:This section is only shown when Capture type is ERSPAN. Here the target IP address or hostname for the ERSPAN encapsulated packets must be specified.


==== ARP incidents ====
*ERSPAN session ID
* '''ARP responses with different MACs for the same IP within 60 seconds''': report an incident when within the duration of 60 seconds two different unicast Ethernet MAC addresses respond as having the same IP address through ARP (address resolution protocol) messages. This may point to a configuration issue as two devices try to use the same IP address.
:This section is only shown when Capture type is ERSPAN. The ERSPAN session ID can be used to differentiate between multiple capture session on the ERSPAN target.


*Transmit speed
: This dropdown menu is only shown when the Capture type is either Interface or ERSPAN and the start time is in the past so that packets are captured from the packet ring buffer. Here the limiting mode can be chosen which controls how fast captured packets are transmitted. Following modes are available:


==== IP incidents ====
:*none
* '''new local IP address''': report an incident when an IPv4 address belonging to a private network address range is seen for the first time.
::No limit will be applied and the packets are transmitted as fast as the network interface and the packet ring buffer allow.
* '''new DPI protocol for local IP''': report an incident when a layer 7 protocol is first detected for an IPv4 address belonging to a private network address range.
* '''local IP address on multiple Ethernet MACs''': report an incident when an IPv4 address belonging to a private network address range is seen with multiple Ethernet MAC addresses. This may point to a configuration issue as two devices try to use the same IP address.
* '''TCP handshake time exceeded threshold''': report an incident when the time needed for the completion of a TCP handshake by a server exceeds the configurable threshold. If the TCP handshake time suddenly rises this may point e.g. to an overload of the server.
* '''TCP zero window packet''': report an incident when a TCP zero window packet is seen. This means that the receive buffer for the connection at the IP sending the TCP zero window packet is full.


:*limit to bandwidth
::A bandwidth limit will be applied so that the given bandwidth in Mbps is not exceeded. The bandwidth can be given as a decimal so that e.g. 500kbps can be configured with a value of 0.5.


==== DNS incidents ====
:*realtime factor
* '''DNS server stopped responding''': report an incident if more than 3 requests to the DNS server went unanswered for a period of more than 5 seconds.
:: Packets will be transmitted based on their recorded timing information. This means that with a real time factor of 1.0 packets will be transmitted approximately with the same timing as they were originally received. Using for example a real time factor of 2.0 would transmit the packets with twice the speed than they were received.


* Transmit bandwidth in Mbps
:This is only shown when limit to bandwidth has been selected in the Transmit speed dropdown menu. The meaning of this value is explained in the Transmit speed section.


==== Global incidents ====
*Transmit realtime factor
* '''interface link status changed''': Report an incident if the Ethernet link status of the network interface changed. This incident is always reported as a new incident even for the same network interface.
:This is only shown when realtime factor has been selected in the Transmit speed dropdown menu. The meaning of this value is explained in the :Transmit speed section.
* '''Interface link speed changed''': Report an incident if the link speed of the network interface changed.
* '''Interface pair link speed and duplex mismatch''': Report an incident if the link speed or the duplex mode of two corresponding mutual interfaces in bridge mode are different.
* '''Bandwidth below lower threshol'''d: Report an incident when the measured bandwidth falls below a certain threshold. The threshold is configured in Mbit/s. The incident is active until the bandwidth is above the threshold again.
* '''Bandwidth above upper threshold''': Report an incident when the measured bandwidth exceeds a certain threshold. The threshold is configured in Mbit/s. The incident is active until the bandwidth falls below the threshold again.
* '''Packet rate below lower threshold''': Report an incident when the measured packet rate falls below a certain threshold. The threshold is configured in packets/s. The incident is active until the packet rate is above the threshold again.
* '''Packet rate above upper threshold''': Report an incident when the measured packet rate exceeds a certain threshold. The threshold is configured in packets/s. The incident is active until the packet rate falls below the threshold again.
* '''Timeout for finishing an active bandwith incident''': Defines the time for how long the bandwidth or packet rate has to be above the lower (or below the upper) threshold again to end the incident. By using this setting, e.g. a traffic burst that is constantly moving around the threshold within this configured range can be reduced to just one incident.


*Truncate packet length:
:This dropdown menu is only shown when the Capture type is either HTTP or disk. You can truncate captured Packets with this setting. All packets will be captured, but truncated to the given length if they are longer than this setting. The length setting is applied on layer 2 without frame check sequence.
:Possible values are:
:*Full length
:*64 Bytes
:*1500 Bytes
:*Custom length with an input field for inserting any length between 64 and 15378 Bytes
:*Capture profile: select a configured capture profile which defines rules about how many bytes are saved depending on the configured rules.


==== Interface throughput ====
*PCAP compatibility:
Interface throughput incidents are generated by the throughput measurement module as soon as a configurablethreshold exceeds. The incident contains a graph of traffic for that interface with some data points before and after the threshold has been exceeded depending on the measurement interval. A PCAP link for capturing from the packet ring buffer is shown. For further investigation of that incident, the button Use as global time range can be used to set the global range to the start and end of the incident graph (at least 5 seconds) so that all modules of the Allegro Network Multimeter show that time span. The incident generation can be configured as follows:
:This section is only shown when the Capture type is either HTTP or disk.
* '''throughput threshold exceeded''': report an incident if the throughput of any network interface exceeded.
:*Omit interface ID
* '''Throughput threshold (Mbit/s)''': The threshold is configured in Mbit/s.
::Enabling this option will generate a PCAP file that only contains a single interface and treats all packets as if they arrived on that interface. This may improve compatibility with third party software that cannot handle PCAPs with multiple interfaces IDs.
* '''How long throughput must be above threshold to generate incident (in milliseconds)''': The throughput must exceed the threshold for this duration in order to generate the incident. If set to zero (defualt) the incident is generated immediately after the threshold has been exceede.
* '''Throughput cool-down period between two incidents in milliseconds''': Defines the time after an incident where no new incident is generated even if the threshold is exceeded. If this period is passed, throughput incidents could be generated again.


*PCAP comment:
:Allows to add a user defined comment to the generated PCAP file.
:*Add device information to comment
::Enabling this option will insert additional device information such as name, serial, memory size or capture filter into he PCAP comment.


==== Incident settings ====
*PCAP packet type:
:This dropdown menu allows to choose the datalink packet type of the PCAP file. It is possible to choose between capturing regular Ethernet packets or raw Radiotap IEEE 802.11 WiFi packets in some places.
:Possible values are:
:*Ethernet
:*Radiotap


'''Configuring incidents'''
* Anonymization
: This option allows enabling or disabling anonymization of the downloaded PCAP file by either apply generic settings or choosing a custom anonymization profile.
: See [[Capture_module#PCAP_anonymization_profile|PCAP anonymization]] for details.


The configuration of incidents is done in the Incidents tab on the settings page located under Settings → Incident settings. An incident detection is enabled by setting the reporting severity to a value different from disabled. Some incidents also have further configuration options like e.g. a threshold value. These options are located below the incident’s reporting severity setting. When finished configuring the incidents the Save Settings button on the bottom of the Incidents settings page will commit the changes.
After pushing the '''Start capture''' button, the capture starts.


=== Webshark ===
The Allegro Nework Multimeter has a preview mode to see the first Megabyte of captured packets directly in the browser. By clicking the Webshark preview button in the capture dialog, the first Megabyte of the requested packets will be extracted. If this is extraction is finished, a modal dialog will open showing the captured packets similar to Wireshark. The capture can be moved from the modal dialog to a separate window by pressing the button in the upper right corner next to the close button.


==== Incident email notifications ====
=== Snort ===
Email notifications for incidents can be configured in the Notification settings tab on the settings page located under Settings → Incident settings:
The Allegro Network Multimeter is able to run a Snort analysis on the capture output and display the analysis output in the web browser. To do this, enable the feature in the [https://allegro-packets.com/wiki/Global_settings#Snort_analysis Global settings] and configure it accordingly. The Snort analysis button can be found at the bottom right of the capture modal.
* '''Enable email incident notifications''': turns the feature on or off.
* '''Severity threshold''': defines the minimum severity an incident must have to trigger an email notification.


=== Capture URL ===
It is possible to use an external tool like '''curl''' for creating and storing a PCAP.
{| class="wikitable sortable" 
|-         
| curl -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/data/modules/capture?startTime=1517306266000000&endTime=1517309267000000&expression=l7Protocol==HTTP&snapPacketLength=65535&fromCaptureBuffer=true' > path_to/capture.pcap
|-
|}
The user name, password and hostname similar to the access of the web interface have to be used.
Following parameters are possible:


==== Viewing and deleting incidents in the web GUI ====
* startTime: The start time of the capture. The first packet with exactly this or a later time will start the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch). If the start time is in the past, make sure you set fromCaptureBuffer parameter accordingly.
Incidents that have occurred can be viewed and deleted on the page located under Generic → Incidents. Here it is possible to filter incidents by their severity using the colored buttons on the top of the page. In the incidents table the incidents can also be filtered by searching for a text in their subject message as well as sorted by their severity or time of last occurrence. The Delete button at the end of each line will discard a single incident and the Delete all incidents button at the top of the page will discard all incidents. If an incident was discarded and is reported again it will be treated as a new incident and e.g. an email notification will be generated again. When the subject of an incident is clicked the detail page for the incident will be displayed. This page contains some more detailed information about the incident as well as links to statistics that may be relevant for investigating the incident.
*endTime: The end time of the capture. The first packet with exactly this or a later time will stop the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch).
*expression: The filter expression. There are no whitespaces allowed. You may use ‘%20’ instead.
*snapPacketLength: The max size of a packet applied on layer 2 without frame check sequence. If a packet is larger than this value, it is truncated. Use 65535 for unlimited size.
* fromCaptureBuffer: Whether to extract data from the packet ring buffer or just live traffic.
*captureToMedia: Whether to store PCAP on external storage device or download with your browser on your computer.
* useSingleInterface: Whether to store only a single interface in the PCAP and treat all packets as if they arrived on that interface. This may improve compatibility with third party software that cannot handle PCAPs with multiple interfaces IDs.

Latest revision as of 07:03, 25 July 2025

Capture module

The Allegro Network Multimeter allows direct capturing of network traffic as a HTTP download to your computer. No packet data is stored on the device itself. Traffic can be directly filtered for specific packets, only the relevant packets will be captured. In addition, it is also possible to capture network traffic to an attached storage device, see the settings section below for details. Capturing network traffic is usually started by clicking on a PCAP button in a certain module. These buttons allow capturing specific traffic, for example for a certain IP address or a network protocol. The capture module allows to configure filter for traffic that has not even started right now, for example for an IP address that is not in use at the moment but might be used later. The capture module page displays all currently running captures and allows starting new captures with specific filters.

Generic modules.png

Current captures

The first part of the page displays all downloads running for the current user session, and all downloads running for other user sessions (like when a download has been started outside the browser by directly using command line tools such as wget or curl). The list contains the client IP and port of the user running the download.

The next three counters describe:

  • the number of packets captured for the corresponding filter.
  • the number of packets dropped by the capturing module. Packet drops can appear when doing live capture and more packets are captured than can be transferred via HTTP to the client, or the storage is not capable of storing all matching packets. During live capture, the drop counter is the exact number of packets matching the filter but were dropped because of the reasons mentioned previously. Packet drops are also accounted when capturing from the past out of the ring buffer. It happens when the ring buffer dropped packets during the capture interval due to insufficient bandwidth available on the storage devices. In retrospective capturing, the drop counter only indicates that some packets may have been missed. The counter is the total amount of packets not available in the ring buffer, which are not necessarily part of the capture filter.
  • the number of ignored packets. Ignored packets do not match the given capture filter.

The following columns list the applied filter criteria. The last column contains a button to stop the corresponding download. Downloads can also be stopped by clicking the same capture button that started the capture in the corresponding module. If multiple devices have been configured, the list also contains all captures from all multi-devices which can be stopped individually.

Finished captures

This list shows the last captures that have finished. It shows up to 100 entries while the oldest entries are discarded when the list is full. For each entry the following information is displayed:

  • the packet capturing mode
  • the type of capture
  • the start- and end-time of the capture (the actual real-time when the capture was initiated and ended)
  • the number of captured/dropped/ignored packets during the capture
  • the amount of data generated for the capture (along with the average throughput of e.g. the capture download)
  • the capture filter which was used for the capture
  • the finishing reason of the capture job (e.g. completed normally, stopped by user or aborted due to no space left on storage)

The button Delete list of finished captures will delete all entries from this list.

Recent filters

This list shows the most recently used capture filters for the current user. The most recent capture filter is displayed on the top. Next to each capture filter there is a button to permanently save this capture filter as a favorite as well as a button to simply start a capture with this filter again. The Use as expert filter button will copy the capture filter into the expert filter input field and allows for customizing the capture. The button Delete list of recent filters will delete all entries from this list.

Favorites

This list shows favorite capture expressions. A capture can be marked as a favorite either in the capture dialog by clicking on the star button in the top right corner or by marking it as a favorite in the Recently captured list. A description can be given and will be displayed in this list. For each favorite capture a PCAP button is available to simply start this capture again. The Remove favorites button allows for cleaning the list. The Use as expert filter button will copy the capture filter into the expert filter input field and allows for customizing the capture.

Planned captures

On this tab captures to a storage device can be planned ahead of time and these captures can even be set to repeat automatically. A click on the Add planned capture button opens a dialog where the planned capture can be configured. This includes settings like the storage device to use, the start date and time of the capture, the duration of the capture, if a capture should repeat and the filter expression used during the capture. It is important to know that planned captures will not function if the chosen storage device is not active.

Capture profiles

Capture profile configuration
Edit capture profile

Capture profiles can be used in the capture dialog for custom packet truncation rules. Such profiles defines custom rules for packet snapshot length similar to ring buffer filters. This allows to use a different snapshot length for specific layer 7 protocols, if for example traffic of one protocol shall be captured completely while for another protocol only the IP header shall be captured. Such a capture profile can be selected in the capture dialog in the "Truncate packet length" select box.

Up to 10 different profiles can be configured by clicking at the "Add profile" button. The add/edit dialog allows to enter a profile name and up to 10 rules for snapshot length. Similar to ring buffer filters, the first rule that matches for the selected type is used to decide for the snapshot length of the actual packet.

To restrict the capture possibilities of an user it is possible to choose an 'restricting profile'. This allows the administrator to stop the user from capturing other sensitive data like SIP- and RTP-Packages.

PCAP anonymization profile

Anonymization profiles can be used in capture dialog to allow for quickly switch between rules to anonymize aspects of the generated PCAPs.

When enabled, following options are available (more than one option is possible):

  • MAC addresses on L2
MAC addresses on L2 will be replaced by random addresses octet-wise. Multicast/broadcast addresses will not be randomized.
  • IP addresses on L3
IP addresses on L3 will be replaced by random addresses octet-wise for IPv4 and hextet-wise for IPv6. Multicast/broadcast addresses will not be randomized. The octets of the IP address will have the same length in textual representation (e.g. 100.20.3.40 -> 105.31.6.41). For IPv6 address short notation will be considered and the randomized result will also have the same textual length.
  • IP addresses on L7
IPv4 and IPv6 addresses in textual representation in L7 payload will be randomized similar to L3.
  • Mapped IP addresses in STUN packets on L7
STUN payload IP addresses will be randomized similar to L3.
  • Phone numbers, name and Call ID in SIP packets on L7
SIP payload data is masked with 'xxx' values for the names and phone numbers in the fields "From", "To", "Contact", "P-Asserted-Identity", "P-Preferred-Identity". Call IDs, usernames and sip.instance values are also replaced. IP addresses are not touched, if they shall be anonymized, please use option "IP addresses on L7".
  • URLs and HTTP hostnames on L7
URLs and HTTP hostnames in L7 payload are masked with 'xxx' values. The length of the masked name/URL will stay the same and line feeds won't be touched.
Examples:
'GET /website.html?param1=value HTTP/1.1\r\n' will be changed to 'GET xxxxxxxxxxxxxxxxxxxxxxxxxx HTTP/1.1\r\n'
'Host: allegro-packets.com\r\n' will be changed to 'Host: xxxxxxxxxxxxxxxxxxx\r\n'
https://www.allegro-packets.com/en/ will be completely masked

Within an anonymization profile it is also possible to define MAC and IP lists with entries that should be anonymized or that should be excluded from anonymization. The proper L2/L3 anonymization option must be turned on in order to have an effect!

The amount of SIP phone numbers last hidden digits can be configured, e.g. with a setting of 4 last hidden digits a phone number +49123456789 becomes +4912345xxxx.

Address anonymization is stable for the whole PCAP, i.e. the same addresses will be replaced by the same random addresses. As an example, if both randomization of IP addresses on L3 and L7 is active and a SIP call with RTP is captured, both IP addresses in L3 and SIP SDP payload are replaced by the same values so that the correlation of the RTP stream is still intact.

Checksums of the changed packets are not being recalculated.

A privileged user may enforce an anonymization profile that is being used for all users that do not have the unrestrictive capture permission in a role. This profile will always be taken into account and the unprivileged user may only add additional anonymizations on top of it for captures (but the user can't overwrite it with less restrictive settings).

SFTP server

Credentials and upload directories for a SFTP server can be configured here. They can be selected in the capture dialog later when choosing "Capture to SFTP server" as capture type. It is possible to either configure a user and password or use authentication with a public key. In the latter case, the displayed SSH public key needs to be inserted in the authorized hosts list on the SFTP server.

Simple capture

The second section of the capture page allow to select some fields to filter network traffic for. By default, only the IP field is visible, the other fields can be enabled by clicking on the corresponding toggle switch. Each line allows to enter a filter criterion for the corresponding network traffic element. To start the capture with the entered filter criteria just click at the Start capture button. For reference, the expert filter expression is shown at the end of the section so it can be used to copy and paste the string into the expert filter section.

Using expert filters to start captures

The third part of the page allows for starting a download for any criterion combination using complex filter expressions. A capture filter is defined in a C-style syntax and supports combination of AND/OR operators, precedence order with parentheses and equal/not equal comparisons. If the filter exp Session can be evaluated to true, the packet is captured. If a value contains a space, the whole value must be quoted with “”. Following operators are supported:

  • and, && : AND operator. The filter expression will match if all operands could be evaluated to true.
  • or, ||: OR operator. The filter expression will match if any operand can be evaluated to true.

Following comparison operators are supported:

  • ==: Will evaluate expression to true if left and right operand are equal.
  • !=: Will evaluate expression to true if left and right operand are not equal.

Following operands are supported:

  • ip: An IP address. The packet is captured if either source or destination IP address of the packet match. A netmask and a port can also be specified. For IPv6 addresses with a specific port, the address must be written in brackets.
Example:

ip == 10.0.0.1

ip == ff02::1:3

ip == 10.0/16

ip == 10.0.0.1:1234

ip == [2a02:810a:1340:1292:1c6b:e58d:6ebc:6cd2]:123

  • ip.src, ip.dst: The source or destination IP address.
Example:

ip.src == 10.0.0.1/8 and ip.dst == 10.0.0.1/8

This will match traffic only within 10.0.0.1/8 subnet. If src/dst is omitted, also in or outbound traffic from or to other subnets is captured!
  • mac: A MAC address. The packet is captured if either source or destination MAC address of the packet match.
Example:
mac == 12:34: :56:78:90:ab
  • port: A TCP or UDP port. The packet is captured if either source or destination port match.
Example:
port == 80
  • portrange: A TCP or UDP port range. The range can be a single number or a comma separated list of values or value ranges.
Example:
portrange == 80,100-120,-10,65000-
  • serverport: A TCP or UDP port of a server. The packet is captured if the given port is a port of the server and not of a client. The range can be a single number or a comma separated list of values or value ranges.
Example:
serverport == 80,100-120,-10,65000-
  • macProtocol: A MAC protocol such as IPv4 or IPv6. For all seen MAC protocols, please consult the MAC Protocol Statistics module.
Example:
macProtocol == IPv4

macProtocol == "Non IP"

  • l4Protocol: A layer 4 protocol such as TCP or UDP or any IP protocol number. Protocols can also be OR combined as a comma seperated list.
Example:
l4Protocol == ICMP,ICMPv6
  • l7Protocol or dpiProtocol: A layer 7 protocol. Protocols can also be OR combined as a comma seperated list. For all seen protocols please consult the Layer 7 protocols module.
  • countryCode: A country code such as US. For all seen country codes please consult the Geolocation module.
  • arpip: An IP address within an ARP request or response.
  • vlan: A VLAN tag of an outer or inner VLAN. May be a number or none or any.
  • outervlan: A VLAN tag of an outer VLAN. May be a number or none or any.
  • innervlan: A VLAN tag of an inner VLAN. May be a number or none or any.
  • multicastGroup: A multicast IP address or any. The filter will match all IGMP or MLD negotiation packets related to that multicast IP address.
  • rtpPayloadType: The RTP payload type such as PCMU or MP2T. This filter will match all RTP packets with the given payload type.
  • interface.name or namedInterface: The physical interface. This can be the name or interface number (as printed on the device). For interface ids please consult the Interface stats page.
Example:
interface.name == uplink || interface.name == 3
  • interface.rawid: This is the raw internal ID of interfaces. It starts from 0. The interface stats page also shows the raw ID.
  • link: The link pair of two interfaces as stated in Interface stats. A single link number can be given.
  • pppoeProtocol: A specific PPPoE protocol (by name or by ID), or "none"/"any"
  • pppoeAuthentication: An authentication state of a PPPoE session
  • pppoeLinkConfiguration: A link configuration state of a PPPoE session
  • ptpMsgType: A specific PTP message type number or any for the whole PTP traffic.
  • profinetFrameId: A specific Profinet frame ID.
  • profinetCmOpnum: A specific operation number for Profinet CM (Context Manager) requests or responses. Can also be any for every operation number. Following values are used:
  1. connect
  2. release
  3. read
  4. write
  5. control
  6. read implicit
  • mpls: A label of an outer or inner MPLS. May be a number or none or any.
  • outerMpls: A label of an outer MPLS. May be a number or none or any.
  • innerMpls: A label of an inner MPLS. May be a number or none or any.
  • qosIpDscp: The DSCP value in the IP header. May be a number.
  • qosMplsTc: The traffic class value in the outermost MPLS label stack entry.
  • qosVlanPcp: The priority code point value in the outermost VLAN tag.
  • group: The name of a configured group or ‘default’. If the name contains whitespaces, the name must be enclosed in quotes.
  • badCRC: The value of this operand will be 1 for packets with a CRC error and will be 0 for good packets. Capturing packets with bad CRC is currently only supported on 1Gb interfaces.
  • icmpType: The value of a certain ICMP type (e.g. Echo request 8, Echo reply 0).
  • dhcpMessageType: The value of a certain DHCP message type (e.g. Request 3, ACK 5).
  • tcpFlags: A single TCP flag or a list of TCP flags joined by the ‘+’ sign. If a list of flags is given, all flags must be present in the packet. Supported TCP flags are syn, ack, fin, rst, psh and urg.
  • callId: The string value of a SIP call ID or similar identifier (e.g. P-Palladion-ID)
  • ipFragment: If set to 1 all IPv4 fragments will be captured (i.e. packets having the 'More fragments' flag and 'Fragment offset' set). If set to 0 all packets without IPv4 fragmentation will be captured.
  • regexp: The packet payload matches the quoted regular expression (RegEx) to the other side of the == operator or does not match the regular expression to the other side of the != operator. In case of IP packets the matching will be performed on the L7 payload of the packet. In case of non-IP packets the matching will be performed on the whole packet except the Ethernet header. Regular expressions largely support the pattern syntax used by the PCRE library with the exception of certain constructs. An invalid pattern will produce a descriptive error message and prevent the capture from being started.
  • ipDoNotFragment: The value of this operand will be 1 for IPv4 packets that are marked as to not be fragmented (packets which have the 'do not fragment' flag set).
  • mactype: The type of the packets destination MAC address. Allowed values are 'unicast, 'broadcast' and 'multicast'.
  • slowSubtype: The subtype of packets with the macProtocol == SLOW (e.g. 1 for LACP).
  • wifiFrequency: The frequency (channel) of a WiFi packet in MHz.
  • wifiBssid: The BSS ID participating in a WiFi packet. This is similar to a MAC address.
  • callNumber: SIP caller (SIP from tag) or callee (SIP to tag) number (firmware >= 4.5)
  • callerNumber: SIP caller (SIP from tag) number (firmware >= 4.5)
  • calleeNumber: SIP callee (SIP to tag) number (firmware >= 4.5)

For a specific precedence you may use parentheses (/).

Examples:

  • The expression
ip == 10.0.0.1:1234 and ip == 10.1.0.1:9876
will match a connection from 10.0.0.1 to 10.1.0.1 or vice versa with the ports 1234 and 9876 involved.
  • The expression
ip == 10.0.0.1 and ip != 10.0.0.2
will match packets having 10.0.0.1 either as source or destination. If a communication peer of 10.0.0.1 is 10.0.0.2 the packets will not be captured.
  • The expression
l4Protocol == ICMP,ICMPv6
will match packets with ICMP or ICMPv6 layer 4 protocols.
  • The expression
portrange == 80,443
will match packets to or from port 80 or 443.
  • The expression
regexp == "allegr[o,a]|HTTP"
will case sensitive match packets that contain the string(s) 'allegro' and/or 'allegra' and/or 'HTTP' anywhere in the payload.
NOTE: The use of regexp is CASE sensitive. You must use the (?i) modifier to enable case insensitive filtering.
  • The expression
regexp == "(?i)allegro|http"
will case insensitive match packets that contain the string(s) 'allegro' and/or 'http' anywhere in the payload.
NOTE: The use of regexp is CASE sensitive. You must use the (?i) modifier to enable case insensitive filtering.

Of course the Allegro Network Multimeter regular expression (RegEx) capture filter, can also be used in combination with our other capture expressions.

  • The expression
regexp == “allegro|analyzer” and l7protocol == "dns"
Will case sensitive match and capture only DNS packets containing the string(s) “allegro” and/or “analyzer.
  • The expression
regexp == “allegro|analyzer” and l7protocol != "dns"
Will case sensitive match and capture all (except DNS) packets containing the string(s) “allegro” and/or “analyzer.

Whenever you are unsure about the outcome of RegEx based packet capturing, you can pre-test the outcome of your expressions on https://pythex.org/. While pre-testing on https://pythex.org/, avoid using the “IGNORECASE” button. Instead use the (?i) modifier for constructing case insensitive expressions, as mentioned above. PCRE/Python based expression examples and explanations you'll find on https://www.programiz.com/python-programming/regex


All captures can be limited to any amount of time or bytes, for example to capture only one minute or one megabyte of traffic.

Below the list of filter criteria, you will find the button to actually start (or stop) the capture. In case the filter expression is invalid, the button is disabled.

Layer 7 protocol capture

Layer 7 protocol detection engine may need several packets to recognize the currently used protocol. For these captures all not yet recognized packets will be skipped. As soon as the protocol recognition is finished, all packets matching the protocol filter will be captured.

Configuration settings

By clicking on the gear button on the top right of the Capture web page, you can access the configuration section.

  • The maximum number of concurrent capture sessions
Per default out-of-box setting (factory default), all Allegro Network Multimeter models allow up to 4 concurrent captures. As of FW ≥ V3.4, the number of allowed concurrent captures, can be increased up to 64, depending on model and available RAM. Note, that Increasing the number of allowed parallel/concurrent captures will decrease memory capacity for the in-memory DB, therewith decreasing the maximum timeframe of statistics held and presented in the web-interface. If the memory usage is too high, the number of parallel captures might be lower/limited by the Allegro itself.
  • Split PCAP file after this size
This option can be used to limit the size of the PCAP file when storing to an attached device. Once the captured traffic would exceed this threshold, a new PCAP file with the current time stamp is created and the traffic is written to the new file. If the time stamp is still the same, an index is attached to the filename.
When set to 0, no splitting will be done.
  • Split PCAP file after this duration
This option can be used to limit the duration of the PCAP file when storing to an attached device. The duration starts counting with the start of the capture. Once the captured traffic would exceed the duration, a new PCAP file with the current time stamp is created and the traffic is written to the new file.
When set to 0, no splitting will be done.
Both split parameters can be combined. The PCAP file will be split as soon as one threshold has been reached.
  • The maximum number of concurrent packet ring buffers
This option is used to specify how many cluster packet ring buffers can run in parallel.
Be aware that each cluster will have it's own queue in memory and therefore the memory required is the number of cluster packet ring buffers multiplied by the setting of The size in MB for the queue of the packet ring buffer.
A reboot of the device or a restart of the processing is needed for a change to this option to take effect. The maximum number of concurrent packet ring buffers is 8.
  • The size in MB for the queue of the packet ring buffer
This option allows to configure the size of the queue that holds processed packets before they are written to the packet ring buffer. Increasing the size of this queue may help if the disk used for the packet ring buffer cannot keep up with bursts of traffic so that packet drops occur in the packet ring buffer.
Be aware that memory allocated to this queue is not available for storing statistics and metadata so that choosing a large value for this queue reduces the overall data storage time.
Most users will not need to change this value from the default value.
A reboot of the device or a restart of the processing is needed for a change to this option to take effect.
  • The maximum size in MB for the packet reorder buffer when capturing from the packet ring buffer
This setting allows to choose the maximum size that the packet reorder buffer may grow to. For performance reasons the packet ring buffer does not ensure a total order of packets when storing them on disk. The packet reorder buffer is used to restore the correct order of packets in a capture when capturing from the packet ring buffer. A larger packet reorder buffer makes it more likely that the packet order can be restored for all packets. The actual amount of memory used for the packet reorder buffer depends on this setting but also on the amount of free memory in the system so that the effectively used amount of memory may be less than this setting indicates.

Capture settings dialog

Choose capture settings.png

This dialog appears after a capture button has been clicked. Following settings are possible:

  • Start time and end time
By clicking on the input field or on the calendar icon you can choose the start and end time of the capture. The input field is also editable with keyboard and allows entering a time on a second basis.
If the start time is in the past, the complete capture is performed on the stored data of the capture ring buffer. When the capture reaches the newest packets it still continues to read from the capture ring buffer. The dialog will limit the start time input to the earliest data of the capture ring buffer. Be aware, that a possible capture ring buffer filter was applied on the past data and is also applied on future data in this mode.
The start time may also be in the future. The capture is scheduled and starts as soon as a packet is received with a time later than the start time.
If the whole time input field is marked and deleted, the start or end time will reset back to the default value. The default value for start time is now, the capture will start with pushing the Start capturing button. The default value of the end time is unlimited, the capture will not stop unless stopped manually by clicking on the stop button.
Eight buttons offer quick selection of often used time settings.
  • Packet ring buffer
If multiple packet ring buffer clusters are active this dropdown menu allows to choose from which cluster the packets should be captured.
  • Capture type
This drop down menu allows to choose the method how packets are captured. The last successful setting is persistently stored per user. Following methods are available:
  • HTTP download
This is the default method. The capture will start a HTTP download of a PCAP file directly in the browser.
Available HTTP download options:
  • Set file name: allow to configure a custom file name for the capture file
  • Download as zip archive: Download the capture file as a compressed zip archive
  • Disk
This method is only visible if at least one storage device is active and has some amount of free storage space. The capture will create a PCAP file on the selected storage device.
If PCAP export via SFTP is enabled, an additional checkbox is displayed to store the capture file in the export directory, slated for upload according the SFTP export settings.
  • Interface
This mode will transmit the captured packets on a physical network interface. It is not available when the system is analyzing a PCAP file or is analyzing the packet ring buffer. The speed of the replay can be chosen below to real-time or specific speed or unlimited speed. The actual achievable speed depends on factors like the speed of the storage device. The maximum speed is typically <= 5 Gbps/400 kpps (also depending on the interface speed).
  • ERSPAN
This mode will transmit the captured packets encapsulated in a GRE + ERSPAN header on the management interface to a given target IP address. On the target system the traffic can be selectively captured using the filter ip proto 0x2f when using an application like Wireshark or tcpdump.
  • Capture to SFTP server
This mode will stream the PCAP to the selected SFTP server using the given credentials. SFTP servers can be configured on the Capture page
  • Storage device
If the 'disk' capture type is chosen, this drop-down menu allows to choose one of the active storage devices. The capture will be stored on the selected device.
  • File name
If browser download or disk storage has been chosen and the "Set file name" checkbox has been selected, the file name of the created capture file can be set in this field. The default value shows the filename with date wildcards and the capture filter. The date wildcards are then replaced by the start time of the capture.
Date format specifier can be used as supported by strftime() function call. Some common specifier:
  • %Y for year
  • %m for month
  • %d for day
  • %H for hours
  • %M for minutes
  • %S for seconds
Filenames are remembered when the dialog is closed and reopened. Clicking the circular arrow resets the filename to its default.
  • Storage directory
If disk storage has been chosen, the target directory on the storage device can be set in this field. Sub-directories on the storage device can be created and inspected on the Storage page.
  • Zip options
If browser download has been chosen and the zip download option is selected, the file size can be configured after which the pcap files within the archive is spit into additional files
The number is entered in megabytes. 0 means no splitting.
  • Interface to transmit on
This dropdown menu is only shown when Capture type is Interface. Here the physical interface on which to transmit captured packets can be selected.
  • ERSPAN target address
This section is only shown when Capture type is ERSPAN. Here the target IP address or hostname for the ERSPAN encapsulated packets must be specified.
  • ERSPAN session ID
This section is only shown when Capture type is ERSPAN. The ERSPAN session ID can be used to differentiate between multiple capture session on the ERSPAN target.
  • Transmit speed
This dropdown menu is only shown when the Capture type is either Interface or ERSPAN and the start time is in the past so that packets are captured from the packet ring buffer. Here the limiting mode can be chosen which controls how fast captured packets are transmitted. Following modes are available:
  • none
No limit will be applied and the packets are transmitted as fast as the network interface and the packet ring buffer allow.
  • limit to bandwidth
A bandwidth limit will be applied so that the given bandwidth in Mbps is not exceeded. The bandwidth can be given as a decimal so that e.g. 500kbps can be configured with a value of 0.5.
  • realtime factor
Packets will be transmitted based on their recorded timing information. This means that with a real time factor of 1.0 packets will be transmitted approximately with the same timing as they were originally received. Using for example a real time factor of 2.0 would transmit the packets with twice the speed than they were received.
  • Transmit bandwidth in Mbps
This is only shown when limit to bandwidth has been selected in the Transmit speed dropdown menu. The meaning of this value is explained in the Transmit speed section.
  • Transmit realtime factor
This is only shown when realtime factor has been selected in the Transmit speed dropdown menu. The meaning of this value is explained in the :Transmit speed section.
  • Truncate packet length:
This dropdown menu is only shown when the Capture type is either HTTP or disk. You can truncate captured Packets with this setting. All packets will be captured, but truncated to the given length if they are longer than this setting. The length setting is applied on layer 2 without frame check sequence.
Possible values are:
  • Full length
  • 64 Bytes
  • 1500 Bytes
  • Custom length with an input field for inserting any length between 64 and 15378 Bytes
  • Capture profile: select a configured capture profile which defines rules about how many bytes are saved depending on the configured rules.
  • PCAP compatibility:
This section is only shown when the Capture type is either HTTP or disk.
  • Omit interface ID
Enabling this option will generate a PCAP file that only contains a single interface and treats all packets as if they arrived on that interface. This may improve compatibility with third party software that cannot handle PCAPs with multiple interfaces IDs.
  • PCAP comment:
Allows to add a user defined comment to the generated PCAP file.
  • Add device information to comment
Enabling this option will insert additional device information such as name, serial, memory size or capture filter into he PCAP comment.
  • PCAP packet type:
This dropdown menu allows to choose the datalink packet type of the PCAP file. It is possible to choose between capturing regular Ethernet packets or raw Radiotap IEEE 802.11 WiFi packets in some places.
Possible values are:
  • Ethernet
  • Radiotap
  • Anonymization
This option allows enabling or disabling anonymization of the downloaded PCAP file by either apply generic settings or choosing a custom anonymization profile.
See PCAP anonymization for details.

After pushing the Start capture button, the capture starts.

Webshark

The Allegro Nework Multimeter has a preview mode to see the first Megabyte of captured packets directly in the browser. By clicking the Webshark preview button in the capture dialog, the first Megabyte of the requested packets will be extracted. If this is extraction is finished, a modal dialog will open showing the captured packets similar to Wireshark. The capture can be moved from the modal dialog to a separate window by pressing the button in the upper right corner next to the close button.

Snort

The Allegro Network Multimeter is able to run a Snort analysis on the capture output and display the analysis output in the web browser. To do this, enable the feature in the Global settings and configure it accordingly. The Snort analysis button can be found at the bottom right of the capture modal.

Capture URL

It is possible to use an external tool like curl for creating and storing a PCAP.

curl -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/data/modules/capture?startTime=1517306266000000&endTime=1517309267000000&expression=l7Protocol==HTTP&snapPacketLength=65535&fromCaptureBuffer=true' > path_to/capture.pcap

The user name, password and hostname similar to the access of the web interface have to be used. Following parameters are possible:

  • startTime: The start time of the capture. The first packet with exactly this or a later time will start the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch). If the start time is in the past, make sure you set fromCaptureBuffer parameter accordingly.
  • endTime: The end time of the capture. The first packet with exactly this or a later time will stop the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch).
  • expression: The filter expression. There are no whitespaces allowed. You may use ‘%20’ instead.
  • snapPacketLength: The max size of a packet applied on layer 2 without frame check sequence. If a packet is larger than this value, it is truncated. Use 65535 for unlimited size.
  • fromCaptureBuffer: Whether to extract data from the packet ring buffer or just live traffic.
  • captureToMedia: Whether to store PCAP on external storage device or download with your browser on your computer.
  • useSingleInterface: Whether to store only a single interface in the PCAP and treat all packets as if they arrived on that interface. This may improve compatibility with third party software that cannot handle PCAPs with multiple interfaces IDs.
Retrieved from "https://allegro-packets.com/wiki/index.php?title=Capture_module&oldid=5173"