REST API description: Difference between revisions

From Allegro Network Multimeter Manual
Jump to navigation Jump to search
Access restrictions were established for this page. If you see this message, you have no access to this page.
No edit summary
Line 221: Line 221:
$ curl --silent -k -u 'admin:allegro'  "https://allegro-mm/API/stats/modules/ip/ips/10.54.0.254?starttime=$(date --date="1 day ago" +%s)&endtime=$(date +%s)&skiphistorydata=true" | jq '.interval[1] + .interval[3]'
$ curl --silent -k -u 'admin:allegro'  "https://allegro-mm/API/stats/modules/ip/ips/10.54.0.254?starttime=$(date --date="1 day ago" +%s)&endtime=$(date +%s)&skiphistorydata=true" | jq '.interval[1] + .interval[3]'
</pre>
</pre>
=== List queries ===
List queries are requested with pagination parameters to reduce the size of the resulting JSON objects and to increase performance. In the resulting JSON object, the list elements are stored in the displayedItems array.
The following list parameters are possible:
* sort: Sorting criteria for the list. Following criterias are supported for most lists:
** bytes: Received and transmitted bytes (either in selected time interval or since start of the Multimeter).
** rxbytes: Received bytes.
** txbytes: Transmitted bytes.
** bps: Received and transmitted bytes per second (either average per second value of the selected time interval or last second, if no interval is specified).
** rxbps: Received bytes per second.
** txbps: Transmitted bytes per second.
* reverse: Sort ascending (= false) or descending (= true).
* page: Requested page.
* count: Amount of entries in the list. Maximum is 100.
* values: Amount of maximal values in history object(s).
This example shows IP address with the highest amount of traffic
<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips_paged?sort=bps&reverse=true&page=0&count=1' | jq .displayedItems[0].ip</code>
This exampe shows up to 9999 peers of a specific IP address:
<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips/10.1.2.3/peers?sort=bytes&reverse=true&page=0&count=9999&timespan=60&values=100' | jq '.displayedItems[].ip'</code>




Line 281: Line 307:


<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/stats/modules/ip/ips/10.1.2.3' | jq</code>
<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/stats/modules/ip/ips/10.1.2.3' | jq</code>
==== List queries ====
List queries are requested with pagination parameters to reduce the size of the resulting JSON objects and to increase performance. In the resulting JSON object, the list elements are stored in the displayedItems array.
The following list parameters are possible:
* sort: Sorting criteria for the list. Following criterias are supported for most lists:
** bytes: Received and transmitted bytes (either in selected time interval or since start of the Multimeter).
** rxbytes: Received bytes.
** txbytes: Transmitted bytes.
** bps: Received and transmitted bytes per second (either average per second value of the selected time interval or last second, if no interval is specified).
** rxbps: Received bytes per second.
** txbps: Transmitted bytes per second.
* reverse: Sort ascending (= false) or descending (= true).
* page: Requested page.
* count: Amount of entries in the list. Maximum is 100.
* values: Amount of maximal values in history object(s).
==== Show IP address with the highest amount of traffic ====
<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/stats/modules/ip/ips_paged?sort=bps&reverse=true&page=0&count=1' | jq .displayedItems[0].ip</code>
PowerShell command:
<code>((Invoke-RestMethod -Uri 'https://allegro-mm-XXXX/API/stats/modules/ip/ips_paged?sort=bytes&reverse=true&page=0&count=10&timespan=60&values=50' -Headers @{Authorization = ("Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f 'USER', 'PASSWORD'))))} -ContentType'application/json; charset=utf-8' -Method 'Get' -SkipCertificateCheck). displayedItems[0]).ip</code>
==== Show all peers of a specific IP address ====
<code>curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/stats/modules/ip/ips/10.1.2.3/peers?sort=bytes&reverse=true&page=0&count=9999&timespan=60&values=100' | jq '.displayedItems[].ip'</code>


==== Capture a specific IP ====
==== Capture a specific IP ====

Revision as of 14:30, 12 May 2020

Warning: This text is under construction

This page describes how to access and use the REST API. It allows to post-process data with 3rd party systems. The Allegro web interface is itself based on this REST API and all displayed statistics can be extracted from the Allegro with this API.


General API Setup

REST API Interface

All Allegro Network Multimeter statistics are derived from HTTPS requests and provided as JSON objects. The requests are stateless, i.e. there are no prerequisites and there is no fixed sequence of requests necessary. Example requests related to a specific module and statistics can be seen in the web interface by opening the browser development console (Ctrl+Shift+I for Chrome and Firefox, F12 for Edge).

Here an example of the structured JSON data for the IP overview. This data has been extracted with the Google Chrome developer console while accessing the IP statsistics page.

Rest api chrome console.png

Credentials

The credentials are the same as for the web interface. The admin user allows to access all APIs. A non-admin user has read access to most of the statistics. If you have enabled the pcap role, the capture URL is also possible for the API.

Allegro recommends to set up a separate non-admin user with or without the pcap role for the REST API of only statistics shall be gathered. This will prevent to accidentally shut down or change any configuration by calling the REST API.

Useful shell commands and their parameters

Curl

Most examples are written for curl [1]. Curl is available as for many operating systems like Linux or Windows. Curl needs a few parameters for the access of the Allegro Network Multimeter:

The parameter -u allows you to set a user name and password for the request.

The parameter -k will allow self-signed certificates.

The parameter -s or --silent mutes any debugging output.

The URL of the API call is the first argument. It is recommended to enclose the API call with the character ' to avoid replacing the argument ( unless you need to replace parts of it )

curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/...'

Please note that you might need to use curl.exe in windows.

PowerShell

The Integrated Windows PowerShell can be used to access the REST API. This guide requires at least PowerShell v6.

The command to call a REST API is Invoke-RestMethod [2]. Invoke-RestMethod on the PowerShell needs a few parameters for the access of the Allegro Network Multimeter:

To set the user name for basic authorization, use the -Headers parameter:

-Headers @{Authorization = ("Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f 'USER', 'PASSWORD'))))}

You also need to announce that you accept JSON as response with:

-ContentType'application/json; charset=utf-8'

To disable the certificate check, use:

-SkipCertificateCheck

The URL must be passed with the parameter -Uri, so the full command is:

Invoke-RestMethod -Uri 'https://allegro-mm-XXXX/...' -Headers @{Authorization = ("Basic {0}" -f [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f 'USER', 'PASSWORD'))))} -ContentType'application/json; charset=utf-8' -Method 'Get' -SkipCertificateCheck

jquery ( jq )

jquery ( [3] ) is a powerful tool to extract parameters from a json document. If called without parameters, jq formats the JSON output into a readable format with indenting and new lines. It also allows to select specific values and do basic operations like addition with this values. Please read the jq documentation for more information.

API hierarchy

Query available URIs with OPTIONS

The Allegro REST API has the fixed URI API/stats for statistics. To see all possible subtrees of a specific request, use the OPTIONS request instead of GET. It can be set with the parameter -X OPTIONS for curl.

$ curl --silent -k -u 'admin:allegro'  'https://allegro-mm/API/stats' -X OPTIONS
{
  "subResources": [
    "modules",
    "reports",
    "incidentReporting",
    "time",
    "ringBufferReplay",
    "pcap",
    "reset",
    "interfacesError",
    "interfaces",
    "load",
    "processing"
  ]
}

This allows you to query for example for all modules that are available for a specific release of the Allegro:

$ curl --silent -k -u 'admin:allegro'  'https://allegro-mm/API/stats/modules' -X OPTIONS
{
  "subResources": [
    "pppoe",
    "lldp",
    "groups",
    "mpls",
    "opc_ua",
    "quality",
    "ipsec",
    "profinet",
    "multicast",
    "burst_analysis",
    "global_incidents",
    "ptp",
    "ntp",
    "icmp",
    "stp",
    "sip",
    "smb",
    "dpa",
    "l4_ports",
    "netbios",
    "crt",
    "dpi",
    "http",
    "ssl",
    "dns",
    "dhcp",
    "location",
    "qos",
    "ip",
    "mac_protocols",
    "vlan",
    "arp",
    "packet_size",
    "mac",
    "capture"
  ]
}

URI content parameters

Some modules allow to use a parameter as part of the URI like the IP or Mac address. The path API/stats/modules/ip/ips allows you to use an IP address as next uri element

$ curl --silent -k -u 'admin:allegro'  'https://allegro-mm/API/stats/modules/ip/ips' -X OPTIONS
{
  "subResources": [
    "protocol",
    ":ip"
  ]
}

The path of an IP address shows all further available elements:

$ curl --silent -k -u 'admin:allegro'  'https://allegro-mm/API/stats/modules/ip/ips/10.0.54.254' -X OPTIONS
{
  "subResources": [
    "sip_request_responses",
    "peers_ports",
    "sip_responses",
    "sip_requests",
    "qos",
    "ports",
    "connections",
    "protocols",
    "macs",
    "peers",
    "tcpStats"
  ]
}

JSON output traffic counters

All counters are aggregated counters, either for the selected time interval or since the processing start of the Allegro. Many traffic counters have 4 separate values. These traffic counters are represented as a JSON array with at least 4 lines. The structure is as follows:

  • line 1: received packets, extraction example: jq .lastSecond[0]
  • line 2: received bytes, extraction example: jq .lastSecond[1]
  • line 3: transmitted packets, extraction example: jq .lastSecond[2]
  • line 4: transmitted bytes, extraction example: jq .lastSecond[3]
  • additional lines are module specific

The following counters exist for many REST APIs like IP, MAC, l4 protocol, l7 protocol and many more:

  • interval: Values of the selected time interval. If no interval is specified, this is similar to lastSecond.
  • allTime: Values since start of the Allegro Network Multimeter.
  • lastSecond: Values of the last second.
  • intervalPerSecond: Average per second value of the selected time interval. If no interval is specified, this is similar to lastSecond.

Please note that all counters are byte counters, not bit counters. You need to multiply the counters by 8 to get the bitrate.

This example extracts the received bytes of the last second of a specific IP.

curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips/10.1.2.3' | jq .lastSecond[1]

This example extracts received and transmitted bytes of the last second of a specific IP.

curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips/10.1.2.3' | jq '.lastSecond[1] + .lastSecond[3]'

API parameters

The Allegro REST API has a number of query parameters that can be added to modify the request. By default, the API will display the real time counters since the last restart of the processing unit.

This example extracts the amount of received and transmitted bytes for an IP address since the processing start of the Allegro. It queries via the REST API the JSON and then adds the values second value ( row 1, rx bytes ) and 4th value ( row 3, tx bytes ) of the interval counters together.

$ curl --silent -k -u 'admin:allegro'  "https://allegro-mm/API/stats/modules/ip/ips/10.54.0.254" | jq '.interval[1] + .interval[3]'

Time interval selection

Requests can be given a time interval. If present, the interval counters are adjusted to this interval. The following GET parameters are necessary:

  • starttime, endtime: Start and end time of the interval. Format: seconds since 1970/01/01 UTC (Unix time, epoch). You can use date +%s on your machine to adjust to the best interval. Please consult the man page of date for more parameters.
  • skiphistorydata: shall the JSON include the history data without datasets, this reduces the amount of transferred bytes if datasets are required to render a graph, can be false/true default: false
  • timespan: required resolution for the graph dataset

This example extracts the amount of received and transmitted bytes for an IP address for the last 24 hours.

$ curl --silent -k -u 'admin:allegro'  "https://allegro-mm/API/stats/modules/ip/ips/10.54.0.254?starttime=$(date --date="1 day ago" +%s)&endtime=$(date +%s)&skiphistorydata=true" | jq '.interval[1] + .interval[3]'


List queries

List queries are requested with pagination parameters to reduce the size of the resulting JSON objects and to increase performance. In the resulting JSON object, the list elements are stored in the displayedItems array. The following list parameters are possible:

  • sort: Sorting criteria for the list. Following criterias are supported for most lists:
    • bytes: Received and transmitted bytes (either in selected time interval or since start of the Multimeter).
    • rxbytes: Received bytes.
    • txbytes: Transmitted bytes.
    • bps: Received and transmitted bytes per second (either average per second value of the selected time interval or last second, if no interval is specified).
    • rxbps: Received bytes per second.
    • txbps: Transmitted bytes per second.
  • reverse: Sort ascending (= false) or descending (= true).
  • page: Requested page.
  • count: Amount of entries in the list. Maximum is 100.
  • values: Amount of maximal values in history object(s).

This example shows IP address with the highest amount of traffic

curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips_paged?sort=bps&reverse=true&page=0&count=1' | jq .displayedItems[0].ip

This exampe shows up to 9999 peers of a specific IP address:

curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips/10.1.2.3/peers?sort=bytes&reverse=true&page=0&count=9999&timespan=60&values=100' | jq '.displayedItems[].ip'


Pcap parameters

The Allegro Network Multimeter allows to extract the raw packets with the REST API with the special capture URI /API/data/modules/capture

curl -k -u USER:PASSWORD 'https://allegro-mm/API/data/modules/capture?expression=ip==10.1.2.3' > path_to/capture.pcap

The available parameters are:

  • startTime: The start time of the capture. The first packet with exactly this or a later time will start the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch). If the start time is in the past, make sure you set fromCaptureBuffer parameter accordingly. If not specified, the current time will be used.
  • endTime: The end time of the capture. The first packet with exactly this or a later time will stop the capture. The time format must be microseconds after January, 1st 1970 UTC (Unix time, epoch). If not specified, unlimited will be used.
  • expression: The filter expression. There are no whitespaces allowed. You may use ‘%20’ instead. See Capture module for available expressions.
  • snapPacketLength: The maximum size of a packet applied on Layer 2 without frame check sequence. If a packet is larger than this value, it is truncated. Use 65535 for unlimited size.
  • fromCaptureBuffer: Whether to extract data from the packet ring buffer (= true) or just live traffic (= false).
  • captureToMedia: Whether to store a pcap on an external storage device (= true) or download to your computer (= false).

Example to capture everything from now on:

curl -k -u USER:PASSWORD 'https://allegro-mm/API/data/modules/capture' > path_to/capture.pcap

Example to capture a specific IP of the last hour

curl -k -u USER:PASSWORD "https://allegro-mm/API/data/modules/capture?expression=ip==10.1.2.3&starttime=$(date --date="1 hour ago" +%s)&endtime=$(date +%s)&fromCaptureBuffer=true" > path_to/capture.pcap

Virtual Link Groups

The Allegro REST API allows to access all link groups by the parameter group. The group index starts at zero, which is the default value. If a virtual link group is enabled.

This example extracts the traffic of the IP 10.54.0.254 from the second virtual link group ( index 1 ).

$ curl --silent -k -u 'admin:allegro'  "https://allegro-mm/API/stats/modules/ip/ips/10.54.0.254?group=1" | jq '.interval[1] + .interval[3]'

Parallel pcap analysis

The Allegro can process in parallel offline traffic like a pcap file or a ring buffer. In case a parallel PCAP analysis is running, the API call must be given the additional header field "X-AllegroPackets-Multimeter-ID: :1" with the PCAP instance ID.

This allows to extract information of automated pcap uploads.

Multi-device analysis

If the Allegro is configured as a gateway for multiple Allegro devices by the Multi-device settings, you need to add the additional header field "X-AllegroPackets-Multimeter-ID: hostname" where the hostname must be the same as configured in the multi-device settings.

REST API Examples

MAC or IP address statistics

Extract the packets per second statistic of the MAC broadcast address

curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/mac/macs/ff:ff:ff:ff:ff:ff'


curl --silent -k -u USER:PASSWORD 'https://allegro-mm/API/stats/modules/ip/ips/10.1.2.3'

Pretty displaying JSON output with jq

curl --silent -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/stats/modules/ip/ips/10.1.2.3' | jq

Capture a specific IP

curl -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/data/modules/capture?expression=ip==10.1.2.3' > path_to/capture.pcap

Capture two IP addresses with ports on a specific Layer 4 protocol

curl -k -u USER:PASSWORD 'https://allegro-mm-XXXX/API/data/modules/capture?expression=IP==10.1.2.3:62887 and IP==10.1.2.100:548 and l4Protocol==TCP' > path_to/capture.pcap