Longterm DB: Difference between revisions
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
==Description== | ==Description== | ||
The | The Long-term DB feature (in firmware >= 4.3) uses an attached storage devices to store traffic information of IP addresses and Layer 7 protocols with low resolution for a much longer time than the live statistics. | ||
The elements stored in the | The elements stored in the long-term DB are as follows, graph data is available in 5 minute resolution: | ||
* IP addresses | * IP addresses | ||
| Line 10: | Line 10: | ||
*# traffic graph in 5 minute resolution | *# traffic graph in 5 minute resolution | ||
The storage is used similar to a swap file mechanism so the | The storage is used similar to a swap file mechanism so the long-term data is not kept between restart unless the [[DB persistence]] feature is enabled too, which is recommended when using the long-term feature. To reduce the amount of time to dump/restore, the DB persistence configuration allow to skip storing live data. | ||
==Usage== | ==Usage== | ||
[[File:Longterm DB dashboard.png|alt=Longterm DB activated dashboard|thumb|Longterm DB activated dashboard]]If this feature is enabled, a view toggle button appears in the top menu bar. This button allows to switch between the real time "RT" view and the | [[File:Longterm DB dashboard.png|alt=Longterm DB activated dashboard|thumb|Longterm DB activated dashboard]]If this feature is enabled, a view toggle button appears in the top menu bar. This button allows to switch between the real time "RT" view and the long-term ("LT") view. | ||
In the | In the long-term view, the IP address information contain only information about the traffic amount in 5 minute resolution. | ||
The navigation menu in the | The navigation menu in the long-term view only contains those modules which are available in this view. | ||
If the | If the long-term view is activated on module pages which do not support long-term data, a corresponding info box is shown. | ||
==Setting== | ==Setting== | ||
The configuration can be found in the global settings page in the " | The configuration can be found in the global settings page in the "Long-term DB and persistence" tab. | ||
To enable this feature, select a storage device to be used, enable the feature and enter a file size. | To enable this feature, select a storage device to be used, enable the feature and enter a file size. | ||
It is recommended to also enable the [[DB persistence]] feature to be able to save and restore the | It is recommended to also enable the [[DB persistence]] feature to be able to save and restore the long-term DB data during restarts. | ||
Once enabled, the utilization of the file is shown and the [[System Info Page]] contains information about how long the data can be kept. | Once enabled, the utilization of the file is shown and the [[System Info Page]] contains information about how long the data can be kept. | ||
Tip: Since the amount of information stored in the | Tip: Since the amount of information stored in the long-term DB is limited by the graph resolution, the file size usually don't need to be similar sized as the main memory. 10 GByte is a good starting point. | ||
The size can be increase but it requires a restart of the packet processing. | The size can be increase but it requires a restart of the packet processing. | ||
| Line 55: | Line 55: | ||
|not recommended, should not be used | |not recommended, should not be used | ||
|} | |} | ||
It is also not recommended to place the | It is also not recommended to place the long-term DB on the same storage device that is used a packet ring buffer as it will deteriorate the performance of both features. | ||
== Limitations == | == Limitations == | ||
# The data in the | # The data in the long-term DB is limited to a selected subset of the data in the In-Memory-DB. See above for an exact list of elements available. | ||
# The data is written into the | # The data is written into the long-term DB in variable intervals depending on traffic and system load. It takes up to 10 minutes (two graph intervals) until the data appears in the graph. Therefore, the last 5-10 minutes appear empty or with less traffic than in live view. | ||
Revision as of 12:19, 11 February 2025
Description
The Long-term DB feature (in firmware >= 4.3) uses an attached storage devices to store traffic information of IP addresses and Layer 7 protocols with low resolution for a much longer time than the live statistics.
The elements stored in the long-term DB are as follows, graph data is available in 5 minute resolution:
- IP addresses
- activity time
- traffic graph in 5 minute resolution
- Layer 7 protocols
- traffic graph in 5 minute resolution
The storage is used similar to a swap file mechanism so the long-term data is not kept between restart unless the DB persistence feature is enabled too, which is recommended when using the long-term feature. To reduce the amount of time to dump/restore, the DB persistence configuration allow to skip storing live data.
Usage
If this feature is enabled, a view toggle button appears in the top menu bar. This button allows to switch between the real time "RT" view and the long-term ("LT") view.
In the long-term view, the IP address information contain only information about the traffic amount in 5 minute resolution.
The navigation menu in the long-term view only contains those modules which are available in this view.
If the long-term view is activated on module pages which do not support long-term data, a corresponding info box is shown.
Setting
The configuration can be found in the global settings page in the "Long-term DB and persistence" tab.
To enable this feature, select a storage device to be used, enable the feature and enter a file size.
It is recommended to also enable the DB persistence feature to be able to save and restore the long-term DB data during restarts.
Once enabled, the utilization of the file is shown and the System Info Page contains information about how long the data can be kept.
Tip: Since the amount of information stored in the long-term DB is limited by the graph resolution, the file size usually don't need to be similar sized as the main memory. 10 GByte is a good starting point.
The size can be increase but it requires a restart of the packet processing.
Notes
Recommended storage device types:
| Storage device | Note |
|---|---|
| NMVe based SSD | recommended |
| SATA based SSD | can be used for moderate traffic, check system load for high system utilization |
| USB based SSD | not recommended, but might be useful for small systems (Allegro 200/500) |
| HDD | not recommended, should not be used |
It is also not recommended to place the long-term DB on the same storage device that is used a packet ring buffer as it will deteriorate the performance of both features.
Limitations
- The data in the long-term DB is limited to a selected subset of the data in the In-Memory-DB. See above for an exact list of elements available.
- The data is written into the long-term DB in variable intervals depending on traffic and system load. It takes up to 10 minutes (two graph intervals) until the data appears in the graph. Therefore, the last 5-10 minutes appear empty or with less traffic than in live view.