Network traffic analysis
Motivation
The continuous growth in network speed allows huge amounts of data to be transferred through a network. An important issue in this context is network traffic analysis to characterize traffic profile and detect Internet security threats. Association rule extraction is a widely used exploratory technique which has been exploited in different contexts (e.g., network traffic characterization). Association rule extraction from network flows, driven by support and confidence constraints, involves (i) generation of a huge number of rules which are difficult to analyze and (ii) pruning rare itemsets even if their hidden knowledge might be relevant.
Generalized association rule algorithm
To address the above issues, we propose a novel approach to analyze network data by means of generalized association rules, which provide a high level abstraction of the network traffic. The proposed technique exploits (user provided) taxonomies to drive the pruning phase of the extraction process. Generalized association rules provide a powerful tool to efficiently extract hidden knowledge which would be discarded by previous approaches.
Technical report: E. Baralis, T. Cerquitelli, and V. D’Elia. Generalized itemset discovery by means of opportunistic aggregation.
The NetMine framework
The NetMine framework allows the characterization of traffic data by means of data mining techniques. NetMine performs generalized association rule extraction to profile communications, detect anomalies, and identify recurrent patterns. Association rule extraction is a widely used exploratory technique to discover hidden correlations among data. However, it is usually driven by frequency constraints on the extracted correlations. Hence, it entails (i) generating a huge number of rules which are difficult to analyze, or (ii) pruning rare itemsets even if their hidden knowledge might be relevant. To overcome these issues NetMine exploits a novel algorithm to efficiently extract generalized association rules, which provide a high level abstraction of the network traffic and allows the discovery of unexpected and more interesting traffic rules. The proposed technique exploits (user provided) taxonomies to drive the pruning phase of the extraction process. Extracted correlations are automatically aggregated in more general association rules according to a frequency threshold. Eventually, extracted rules are classified into groups according to their semantic meaning, thus allowing a domain expert to focus on the most relevant patterns. Experiments performed on different network dumps showed the efficiency and effectiveness of the NetFrame framework to characterize traffic data.
Welcome