ONTIC
ONTIC project – ONline Network TraffIc Characterization
Keywords
Big data, data mining and machine learning, network traffic analysis, Apache Hadoop and SPARK
Abstract
Accurate identification and categorization of network traffic is a crucial task in many network management and engineering activities, such as QoS evaluation, capacity planning and attack detection. Terabytes of data may be transferred through the core network of a typical ISP every day, and an exponential growth in traffic is expected
in the short-term, with more than 50 billions of Internet-connected devices. This scenario represents a tough challenge for network data capture and analysis. An accurate and massively scalable platform for both online and offline characterization of network traffic pattern evolution is a key element in facing the challenge. Top applications benefitting from such a platform include proactive congestion control mechanisms and intrusion detection systems.
To this end, the ONTIC project proposes to design, develop and evaluate:
1) A novel architecture of massively-scalable online techniques able to (a) characterize network traffic data streams, identifying traffic pattern evolutions, and (b) proactively detect anomalies in real time at very high network speeds, i.e., hundreds of thousands of packets per second.
2) An innovative set of massively-scalable offline data mining techniques to characterize network traffic, exploiting big-data analytic approaches and cloud-based distributed computation paradigms on extremely large network traffic datasets.
ONTIC will integrate these techniques into an autonomous network traffic characterization platform, to be used as cornerstone of a new generation of scalable and proactive network management and analysis applications.
Dissemination and adoption of project outcomes in other application domains (e.g. bioinformatics, medicine, physics, social sciences, and finances) will be fostered by releasing the code and documentation of the ONTIC framework to the public by means of popular open-source repositories.
For more info see the official website at http://ict-ontic.eu/
Partners
Work Packages
The project is structured in six work packages (WP). Our research group is mainly involved in WP3.
- WP 1 Management
- WP 2 Big Data Network Traffic Summary Dataset
- WP 3 Scalable Offline Network Traffic Characterization System
- WP 4 Scalable Online Network Traffic Characterization System
- WP 5 Implementation, Integration and Verification
- WP 6 Exploitation and Dissemination
WP3 Scalable offline network traffic characterization system
The goal of WP3 is to design and develop a scalable analysis system for offline characterization network traffic traces.
WP3 specifically addresses the limits of the existent analysis techniques for big data:
- available large-scale data mining algorithms are poorly optimized for cloud computing environments;
- none or few complex mining techniques are available to be applied to petabyte-scale datasets;
- large-scale mining algorithms are much less rich of features withj respect to the vast amount of algorithms that can be applied to small-scale datasets on a single machine;
- recent advances in the networking domain, such as Software Defined Networking, and the presence of largely distributed phenomena, such as DDoS (Distributed Denial of Service) attacks, cloud storage services and CDN (Content Delivery Networks), require new network characterization approaches.
Algorithms developed in the ONTIC framework will run on top of the de-facto Big Data standard Hadoop platform. Furthermore, the Spark framework will be explored, thanks to the promising results of the latest reports. Further goals of WP3 also include exploring the feasibility of applying the devised algorithms into new domains, such as bioinformatics, genomic, physics , etc., and proper visualization of the characterization system results, to facilitate domain expert interpretation and interactivity.
Links
ONTIC official website: http://ict-ontic.eu/
Project reference in CORDIS: http://cordis.europa.eu/
Welcome