Design, Implementation and Evaluation of Network Monitoring Tasks with the Borealis Stream Processing Engine

Abstract

Several data stream management systems (DSMSs), enabling their users to measure and analyze the behavior of a data stream on-line, are now available to the public. Network monitoring, as a challenging application domain, has shown to fit into the domain of DSMSs. With the use of the Borealis DSMS, we have implemented a set of network monitoring tasks. By performing these network monitoring tasks in a network on generated traffic, we have measured the loads that Borealis can handle. As we know the behavior of the generated network traffic, we have been able to investigate the correctness the monitoring results. Based on our evaluation, we have shown that Borealis can handle a 40 Mbit/s network load, with close to 99% accuracy performing measurements of average amount of received packets per second, as an example. The query language of Borealis has in addition enabled us to express complex network monitoring tasks. When increasing the complexity for the monitoring tasks, we see that the supported network load drops down to 10 to 30 Mbit/s. Borealis has shown to handle significant higher traffic loads than expected, although we have found its load shedding mechanism for preventing overload situations as not efficient. We expect Borealis to handle higher loads than today, by altering parameters that are set in its source code.