Multithreaded L7-filter for Linux and Multicore Schedulers
Danhua Guo and Laxmi Bhuyan
Department of Computer Science and Engineering
University of California Riverside
{dguo, bhuyan} @ cs dot ucr dot edu
Overview of the original L7-filter
L7-filter is a Deep Packet Inspection (DPI) classifier for Linux's Netfilter that identifies packets based
on application layer data. It can classify packets as Kazaa, HTTP, Jabber, Citrix,
Bittorrent, FTP, Gnucleus,
eDonkey2000, etc., regardless of port. It complements existing
classifiers that match on IP
address, port numbers and so on.
The original intent is for L7-filter to be used in conjunction with Linux QoS
to do bandwith arbitration ("packet shaping") or traffic accounting.
To download the original L7-filter, see Sourceforge project page
What to expect from this page
As convenient as the original L7-filter package is, it is still a single threaded network application. As multicore web servers become the mainstream backend platform, single threaded applications not only waste the available physical resources but also lack behind the performance requirements in terms of throughput and latency. Part of my PhD thesis focuses on how to parallelize network applications and run them efficiently on multicore web servers. Specifically for L7-filter, we have written a multithreaded version of L7-filter, and several multicore schedulers to improve its performance by a factor of 13 compared to the single threaded version and by 57% compared to our baseline multithreaded version. In summary, we developed the following functionalities for the L7-filter:
- A trace-driven model - We separate the major bottleneck of L7-filter, pattern matching, from network noises by moving the TCP stack to the userspace. This version is extremely useful for research purpose.
- An online L7-filter - The actual L7-filter that works in any Linux box with iptable support. It reads packets straight from the network.
- A multithreaded L7-filter - We use pthread to parallelize the pattern matching process.
- Affinity based scheduler - Similar to Receive Side Scaling (RSS) in modern commercial NIC card, this scheduler distributes all the packets in the same connection to the same core.
- Hierarchical Adjusted Highest Random Weight (H-AHRW) scheduler - When connection sizes vary, load balancing becomes on issue. This scheduler balances the workload at the packet level while maintaining the connection locality.
- Cache-Aware HRW (CA-HRW) scheduler - How to generalize the H-AHRW to multicore servers with different cache/core topology?
- An accurate hash adjustment method - What is the theory behind hash adjustment? How to maintain uniformity of a hash while achieving heterogeneous hash space?
Source code
While for research convenience, we tested our proposals using a trace-driven model, we also developed the corresponding online version for all our schedulers.
Offline trace-driven model
The offline trace driven model decouples the L7-filter into two components: network processing and pattern matching. Without noises from the network, it is very convenient for DPI studies to focus on the performance bottleneck - pattern matching.
This model requires a libnids package to simulate the TCP stack in the userspace and a libpcap to read packets from the tcpdump trace file located on the disk.
Online L7-filter that ACTUALLY works in Linux
Instead of reading packet trace from the disk, the online L7-filter works with iptables in Linux, and processes packets received from the NIC. Please refer to the HOWTO page for system setup. A snapshot of the steps:
- Patch the kernel and recompile. You need to enable "Layer 3 Dependent Connection tracking" and "Connection tracing netlink interver" from the kernel config menu. That way ip_conntrack_netlink or nf_conntrack_netlink will be compiled into the kernel
- Load ip_conntrack_netlink module into your kernel
- Forward packets from Netfilter to NFQUEUE:
#iptables -A FORWARD -j (NF)QUEUE
- Now let L7-filter to read packets from the NFQUEUE:
#l7-filter -f [configuration file]
Feel free to contact me if you have any questions.
Documentation
Absolutely essential reads
l7-filter related publications and development docs
- Technical details
- Danhua Guo, Laxmi Bhuyan and Bin Liu,
"An Efficient Parallelized L7-Filter Design for Multicore Servers",
To appear in IEEE/ACM Transaction on Networking, 2011
- D. Guo, et al.,
"A QoS Aware Multicore Hash Scheduler for Network Applications",
IEEE International Conference on Computer Communications (INFOCOM), 2011 
- D. Guo, et al.,
"An Adaptive Hash-based Multilayer Scheduler for L7-filter on a Highly Threaded Hierarchical Multicore Server",
ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), 2009
- D. Guo, et al.,
"A Scalable Multithreaded L7-filter Design for Multicore Servers",
ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), 2008
Feature overview
- Everything listed on the L7-filter page.
- A trace driven model and an online model to assist the study of L7-filter.
- Several published schedulers for multicore servers. The schedulers complete a thorough analysis for network scheduling issues in multicore servers - connection locality, load balancing, cache affinity, and core topology.
- Compatibility across a wide spectrum of state-of-the-art multicore architectures on modern web servers: Intel Xeon 5355, AMD Opteron, Sun Niagara 2.
More documentation
Contact us
Comments, suggestions, looking for collaboration or donations? Feel free to Email either me or Dr. Bhuyan.
Related software
Similar (open source/partially open source) projects
Credits
The original coders were Justin Levandoski, Ethan Sommer, and Matthew
Strait, with support from Sebastian Celis, Andy Exley and Lillie
Kittredge. The primary maintainers are now Ethan Sommer and Matthew
Strait.
The multithreaded version L7-filter and all the multicore schedulers were created, coded, published and maintained by Danhua Guo.
Last updated November 19th 2010 by Danhua Guo