eBPF: The Superpower Hiding Inside the Linux Kernel

A practical guide to understanding, evaluating, and using Extended Berkeley Packet Filter — with step-by-step examples you can run today.

10-16 minutes(2205 words)complex

Difficulty: Advanced
Estimated Time: 30-45 minutes
Prerequisites: Solid Linux knowledge, Basic C understanding, Command line experience, Familiarity with networking and syscalls

Introduction

For decades, extending the Linux kernel meant one of two painful choices: write a kernel module (and risk crashing the entire system) or submit a patch upstream and wait years for it to land. Then came eBPF — a technology that quietly turned the kernel into a programmable platform, the way JavaScript turned the browser into one.

Today, eBPF powers networking at Cloudflare, observability at Netflix, security at Meta, and load balancing in Kubernetes clusters worldwide. Tools like Cilium, Falco, Pixie, and bpftrace are all built on top of it. If you work with Linux infrastructure in any capacity, eBPF is no longer optional knowledge — it's a core skill.

In this article, we'll demystify eBPF from the ground up: what it is, how it works, where it shines, where it struggles, and — most importantly — how to actually write and run eBPF programs through four concrete examples.

What Is eBPF?

eBPF (Extended Berkeley Packet Filter) is a kernel technology that lets you run sandboxed programs inside the Linux kernel without modifying kernel source code or loading kernel modules. Think of it as a virtual machine running in kernel space — but with strict guarantees that your code can't crash, hang, or compromise the system.

The name is historical and somewhat misleading. The original BPF (1992) was a tiny filtering language for tcpdump. In 2014, Alexei Starovoitov rewrote it into a general-purpose in-kernel virtual machine — that's "extended BPF." Today, "eBPF" is treated as a standalone term; the acronym no longer expands meaningfully, much like how "AWK" or "GNU" outgrew their origins.

How It Works

An eBPF program follows a clear lifecycle:

Write the program in a restricted subset of C (or Rust, Go via cilium/ebpf, Python via BCC).
Compile it to eBPF bytecode using Clang/LLVM.
Load it into the kernel via the bpf() syscall.
Verify — the kernel's verifier statically analyzes the bytecode to prove it terminates, doesn't access invalid memory, and has bounded loops.
JIT-compile the bytecode to native machine code for performance.
Attach to a hook point: a kernel function (kprobe), tracepoint, network interface (XDP/TC), socket, cgroup, or LSM hook.
Execute every time the hook fires — with overhead measured in nanoseconds.

Programs communicate with user space through maps — efficient key-value data structures (hash maps, arrays, ring buffers, LRU caches) shared between kernel and user space.

Advantages

eBPF earned its rapid adoption for good reasons:

Safety without sacrificing speed. The verifier rejects any program that could harm the kernel, yet JIT compilation makes execution nearly as fast as native kernel code. You get the performance of a kernel module with the safety of a sandboxed runtime.
No kernel recompilation. You can deploy new functionality — a new packet filter, a new tracer, a new security policy — without rebooting, without patching the kernel, and without loading a module. This is transformative for production systems.
Universal observability. eBPF can attach to virtually any kernel function and most user-space functions (via uprobes), giving you visibility no traditional tool can match. You can trace a TCP retransmission down to the exact code path that triggered it.
Programmable networking at line rate. Through XDP (eXpress Data Path), eBPF processes packets before the kernel networking stack even sees them — making it possible to implement DDoS mitigation, load balancing, and routing at speeds approaching dedicated hardware.
Portable across kernel versions. With CO-RE (Compile Once — Run Everywhere) and BTF (BPF Type Format), modern eBPF programs can be compiled once and run on many kernel versions without recompilation.

Disadvantages and Limitations

eBPF is powerful, but it's not magic. The constraints are real:

A steep learning curve. Writing eBPF requires understanding kernel internals, the verifier's rules, helper functions, map types, and program types. The error messages from the verifier are notoriously cryptic — "invalid mem access" can mean a dozen different things.
Verifier restrictions are strict. No unbounded loops, no arbitrary pointer arithmetic, limited stack size (512 bytes), bounded program complexity (~1 million instructions). Algorithms that look trivial in regular C often need to be rewritten to satisfy the verifier.
Linux-centric. Although Windows now has an eBPF runtime in development, eBPF remains fundamentally a Linux story. Production usage on other platforms is still niche.
Kernel version fragmentation. Many features require recent kernels (5.x or later). CO-RE helps, but enterprises stuck on RHEL 7 or older Ubuntu LTS releases have limited options.
Debugging is painful. You can't run a debugger on kernel-loaded eBPF code. Debugging means bpf_printk(), reading /sys/kernel/debug/tracing/trace_pipe, and a lot of trial and error.
Security surface area. While the verifier is robust, eBPF has had its share of CVEs. Loading eBPF programs requires CAP_BPF (or root on older kernels), and misconfigured permissions can expose the kernel to attack.

Common Use Cases

Four domains dominate real-world eBPF adoption:

Networking — Load balancing (Cilium, Katran), DDoS mitigation, service mesh data planes, firewalls, and traffic shaping. XDP can handle tens of millions of packets per second per core.
Observability and Tracing — System-wide profiling (BCC, bpftrace), application performance monitoring (Pixie), distributed tracing without code instrumentation, and detailed I/O / syscall tracking.
Security — Runtime threat detection (Falco, Tetragon), syscall filtering beyond seccomp's capabilities, container isolation, and intrusion detection based on real-time kernel events.
Performance Profiling — CPU profiling without sampling overhead, off-CPU analysis, lock contention analysis, and flame graph generation (Brendan Gregg's toolkit).

Implementation Examples

Now let's get concrete. We'll walk through four examples — one per use case — with full step-by-step instructions. All examples assume Ubuntu 22.04 or later with kernel 5.15+.

Prerequisites (one-time setup)

Before any example, install the tooling:

sudo apt update
sudo apt install -y bpfcc-tools libbpfcc libbpfcc-dev linux-headers-$(uname -r) \
  bpftrace clang llvm libelf-dev libbpf-dev gcc-multilib

Verify the kernel supports BTF (required for CO-RE):

ls /sys/kernel/btf/vmlinux && echo "BTF available"

Example 1 — Networking: An XDP Packet Dropper

Goal: Drop all incoming ICMP (ping) packets at the earliest possible point in the network stack, before they reach the kernel networking subsystem.

Step 1 — Write the eBPF program (xdp_drop_icmp.c):

#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/in.h>
#include <bpf/bpf_helpers.h>

SEC("xdp")
int xdp_drop_icmp(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;
    if (eth->h_proto != __constant_htons(ETH_P_IP))
        return XDP_PASS;
    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;
    if (ip->protocol == IPPROTO_ICMP) {
        bpf_printk("Dropped ICMP packet\n");
        return XDP_DROP;
    }
    return XDP_PASS;
}
char _license[] SEC("license") = "GPL";

Step 2 — Compile to eBPF bytecode:

clang -O2 -g -target bpf -c xdp_drop_icmp.c -o xdp_drop_icmp.o

Step 3 — Attach the program to a network interface (replace eth0 with your interface, find it with ip link):

sudo ip link set dev eth0 xdp obj xdp_drop_icmp.o sec xdp

Step 4 — Test it. From another machine, try ping <your-ip>. The pings will fail.

Step 5 — Watch the trace output:

sudo cat /sys/kernel/debug/tracing/trace_pipe

Step 6 — Detach when done:

sudo ip link set dev eth0 xdp off

What just happened: Your eBPF program ran on every incoming packet before the kernel allocated an skb (socket buffer) for it. That's why XDP is so fast — it short-circuits the networking stack entirely.

Example 2 — Observability: Tracing All execve() Syscalls with bpftrace

Goal: Log every program execution on the system in real time — who ran what, when.

For tracing, bpftrace is the easiest entry point. It's the awk/dtrace of eBPF — a high-level scripting language that compiles to eBPF behind the scenes.

Step 1 — Write the script (exec_trace.bt):

#!/usr/bin/env bpftrace
BEGIN {
    printf("Tracing exec() calls. Hit Ctrl-C to end.\n");
    printf("%-9s %-6s %-16s %s\n", "TIME", "PID", "COMM", "ARGS");
}
tracepoint:syscalls:sys_enter_execve {
    printf("%-9d %-6d %-16s %s\n",
        elapsed / 1000000,
        pid,
        comm,
        str(args->filename));
}

Step 2 — Make it executable and run it:

chmod +x exec_trace.bt
sudo ./exec_trace.bt

Step 3 — Trigger some activity in another terminal: run ls, cat /etc/hostname, open a new shell, etc.

Step 4 — Observe the output:

TIME PID COMM ARGS
142 12453 bash /usr/bin/ls
389 12454 bash /usr/bin/cat

Step 5 — Extend it. Want to also capture command arguments? Add a second probe:

tracepoint:syscalls:sys_enter_execve {
    printf("%-6d %-16s %s", pid, comm, str(args->filename));
    join(args->argv);
}

What just happened: bpftrace generated, compiled, loaded, and attached an eBPF program to the sys_enter_execve tracepoint. Every time any process on the system calls execve(), your handler fires — with negligible overhead (a few hundred nanoseconds per event).

Example 3 — Security: Detecting Suspicious File Opens

Goal: Alert whenever any process tries to read /etc/shadow — a classic indicator of credential theft attempts.

Step 1 — Write the script (shadow_watch.bt):

#!/usr/bin/env bpftrace
BEGIN {
    printf("Monitoring access to sensitive files...\n");
}
tracepoint:syscalls:sys_enter_openat
/ str(args->filename) == "/etc/shadow" /
{
    printf("[ALERT] PID %d (%s) by UID %d tried to open /etc/shadow\n",
        pid, comm, uid);
}

Step 2 — Run it:

sudo ./shadow_watch.bt

Step 3 — Trigger an alert from another terminal:

cat /etc/shadow       # Will be denied for non-root, but the attempt is still logged
sudo cat /etc/shadow  # Allowed, but still logged

Step 4 — Extend to multiple files by changing the filter:

tracepoint:syscalls:sys_enter_openat
/ str(args->filename) == "/etc/shadow" ||
  str(args->filename) == "/etc/sudoers" ||
  str(args->filename) == "/root/.ssh/authorized_keys" /
{
    printf("[ALERT] PID %d (%s) UID %d -> %s\n",
        pid, comm, uid, str(args->filename));
}

Step 5 — Pipe to your SIEM. In production, you'd typically have a daemon written with libbpf or cilium/ebpf that reads alerts from a ring buffer map and ships them to Falco, Wazuh, or Splunk.

What just happened: You built a tiny intrusion detection system in 10 lines of script. Unlike auditd, this runs with virtually no overhead and can filter intelligently in the kernel — only sending events to user space when conditions match.

Example 4 — Performance Profiling: Counting System Calls Per Process

Goal: Find which processes are syscall-heavy — a common bottleneck in containerized workloads.

Step 1 — Write the script (syscall_count.bt):

#!/usr/bin/env bpftrace
BEGIN {
    printf("Counting syscalls. Ctrl-C to stop and print results.\n");
}
tracepoint:raw_syscalls:sys_enter {
    @syscalls[comm] = count();
}
END {
    printf("\n=== Syscall counts by process ===\n");
    print(@syscalls);
}

Step 2 — Run for ~30 seconds, then Ctrl-C:

sudo ./syscall_count.bt

Step 3 — Read the output, sorted descending:

=== Syscall counts by process ===
@syscalls[chronyd]: 412
@syscalls[sshd]: 1834
@syscalls[node]: 28492
@syscalls[python3]: 91204

Step 4 — Drill down into the worst offender. Suppose python3 dominates. Find which specific syscalls:

#!/usr/bin/env bpftrace
tracepoint:raw_syscalls:sys_enter
/ comm == "python3" /
{
    @[args->id] = count();
}

Then map syscall numbers to names with ausyscall <num> or by checking /usr/include/asm/unistd_64.h.

Step 5 — Generate a flame graph for CPU profiling. For deeper analysis, use Brendan Gregg's profile tool from BCC:

sudo profile-bpfcc -F 99 -f 30 > out.stacks
git clone https://github.com/brendangregg/FlameGraph
./FlameGraph/flamegraph.pl out.stacks > flame.svg

Open flame.svg in a browser — you'll see exactly where every CPU cycle went, system-wide, across kernel and user space.

What just happened: You built a system-wide profiler that costs less than 1% CPU overhead, attributing every kernel event to its originating process. This kind of observability is impossible with traditional tooling without significant performance impact.

Conclusion

eBPF represents a fundamental shift in how we interact with the operating system. Instead of treating the kernel as a black box, we can program it — safely, dynamically, and at production scale. The four examples above are intentionally simple, but they map directly onto the technology powering some of the largest infrastructure deployments in the world.

If you're just starting out, my recommendation is to climb the abstraction ladder gradually: begin with bpftrace for one-liners and quick investigations, graduate to BCC Python tools for richer analysis, and finally move to libbpf + C or cilium/ebpf (Go) when you need production-grade performance and CO-RE portability.

The Linux kernel has always been the foundation of modern computing. With eBPF, that foundation is no longer immutable — it's programmable. The question is no longer whether you'll encounter eBPF in your career, but how soon, and how deeply you'll want to go.

Start small. Run the examples above. Break them, fix them, modify them. The verifier will yell at you. That's part of learning. Within a week, you'll be reading eBPF code with confidence — and within a month, you'll be writing tools you couldn't have imagined building before.