عجفت الغور

Gregg - BFP Performance Tools

Tags: computers, books, bpf

Chapter 1 - Intro

  • Turned into a general purpose execution engine in 2014
  • VM

Tracing, Snooping, Sampling, Profiling, and Observability

  • Tracing: event based recording
    • Tracing and snooping are the same thing
    • strace, tcp-dump, etc
  • Sampling: tools that take subsets of a measurements to paint a coarse picture
    • Only rough picture
  • Observability: understanding a system through observation
    • Includes tracing tools, sampling tools, and fixed counters
    • Does not include benchmark tools

BCC/BFPTrace/IO Visor

BCC: BFP Compiler Collection

  • First high levle tracing framework developed for BPF
    • Provides C, with interfaces for Python, lua, and C++
    • Much more complex than bfptrace
    • Meant for complex tools

bfptrace

  • Newer frontend, special purpose high level programming langauge
  • meant for one line scripts

IO Visor

  • Linux foundation project on GH
  • Encompases both bcc and bpftrace

BCC

execsnoop(8)

  • Traces execve(2) system call
  • Output reveals which processes were executed while tracing
  • this prints out per-event data, but other tools allow for summaries

biolatency(8)

  • Traces block io latency
  • Displays an ASCII histogram of sizes

Visability

  • BPF tracing gives you visibility across the full software stack
  • Supports:
    • Applications tracing with runtimes
    • Application tracing with compiled code
    • /lib/* information
      • same as ltrace
    • syscalls interface
      • same as strace or perf
      • linux scheduler, file systems, TCP/IP, etc
        • same as ftrace or perf
    • hardware tracing
      • perf, sar, /proc counters

Dynamic instrumentation: kprobes and uprobes

  • Dynamic instrumentation allows for probing of live running software
  • Examples:
    Probe Description
    kprobe:vfs_read Beginning of vfs_read()
    kretprobe:vfs_read End of vfs_read()
    uprobe:/bin/bash:readline Beginning of readline() function in bash
    uretprobe:/bin/bash:readline End of readline() function in bas

Static instrumentation: tracepoints and usdt

  • Function names change and dynamic instrumentation may not always be correct
  • Can use tracepoints or user statically defined tracing:
    Probe Description
    tracepoint:syscalls:sys_enter_open instrument open(2) syscall
    usdt:/usr/sbin/mysqld:mysql:query__start query__start probe

bpftrace

  • takes -e for execute
  • -l for listing open tracepoints