Keyboard shortcuts

/ or ⌘/Ctrl K
Find a note
j / k
Next / previous section or linked note
h / l
Collapse or go to parent / expand or enter
e or Alt-click
Read a linked note here
o
Open focused note on its own
g g / G
First / last section or linked note
g h / g a
Home / all notes
g b / g t
Backlinks / table of contents
t
Cycle System, Light, Dark
? / Esc
Show / close this reference

Search: ↑/↓ or Ctrl N/P, Enter to open. Shortcuts pause while typing.

Gregg - BFP Performance Tools [1459c115]

Tags: computers, books, bpf

1. Chapter 1 - Intro

  • Turned into a general purpose execution engine in 2014
  • VM

1.1. Tracing, Snooping, Sampling, Profiling, and Observability

  • Tracing: event based recording

    • Tracing and snooping are the same thing
    • strace, tcp-dump, etc

  • Sampling: tools that take subsets of a measurements to paint a coarse picture

    • Only rough picture

  • Observability: understanding a system through observation

    • Includes tracing tools, sampling tools, and fixed counters
    • Does not include benchmark tools

1.2. BCC/BFPTrace/IO Visor

1.2.1. BCC: BFP Compiler Collection

  • First high levle tracing framework developed for BPF

    • Provides C, with interfaces for Python, lua, and C++
    • Much more complex than bfptrace
    • Meant for complex tools

1.2.2. bfptrace

  • Newer frontend, special purpose high level programming langauge
  • meant for one line scripts

1.2.3. IO Visor

  • Linux foundation project on GH
  • Encompases both bcc and bpftrace

1.3. BCC

1.3.1. execsnoop(8)

  • Traces execve(2) system call
  • Output reveals which processes were executed while tracing
  • this prints out per-event data, but other tools allow for summaries

1.3.2. biolatency(8)

  • Traces block io latency
  • Displays an ASCII histogram of sizes

1.4. Visability

  • BPF tracing gives you visibility across the full software stack

  • Supports:

    • Applications tracing with runtimes
    • Application tracing with compiled code

    • /lib/* information

      • same as ltrace

    • syscalls interface

      • same as strace or perf

      • linux scheduler, file systems, TCP/IP, etc

        • same as ftrace or perf

    • hardware tracing

      • perf, sar, /proc counters

1.5. Dynamic instrumentation: kprobes and uprobes

  • Dynamic instrumentation allows for probing of live running software

  • Examples:

    Probe Description
    kprobe:vfs_read Beginning of vfs_read()
    kretprobe:vfs_read End of vfs_read()
    uprobe:/bin/bash:readline Beginning of readline() function in bash
    uretprobe:/bin/bash:readline End of readline() function in bas

1.6. Static instrumentation: tracepoints and usdt

  • Function names change and dynamic instrumentation may not always be correct

  • Can use tracepoints or user statically defined tracing:

    Probe Description
    tracepoint:syscalls:sys_enter_open instrument open(2) syscall
    usdt:/usr/sbin/mysqld:mysql:query__start query__start probe

1.7. bpftrace

  • takes -e for execute
  • -l for listing open tracepoints