Performance Tuning

Optimize Rockfish Probe for high-speed network capture.

AF_PACKET Tuning

Ring Buffer Size

For 10 Gbps+ capture, increase the ring buffer:

afpacket:
  block_size: 4194304   # 4 MB per block
  block_count: 128      # 512 MB total ring buffer

Use Fanout for Multi-Queue NICs

Modern NICs have multiple RX queues. Use fanout to utilize all cores:

# Run multiple instances with same fanout group
taskset -c 0 rockfish_probe -i eth0 --live afpacket \
    --afp-fanout-group 1 --parquet-dir ./flows1 &

taskset -c 1 rockfish_probe -i eth0 --live afpacket \
    --afp-fanout-group 1 --parquet-dir ./flows2 &

Use hash fanout mode to keep flows together.

CPU Pinning

Pin to specific CPU cores:

taskset -c 0 rockfish_probe -i eth0 --live afpacket ...

Or use CPU isolation:

# /etc/default/grub
GRUB_CMDLINE_LINUX="isolcpus=0,1"

System Tuning

Socket Buffers

Increase kernel buffer sizes:

# Temporary
sudo sysctl -w net.core.rmem_max=134217728
sudo sysctl -w net.core.rmem_default=134217728

# Permanent (/etc/sysctl.conf)
net.core.rmem_max=134217728
net.core.rmem_default=134217728

Network Budget

Increase NAPI budget for high packet rates:

sudo sysctl -w net.core.netdev_budget=600
sudo sysctl -w net.core.netdev_budget_usecs=8000

IRQ Affinity

Distribute NIC interrupts across CPUs:

# Find NIC IRQs
cat /proc/interrupts | grep eth0

# Set affinity (example for 4 queues)
echo 1 > /proc/irq/24/smp_affinity
echo 2 > /proc/irq/25/smp_affinity
echo 4 > /proc/irq/26/smp_affinity
echo 8 > /proc/irq/27/smp_affinity

Or use irqbalance with proper configuration.

Disable CPU Power Saving

Prevent CPU frequency scaling:

# Set performance governor
for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
    echo performance > $cpu
done

Flow Table Sizing

Limit memory usage under high connection rates:

flow:
  max_flows: 1000000    # Limit to 1M concurrent flows
  idle_timeout: 60      # Shorter timeout for faster cleanup

Parquet Output Tuning

Batch Size

Larger batches = fewer files, better compression:

output:
  parquet_batch_size: 2000000   # 2M flows per file

S3 Aggregation

Reduce small file overhead:

s3:
  aggregate: true
  aggregate_hold_minutes: 5   # Merge files for 5 minutes
  delete_after_upload: true

Monitoring

Statistics Output

Enable periodic statistics:

output:
  stats: true
  verbose: 2    # Debug level

Key Metrics to Watch

• Packets/sec: Compare with NIC stats (ethtool -S eth0)
• Drops: Check for ring buffer overflows
• Flows/sec: Flow export rate
• Memory usage: Monitor with top or htop

Check for Drops

# NIC drops
ethtool -S eth0 | grep -i drop

# Kernel drops
cat /proc/net/dev | grep eth0

# AF_PACKET drops
cat /proc/net/packet

Hardware Recommendations

NIC Selection

For high-speed capture:

• Intel X710/XL710 (40 GbE)
• Intel E810 (100 GbE)
• Mellanox ConnectX-5/6

Enable RSS (Receive Side Scaling) for multi-queue distribution.

CPU

• Modern Intel Xeon or AMD EPYC
• At least 1 core per 10 Gbps
• Large L3 cache helps

Storage

For sustained capture:

• NVMe SSD for local Parquet files
• Fast S3-compatible storage with adequate bandwidth

Example: 10 Gbps Configuration

license:
  path: /opt/rockfish/etc/license.json

input:
  source: eth0
  live_type: afpacket

flow:
  idle_timeout: 120
  active_timeout: 900
  max_flows: 2000000
  max_payload: 256

afpacket:
  block_size: 4194304
  block_count: 128
  fanout_group: 1
  fanout_mode: hash

output:
  parquet_dir: /data/flows
  parquet_batch_size: 2000000
  observation: sensor-01

s3:
  bucket: flow-data
  region: us-east-1
  aggregate: true
  aggregate_hold_minutes: 2
  delete_after_upload: true

Run with CPU pinning:

sudo taskset -c 0-3 rockfish_probe -c config.yaml