IP Reputation

Rockfish Probe integrates with threat intelligence services for IP reputation lookups.

Overview

Two approaches are available:

These features are mutually exclusive.

IP Reputation (Direct AbuseIPDB)

Query AbuseIPDB directly with local caching.

Configuration

ip_reputation:
  enabled: true
  api_key: "your-abuseipdb-api-key"
  cache_ttl_hours: 24
  max_age_in_days: 90
  s3_upload: true

How It Works

1. For each flow, source and destination IPs are queued for lookup
2. Lookups run in a background thread
3. Results are cached in memory with reference counting
4. Cache is exported to Parquet every hour

Rate Limiting

AbuseIPDB free tier: 1000 requests/day.

When rate-limited (HTTP 429):

1. API requests pause
2. Local cache continues serving
3. Resumes at the next hour boundary
4. Repeats if still rate-limited

Output Schema

Hourly Parquet exports include:

Threat Intel (External API)

Use an external threat intelligence server (e.g., rockfish_intel) for centralized lookups.

Configuration

threat_intel:
  enabled: true
  endpoint_url: "http://localhost:8080"
  api_token: "your-api-token"
  batch_size: 100
  timeout_seconds: 10

Benefits

• Centralized caching: Share cache across multiple sensors
• Rate limit management: Server handles provider limits
• Multiple providers: Server can aggregate multiple sources

Output

Threat intel Parquet files are written to <parquet_dir>/intel/.

With S3 and Hive partitioning:

s3://bucket/prefix/intel/year=YYYY/month=MM/day=DD/filename.parquet

Setup with rockfish_intel

1. Start the intel server with your AbuseIPDB key
2. Create a client entry in clients.yaml
3. Configure the probe:

threat_intel:
  enabled: true
  endpoint_url: "http://threatintel-server:8080"
  api_token: "client-token-from-clients-yaml"

Choosing Between Options

Getting an AbuseIPDB API Key

1. Create account at abuseipdb.com
2. Go to API settings
3. Generate API key

Free tier: 1000 checks/day Paid tiers: Higher limits, additional features