Operational Alert Automation

Eliminating procedural toil from SOC operations through intelligent automation

Role:
Director of Product - Operated as primary PM

Timeline:
4 Months (April 2023)

Team:
1 Junior PM, 6 Developers

The Challenge

Arctic Wolf's SOC processed 2,000+ low-value operational alerts monthly - routine configuration changes and operational issues that required manual investigation. Each alert consumed ~5 minutes of analyst time, totalling half an FTE on procedural work rather than threat hunting.

I inherited this project from another PM with an large set of requirements that poorly matched the problem statements. I narrowed focus to the 6 alert types representing 80% of operational volume - optimizing for impact over comprehensive coverage.

My Approach

I identified the 6 alert types representing 80% of operational volume - optimizing for impact over comprehensive coverage. Working with SOC analysts, we mapped each investigation into flowcharts with 10-20 decision points and validation logic. For edge cases requiring human judgment, we designed intelligent failover that handed off to analysts with pre-gathered investigation context - making even partial automation faster than fully manual workflows.

The biggest risk: incorrectly alerting customers or missing critical events. We mitigated the risk by running automations in parallel with human investigations, validating accuracy before full deployment.

The Solution

Alert Runner automated routine operational alerts end-to-end, freeing analysts from procedural work to focus on threat hunting and incident response.

Key capabilities:

  • End-to-end automation for known patterns: Decision-tree logic handled investigation, validation, and customer notification without human intervention.

    Intelligent fail-over for edge cases: When automation couldn't complete (natural language processing requirements, conflicting instructions), the system handed off to analysts with investigation context pre-gathered, reducing manual handling time from 5 minutes to under 2 minutes.

The Impact

  • 61% of operational alerts triaged by automation end-to-end

  • Remaining 39% partially automated, reducing analyst handling time by 60%

  • ~85 hours/month freed for threat hunting and incident response

  • Proved the automation model - the company later purchased a SOAR platform and scaled this approach to additional alert types

Key Learnings

When inheriting ambiguous projects, ruthlessly prioritize scope. Automating 6 alert types (80% of volume) delivered immediate value while avoiding diminishing returns. The framework proved automation was viable, unlocking broader investment in security operations automation.