Detection, Response and Optimisation in Modern SOCs

Detection, Response and Optimisation in Modern SOCs

This article is generated from a panel discussion at the recent CyberSecure Online event hosted by the Executive Leaders Network. The session explored how organisations can improve the cycle of detection, response and optimisation within modern Security Operations Centres (SOCs), particularly as AI, automation and large-scale data processing continue to reshape cybersecurity practices.

The discussion brought together several experienced security leaders and practitioners who shared operational insights from different industries.

Panel Speakers

• Dominic Vadakkan – Principal Information Security Engineer (CISSP), Mastercard
  Dominic works on enterprise-scale security engineering and data architecture, focusing on how detection engineering and security telemetry support effective SOC operations.
• Andres Andreu – CEO, Constella
  Andres leads Constella and brings experience in threat intelligence and identity risk, with a focus on emerging cyber threats and digital identity protection.
• Peter Marshall – Director of Developer Relations, Imply
  Peter focuses on data platforms and observability technologies that support large-scale analytics and real-time data processing in security environments.
• Philip Curran – Chief Information Assurance and Privacy Officer, Cooper Health
  Philip oversees security and privacy for a major healthcare organisation, where operational uptime and system availability are critical.

Together, the panel explored practical challenges faced by modern SOC teams, including data overload, analyst burnout, the promise and limitations of AI, and how organisations should measure security performance.

Managing Data in Modern Security Operations

One of the first themes discussed was the growing volume of security data within SOC environments.

Many organisations ingest extensive log data in order to maintain visibility or meet compliance requirements. However, this approach can unintentionally create what some practitioners refer to as a “data swamp”, where large quantities of information exist but extracting actionable insight becomes difficult.

Dominic Vadakkan explained that security teams often optimise for data collection and storage rather than detection capability. As a result, analysts may spend significant time processing irrelevant data instead of focusing on high-value signals.

To address this issue, organisations should begin with detection goals rather than log ingestion strategies. By defining which threats must be detected—such as credential abuse, lateral movement or privilege escalation—teams can determine which data sources are truly necessary.

Another recommended approach is tiered data architecture. High-risk signals can be placed in real-time monitoring pipelines, while lower-priority data can be stored in colder storage layers for compliance or retrospective investigation.

This strategy helps balance detection speed, storage cost and investigative capability.

Data Architecture and Accessibility

Peter Marshall emphasised that data volume itself is not necessarily the problem. The challenge arises when data becomes difficult to access or analyse efficiently.

Security platforms frequently struggle with changing data formats, ingestion pipelines and query performance. When analysts cannot quickly retrieve relevant telemetry during investigations, response times increase.

Modern security architectures increasingly rely on decoupled data systems, where ingestion, storage and analysis components operate independently. This allows organisations to scale each layer according to operational requirements.

Marshall argued that security teams should define a future vision of their SOC capabilities and then identify the architectural gaps preventing them from achieving that state.

From Reactive Detection to Active Threat Hunting

Another major theme was the shift from reactive alert-based detection toward proactive threat hunting.

Traditional SOC operations rely heavily on signature-based alerts that trigger when suspicious activity is detected. However, sophisticated attackers often evade these mechanisms by blending into normal system behaviour.

Threat hunting takes a different approach. Analysts actively search for anomalous activity across networks and systems rather than waiting for alerts to surface.

Philip Curran highlighted that while threat hunting can significantly improve detection capabilities, many organisations face resource limitations. In sectors such as healthcare, security teams often operate with limited budgets and staff, making it difficult to dedicate analysts to full-time hunting activities.

As a result, organisations are increasingly exploring automation and machine learning tools to assist with behavioural detection.

AI in Security Operations: Promise and Limitations

Artificial intelligence was a central topic throughout the discussion.

Panelists noted that AI is particularly effective at analysing patterns across large datasets, making it useful for tasks such as alert triage and anomaly detection.

Security analysts frequently spend large portions of their day reviewing alerts to determine whether they represent legitimate threats or false positives. AI systems can assist by correlating data across identity logs, cloud activity and device telemetry to identify suspicious patterns more quickly.

However, the panel agreed that AI should assist analysts rather than replace them.

Human expertise remains essential for interpreting context, making investigative decisions and determining appropriate responses. AI models may detect anomalies, but determining whether those anomalies represent genuine security incidents requires human judgement.

Controlling the Risks of Automated Response

Automation also introduces potential risks, particularly in environments where system availability is critical.

For example, healthcare organisations cannot risk automated security actions shutting down essential systems. Similarly, industrial or operational technology environments require strict safeguards.

The panel discussed how automation should be implemented within controlled boundaries. For instance, automated responses may be appropriate for low-risk actions such as blocking suspicious login attempts or triggering additional authentication requirements.

More complex actions—such as isolating network segments or shutting down systems—should remain under human control.

Segmentation and clearly defined response policies help limit the potential impact of automated mistakes.

Measuring Security Effectiveness

The panel also examined how SOC performance should be measured.

Many organisations rely on Mean Time to Respond (MTTR) as a primary performance indicator. While useful, the speakers suggested that relying on a single metric can oversimplify the complexities of security operations.

Instead, SOC leaders may benefit from measuring multiple stages of the incident lifecycle, including:

• Detection speed
• Alert triage efficiency
• Investigation time
• Response effectiveness
• Recovery time

Breaking these stages into separate metrics allows organisations to identify where improvements are needed.

For internal performance monitoring, reducing false positives and improving detection accuracy may provide more meaningful insights than high-level metrics alone.

Emerging Security Concerns

Looking ahead, the panel identified several emerging concerns for security leaders.

One major issue is the growing use of AI by cybercriminals, particularly in phishing, social engineering and automated attack campaigns. As attackers adopt AI-powered tools, defenders must adapt detection methods accordingly.

Another concern is the rise of AI-driven identities and automated system accounts, which may operate with elevated privileges. Managing and securing these non-human identities will become an important part of enterprise security strategy.

Key Takeaways from the Panel

The discussion highlighted several important lessons for security leaders:

• Detection engineering should guide data collection strategies.
• Excessive telemetry can hinder investigations if not properly structured.
• Threat hunting is becoming a key component of modern SOC operations.
• AI can reduce analyst workload but should not replace human oversight.
• Automated responses must be carefully controlled to avoid operational disruption.
• SOC performance should be measured across multiple stages of the security lifecycle.

Frequently Asked Questions

What is the detection, response and optimisation cycle in a SOC?

The detection, response and optimisation cycle refers to the continuous process of identifying threats, responding to incidents and improving security processes based on lessons learned. Modern SOCs aim to shorten detection and response times while refining tools, workflows and data strategies.

How does AI help security operations centres?

AI helps SOC teams analyse large volumes of security data, identify patterns and reduce analyst workload. It is particularly useful for alert triage, anomaly detection and log analysis, although human analysts are still required to interpret results and make final decisions.

Why is too much security data a problem?

Excessive log ingestion can reduce the signal-to-noise ratio, making it harder for analysts to identify meaningful threats. Without clear detection goals, organisations may store large volumes of data that provide little operational value.

What is threat hunting in cybersecurity?

Threat hunting is a proactive security practice where analysts actively search for signs of malicious activity within networks and systems rather than waiting for automated alerts.

What metrics should SOC teams use to measure performance?

While metrics like Mean Time to Respond (MTTR) remain useful, SOC teams should also measure detection accuracy, false positive rates, investigation time and recovery speed to gain a more complete view of security performance.

Watch the Full Session On-Demand

To explore the full discussion and gain further insights into this cybersecurity panel, watch the complete session here:
CyberSecure Online Summit | Virtual Event for Cybersecurity Leaders

#securityoperations #SOCoptimisation #threatdetection #incidentresponse #cybersecurityoptimisation #detectionengineering #securitytelemetry #logdatamanagement #dataswamp #SOCdataarchitecture #securitydatapipelines #realtimedetection #colddatastorage #securityanalytics