Designing Real-Time Screening at Scale: Latency, Throughput, and Webhooks

Most discussions of sanctions screening focus on what to buy and why. This one focuses on how a screening system actually has to be built to function at production scale, for engineering teams designing the architecture that sits behind a screening feature, whether they are building it in-house or integrating a vendor API into a high-throughput product.

The considerations here are deliberately architectural: latency budgets, throughput under load, the batch-versus-real-time tradeoff, and webhook design for continuous monitoring. The goal is to separate the engineering decisions that determine whether a screening system holds up in production from the buying criteria that dominate most vendor comparisons.

Latency Budgets

Why Milliseconds Matter

Real-time sanctions screening returns results in milliseconds, which requires fundamentally different infrastructure than traditional batch compliance tools. Real-time screening engines should hold sanctions lists in memory or in extremely low-latency data stores. The engine must apply fuzzy matching logic while accounting for name transliterations, aliases, spelling variations, and partial matches, doing all of this without giving up speed.

The latency budget for a screening call is not an abstract performance target. It is constrained by where in the user or transaction flow the call sits. Latency is a critical factor in real-time systems, and the API must consistently return results within milliseconds to avoid disrupting user experience. Providers should be transparent about both average response times and worst-case performance under high load conditions.

Published benchmarks from production screening APIs illustrate the range engineering teams should expect and design around. One API reports p50 latency of 45ms, p95 of 202ms, and p99 of 276ms. Another reports an average response time of 350ms, sufficient to support real-time use cases such as live onboarding and event-driven transaction screening. A reasonable expectation for single-entity screens is sub-500ms, with sub-2-seconds for batch requests of around 100 entities. More important than raw speed is consistency: an API averaging 200ms that spikes to 5 seconds during list update windows will cause timeouts in the calling system.

Designing for Tail Latency, Not Just Averages

The architectural lesson from these benchmarks is that p99 latency, not average latency, determines whether a screening call will cause visible friction or timeouts in a live user flow. A screening API that averages 100ms but occasionally spikes to several seconds during list ingestion windows will produce intermittent, hard-to-debug failures in any synchronous integration.

Engineering teams should request p50, p95, and p99 figures specifically when evaluating a provider, not just an average, and should design their own integration with appropriate timeout and retry logic rather than assuming the call will always return quickly.

Throughput at Scale

Batch Limits and Concurrent Processing

Throughput, not just per-call latency, becomes the binding constraint once volume increases. For batch screening of existing customer databases, throughput in screens per second matters more than individual response latency. An API capping batch requests at a low number of entities per call turns a 30-second job into a 30-minute one when scaled across thousands of records.

Production-grade batch endpoints typically support up to 1,000 entities per call, though this varies by provider, and engineering teams running large historical re-screens or periodic full-database refreshes should explicitly test the batch ceiling against their actual data volume rather than assuming any vendor's batch endpoint will scale linearly with their needs.

Horizontal Scalability and Stability Under Load

Strong and newer real-time screening platforms are built on highly available and horizontally scalable architectures. It is important to understand how the system performs at scale, as screening performance must remain stable even during peak usage periods, for example during a product's own traffic spikes, a major sanctions list update affecting a large proportion of a customer base, or a batch re-screen running concurrently with live real-time traffic.

A production screening API should support concurrent processing of high-volume batches alongside webhook-based notifications for screening result payloads, suited to periodic screening of entire business partner databases without degrading the real-time path.

This dual capability, sustained high-throughput batch processing running alongside low-latency real-time calls without one degrading the other, is a meaningful architectural differentiator between a production-grade screening system and a basic lookup tool that was designed for one use case at a time.

{{snippets-guide}}

Batch vs. Real-Time: Why Both Are Needed

The Complementary Architecture

Real-time and batch sanction screening are both needed for a strong compliance program. These two processes are complementary capabilities that address different risk vectors and operate in different parts of the compliance program. In a hybrid architecture, real-time screening does the heavy lifting in onboarding and payment processing through API-driven screening calls that block sanctioned parties, while batch screening runs in the background on a daily schedule. Real-time screening handles the point of entry; batch screening handles the ongoing monitoring part of the program.

The Failure Mode of Choosing Only One

If a real-time screening system experiences downtime, transactions and onboarding events may need to stop entirely. An even worse scenario is onboarding events or transactions continuing without being checked while the real-time system is down. If a batch screening run fails or is delayed, the entire customer database stays unchecked for that period.

This is a specific design requirement, not a general caution: the architecture needs an explicit failure mode decision for what happens when the real-time screening call does not return within the expected latency budget. Failing closed (blocking the transaction until screening succeeds) protects compliance posture at the cost of availability.

Failing open (allowing the transaction to proceed and flagging it for delayed screening) protects availability at the cost of a temporary compliance gap. Most production systems should fail closed for high-risk flows and document any exception explicitly, rather than leaving the behavior undefined and discovering it during an actual outage.

Webhook Architecture for Continuous Monitoring

Why Webhooks Are the Critical Path, Not an Add-On

The real risk isn't the screen at onboarding. An automated screening system with webhook callbacks means the application gets notified within hours of a list update, not whenever someone remembers to re-run a batch.

A well-designed system allows entities to be added to a watchlist and delivers near real-time webhook alerts when their risk status changes due to list updates. The webhook delivery latency, the time between a list publishing a new designation and the webhook firing to notify the application, is itself a metric engineering teams should benchmark across vendors.

Best-in-class vendors deliver within one to four hours of the underlying publication. Some vendors run only daily batch cycles, meaning a morning designation might not reach the application until the next day.

Designing for Webhook Reliability

A webhook architecture needs to handle delivery failures, duplicate deliveries, and out-of-order delivery as standard engineering concerns, not edge cases. The receiving system should implement idempotent processing, so a duplicate webhook delivery for the same underlying event does not trigger duplicate downstream actions such as a duplicate compliance alert or a duplicate account freeze.

It should also maintain a persistent mapping between the webhook's entity identifier and the application's own internal record, so that incoming events can be reliably matched without ambiguity even as the application's data model evolves.

List Update Frequency as a Distinct Metric From API Latency

A common architectural confusion is treating API response latency and underlying list data freshness as the same metric. They are not. It is a common misconception that real-time means the underlying sanctions data updates every second. In practice, list updates occur periodically and are published at specific intervals, separate from how quickly the API itself returns a result for any given query.

Published update cadences vary meaningfully across providers and lists: data from global sanctions lists updated every 60 minutes represents a tighter cadence than consolidated global sanctions lists refreshed every three hours, or SDN list checks updated every six hours. Engineering teams should evaluate this cadence as a distinct requirement from API response time, since a screening call that returns in 50 milliseconds against data that is six hours stale provides a different risk profile than a call that returns in 300 milliseconds against data updated every hour.

What Separates Production-Grade From Basic

Pulling these architectural dimensions together, the checklist that separates a production-grade screening API from a basic lookup tool includes:

• Documented p50, p95, and p99 latency figures under realistic load, not just an average
• Sustained throughput at the batch volumes the application actually needs, with batch limits tested against real data, not assumed
• Demonstrated stability during simultaneous batch and real-time traffic, not just isolated benchmarks of each in turn
• Webhook delivery within hours of list publication, with documented delivery latency, not an undefined "real-time" claim
• An explicit, documented failure mode for what happens to screening calls during a provider outage
• List update cadence disclosed separately from API response time, since the two are independent properties of the system
• Horizontal scalability that has been demonstrated at production volumes, not just claimed in marketing material

{{snippets-case}}

Conclusion

The architecture decisions covered here, latency budgets, throughput ceilings, the batch-versus-real-time split, and webhook reliability, are what determine whether a screening system holds up once it is carrying production traffic, independent of how good the underlying matching algorithm is.

An engineering team that evaluates a screening provider purely on matching accuracy and list coverage, without testing these architectural properties under realistic load, risks discovering the gaps only when the system is already live and the consequences of a missed designation or a timeout cascade are no longer theoretical.

sanctions.io is a highly reliable and cost-effective solution for real-time screening. AI-powered and with an enterprise-grade API with 99.99% uptime are reasons why customers globally trust us with their compliance efforts and sanctions screening needs.

To learn more about how our sanctions, PEP, and criminal watchlist screening service can support your organisation's compliance program: Book a free Discovery Call.

We also encourage you to take advantage of our free 7-day trial to get started with your sanctions and AML screening (no credit card is required).