Different tools, different jobs
Financial-crime monitoring rarely improves by replacing every control with one model. Explicit rules remain useful for known patterns and policy limits. Statistical methods describe expected behaviour and highlight deviations. Machine-learning models can combine many signals and rank complex patterns. Language models can organise text and explain structured findings in readable form.
These layers should be composed around a decision, not marketed as interchangeable “AI.” A rule can state exactly why it fired. An anomaly score can identify unusual activity without knowing whether it is harmful. A supervised model can learn patterns from labelled outcomes, but those labels may be incomplete or biased by past investigation choices. A language model can summarise evidence, but it should not silently become the numerical risk engine.
A clear system diagram should show which layer generated each signal, which data it used, how outputs are combined, and where analyst review occurs. That clarity makes testing and challenge possible.
What each layer contributes
Rules encode explicit controls: an amount threshold, a rapid sequence, or a prohibited combination. They are easy to inspect, but rigid rules can create alert volume when customer context differs. Statistics provide baselines, peer comparisons, and measures such as velocity or deviation. Their value depends on stable data and meaningful comparison groups.
Machine learning can rank alerts or identify multivariate anomalies that single thresholds miss. Supervised models need outcomes that represent the intended task; unsupervised models need careful interpretation because unusual does not mean suspicious. Performance must be tested on relevant populations and periods, not only on training data.
Language models are strongest when working with language: summarising linked evidence, extracting information from approved documents, drafting a reviewable chronology, or explaining structured reason codes. They can omit facts or generate unsupported statements, so outputs need source links, constrained prompts, and human review. Numerical calculation and alert generation should remain traceable to validated controls rather than an ungrounded narrative.
Avoid counting the same fact three times
Correlated signals can inflate risk. Suppose one burst of payments triggers a “high velocity” rule, a statistical deviation score, and an ML feature based on the same count. Adding all three as independent evidence effectively counts one behaviour several times.
Start by mapping signal lineage: source fields, time windows, transformations, and downstream features. Measure correlations and test whether each signal adds information after the others are considered. Combining logic may cap a family of related signals, choose the most informative representative, or let a model account for overlap—provided the result remains testable and understandable.
Language summaries introduce another form of duplication. A narrative should not describe three technical representations as three separate suspicious events. It should group them under the underlying observation and explain that several controls detected the same pattern.
Evaluate both false positives and missed cases
No detector can be assessed with one headline accuracy number. Teams should examine precision or alert yield, recall where outcomes are sufficiently known, investigation workload, time to disposition, and performance across relevant customer segments. Because undiscovered crime is not labelled, recall estimates have limits; targeted sampling and retrospective review can provide additional evidence.
Thresholds express a trade-off. Lowering one may find more relevant cases while creating more review work. Raising it may improve alert yield while missing subtle activity. The right choice depends on risk appetite, control purpose, operational capacity, and the consequences of error.
Calibration also needs precise language. If a score is presented as a probability, observed outcomes should support that interpretation in the target population. If it is only a ranking score, say so. Bands and actions can still be useful without claiming probability.
Watch for leakage and drift
Leakage occurs when model development uses information that would not have been available at decision time, or indirectly includes the outcome. An analyst disposition entered weeks later cannot be an input to an earlier alert model. Randomly splitting transactions can also leak a customer’s later behaviour into training when evaluation should simulate future operation.
Drift occurs when inputs, behaviour, or the relationship between signals and outcomes changes. A new product, market, fraud pattern, or data pipeline can shift performance. Monitor data completeness, feature distributions, score distributions, alert outcomes, and override patterns. Define what triggers investigation, retraining, threshold review, or temporary fallback.
NIST’s AI RMF resources emphasise ongoing monitoring because system performance and trustworthiness can change after deployment. That is general AI risk guidance, not an AML certification. In financial-crime operations, it reinforces a practical point: model approval is the start of operating discipline, not the end.
Keep the analyst in a meaningful role
Human review should be designed, measured, and supported. Analysts need the event timeline, customer context, signal lineage, missing-data warnings, and direct links to evidence. They need a way to record disagreement and to escalate novel patterns.
The system can prepare; the analyst judges. A language model may draft a case assessment, but the reviewer should verify every material assertion against evidence. An anomaly model may rank activity, but policy defines the next step. A rule may fire, but the alert remains a review trigger—not proof of an offence.
A layered approach works when each component has a bounded role, the combined output is tested, and the decision remains reviewable. The objective is not maximum automation. It is better use of evidence under controlled, measurable human oversight.
Primary sources
These sources support the regulatory and standards-related statements in this article. They do not endorse Merid or certify any product.
- Risk-Based Approach Guidance for the Banking SectorFinancial Action Task Force (FATF), October 2014
- AI Risks and TrustworthinessUS National Institute of Standards and Technology (NIST), AI RMF 1.0 resources
- Manage — AI RMF PlaybookUS National Institute of Standards and Technology (NIST)
- Appendix C: AI Risk Management and Human-AI InteractionUS National Institute of Standards and Technology (NIST)