Enhancing an Anti-Money Laundering (AML) program using Automated Machine Learning

Over the past several years, large banks have transitioned many of their traditional predictive models to machine learning-based systems in core risk management areas, such as credit risk and fraud. As these newer, more effective systems have been successful in replacing their predecessors, executives are thinking more broadly about other areas of the company that would benefit from machine learning.

One function that is ripe for the application of machine learning is Compliance—particularly Anti-Money Laundering (AML). AML teams have the responsibility to monitor all activities occurring throughout their institution in search of behavior consistent with money laundering. Money laundering is essentially the process of trying to conceal the source of ill-gotten funds by passing them through the financial system to make the funds appear legitimate.

Anti-Money Laundering teams have the responsibility to monitor all activities occurring throughout their institution in search of behavior consistent with money laundering.

For example, a money launderer might structure a “dirty” $10,000 cash deposit into 10 separate smaller deposits over several days and at different branches, in an attempt to avoid being the subject in a Currency Transaction Report (CTR). Once these funds have been placed into the financial system, they might be layered through multiple transfers across several different bank accounts before being integrated back into the economy by spending it. The purpose of these steps is to obscure the original source of the $10,000 in cash.

Money laundering goes hand-in-hand with illegal activities like tax evasion, drug dealing, human trafficking, and terrorist financing. That is why its detection is a top priority of government agencies like the Financial Crimes Enforcement Network (FinCEN) in the United States. This unit of the U.S. Department of the Treasury requires financial institutions to have robust AML compliance programs. Programs which do not meet the required standards or have significant lapses can incur large fines, sometimes to the tune of hundreds of millions or even climbing into the billions of dollars! For this reason, banks invest huge sums to operate AML compliance programs in order to avoid incurring these penalties. And these programs are about to be tested more so than ever before with the emergence of cryptocurrencies as a new transaction and payment vehicle for money-launderers, which are cloaked in the anonymity of the blockchain.

For this reason, banks invest huge sums to operate AML compliance programs in order to avoid incurring these penalties.

Rule-based Transaction Monitoring

A key pillar of an AML compliance program is the transaction monitoring framework aiming to detect suspicious activity. The scope of transactions is broad, covering deposits, withdrawals, fund transfers, purchases, merchant credits, payments, and others. Typically, monitoring starts with a rule-based system which scans customer transactions for red flags consistent with money laundering. When a matching pattern is observed, an alert is generated and the case is referred to the bank’s internal investigation team for manual review.

The review process consists of a deep dive analysis by investigators. All data within the bank’s system (in theory) is made available for review so that the investigators can develop a 360° view of the customer, including customer profile, demographic, and transaction history. Additional data from third party data providers and web crawling can supplement this information to complete the picture. After potentially multiple rounds of review, if the investigators conclude the behavior is in fact indicative of money laundering, then the bank will file a Suspicious Activity Report (SAR) with FinCEN.

The standard transaction monitoring system described above has big drawbacks. The false positive rates generated by this rule-based system can reach very high levels – 90% or more. And since the system is rule-based, it is rigid and likely fails to consider complex interactions between the various behaviors used to launder money. Furthermore, maintaining this program is expensive, partially due to the cost of assigning a large team of investigators to the task. Teams can involve hundreds of reviewers at the larger banks, and for the largest banks, more than a thousand. That being said, the programs generate no revenue for the business. All of these factors provide the motivation to evolve the current transaction monitoring process to make detection more efficient. Enter automated machine learning.

Efficient Transaction Monitoring using Automated Machine Learning

Automated machine learning is a state-of-the-art approach by which Artificial Intelligence (AI) is used to select the best machine learning algorithms for making predictions from a particular dataset. All of the information being passed to the reviewer can instead first be assembled into a modeling dataset and passed into a set of machine learning models. The AI engine can then guide the process of selecting the model which most accurately predicts which cases in the historical data ultimately resulted in a SAR filing. Once the best model is identified, it can be applied to the validation cases–a random sample from the modeling dataset that was not used to train the model. The cases can be scored, and the cut-off score that captures all of the suspicious activities in this historical dataset is noted for use in deployment.

The model is then deployed into production to score future alerts. The scores output by the model are used to rank the new cases. Any case exceeding the score threshold (obtained during the model development phase) is sent for review by investigators, and the cases below the threshold can be discarded based on the historical evidence (or more conservatively, made subject to a lighter review). This enhancement creates a more SAR-heavy population for investigators to concentrate their efforts on. In other words, applying automated machine learning helps reduce the false positive rate for the cases selected for manual review, which translates into greater efficiency for the Compliance team.

Consider the following simple illustration where six hypothetical rule-based alerts are shown in the order they were manually reviewed. The last column represents whether investigators ultimately filed a SAR or not, following their hard work reviewing the cases:

Without machine learning, there is no prioritization and the false positive rate generated by the rules engine here is 4/6 = 67% (already low by industry standards… again, this is just an illustration). In hindsight, the bank has wasted resources reviewing several cases that ultimately were determined not to be suspicious. Alternatively, machine learning could provide a suspicious activity score up front that can be used to prioritize the review from most likely to least likely, and investigators could choose not to review cases with scores below the threshold. For instance, if the threshold score is 0.30:

Note that the SAR filing outcomes are the same as before, but the false positive rate for cases reviewed has dropped to ⅓ = 33% in this illustration. The key here was to determine the appropriate score threshold for further investigation based on a historical validation sample where the SAR filing outcomes were known, and then applying the threshold to these alerts.

Model and Threshold Selection in Practice

In order for this to work, the modeling dataset containing historical alerts and their outcomes (SAR filed or not) must be carefully constructed from mostly the same data that is made available to reviewers. An analyst, with little or no expertise in machine learning, can then feed the modeling dataset into the automated machine learning platform, which will perform the key steps of the model development process for them automatically. It will determine the best ways to pre-process the data, structure it for validation and final holdout assessment, distill out the relevant features related to money laundering according to the historical data, and identify the best machine learning algorithms to apply based on this dataset.

The analyst would then select the model they think is most robust by its performance on the validation data. They can observe the score threshold below which no SARs were present (i.e., the threshold yielding zero False Negatives, or FN). The True Negatives (TN) are cases that investigators would have been correct not to review, and represent the efficiency gain. All of the True Positives (TP), the actual SARs, have been captured above the threshold. The resulting new False Positive (FP) rate is reduced relative to reviewing all the cases.

Once the analyst verifies the threshold on the holdout sample (which is the random sample not used to train or validate the model, including selecting the score threshold), the model is ready for production.

Beyond the Horizon

This is only the beginning of the story of how automated machine learning can be used to enhance an AML compliance program. The point of arrival for AML will be reached when machine learning and AI are being used to drive almost all aspects of the program. This will involve machine learning techniques beyond the supervised learning approach described above, where historical outcome data is required. For example, approaches which require few or no labels, such as anomaly detection, have the potential to improve the breadth of detection by uncovering new money laundering patterns. Let’s save that discussion for a future blog post.

The sun is setting on the traditional, inefficient, rule-based-only approach to monitoring customer transactions for potentially suspicious activity. Competitive forces will drive banks to look to enhance these systems using machine learning, whether Compliance teams have the expertise to do so or not. Those responsible for both meeting regulatory obligations and evolving the organization’s traditional approach, will need to make a decision about the best way to incorporate machine learning into their practice.

.

.

About the Author:

Dan Yelle is a Data Scientist at DataRobot, where he focuses on ensuring customers are successful applying automated machine learning. He has 15 years of experience in predictive modeling and data science, and has successfully led machine learning projects at some of the leading financial institutions in the United States. Dan enjoys pioneering the application of machine learning in new initiatives and areas within an organization, having done so in AML compliance, merchant recommendation systems, and spend-based digital ad targeting and measurement.

About the author

Dan Yelle

Data Scientist, DataRobot

Meet Dan Yelle