While asset managers may outsource their fund administration, they cannot abdicate their accountability to oversee accurate accounting and reporting to their customers. The growing complexity of investment products, domiciles, regulations, and share classes have rendered traditional top-down, threshold break methods obsolete. Even certain shadow NAV services can't keep up with the nuances and permutations of these different factors.

We have now worked for some of the largest global asset managers and their service providers on NAV oversight systems and algorithms. Among the challenges we helped address:

• Reducing the percent of exceptions that prove to be false positives, wasting a significant portion of back office labor time and expense.
• Ensuring that the top NAV incidents in an asset manager's history and in the industry are tracked and predicted with easy-to-configure and shared checks and algorithms.
• Reducing the delays in reporting and communicating incidents and exceptions across asset manager and service provider teams.

Labor in back-office operations spend a significant amount of time identifying exceptions. Our analysis across many asset managers indicates that most exceptions are false positives. The challenge is reducing the false positive rate without increasing the risk that we might miss an actual error (false negative). We therefore deployed two independent types of algorithms to meet these objectives. First, as illustrated below, we trained a securities level (bottoms-up) anomaly detection algorithm. The benefit of this algorithm was it applied many different variables to isolate true outliers. It then improved its accuracy by running thousands of samples through a kind of Monte Carlo simulation. Finally, we worked with our customer diligently to validate whether each exception flagged by the algorithm was "worth seeing." Over time, we were able to significantly reduce the number of exceptions, while increasing the percent of exceptions deemed worth seeing.

In conjunction with the anomaly detection algorithm, we have developed incident detection algorithms that look for parameters that matched historic NAV errors. Each of these incidents detectors are written to preemptively locate the combination of conditions that match past errors.

The service provider and asset manager agree to share the same algorithms. This is often enhanced when the service provider contributes data with more variables than normally provided to the asset manager. OnCorps agrees to run the algorithms daily and post the exceptions, trends, and risks on a shared, visual dashboard. This method enables faster resolution of issues while both parties contribute to improving algorithm performance and accuracy. The goal is to reduce labor on both sides by reducing the number of false positives (exceptions) handled.

Each user is provided a shared dashboard showing the progress, incidents flagged, and resolution. The dashboards are managed with sophisticated user authorizations.

There are three major outcomes we have achieved in this NAV oversight case.

• We have significantly reduced false positives. We have reduced the number of exceptions by volume - which has translated to significant time savings. Moreover we carefully have asked whether the remaining exceptions were "worth seeing" and have increased that percent to around 95 percent of exceptions.
• We successfully passed a stringent red team test by a major fund administrator. In this test, we were asked to run algorithms with planted errors. In this test, we caught a significant percent of the planted errors. We also identified other meaningful errors that were not planted but deemed meaningful. Finally, when asked to define each exception caught by our algorithms, the customer designated 96 percent as worth seeing. (see diagram below).
• Identified over $100 million in potential NAV errors. Over the last several years, our algorithms have played a material role in flagging NAV errors.