Of Course It’s Validated, Right?
Every insurer is racing to provide digital offerings powered by advanced analytics that can deliver policy quotes in real time. This is a massive undertaking, and business and technical teams are under tremendous pressure to deliver fast.
With so many moving parts and so many teams involved, most leaders in the organization presume trust in the underwriting platform and don’t question the core ratings engine. Given how foundational that ratings engine is to the business, surely it’s been validated?
But by whom? The actuarial, underwriting and compliance teams aren’t in the guts of the platform and don’t have visibility into how ratings are being calculated. The engineering teams building the platform aren’t underwriting experts and don’t have the business knowledge required to test the accuracy of the system.
Did You Mean -20 or -20.5?
Whether you’re leveraging third-party vendor platforms, or building your own homegrown systems, engineers without the underwriting expertise to correctly implement filings are making seemingly “small” decisions in order to build functioning code. Often, engineers may not know when to consult with underwriters or may not have the access to experts when they need to quickly check their assumptions. The result is a multitude of decisions being made all throughout the development lifecycle without anyone being aware of the potential downstream impacts on the business.
One such example is how or when rounding occurs throughout the premium calculation. If engineers are relying on the rounding functions built into the language of the underwriting tool, that could affect premiums. For commercial insurers with exposure or limits on the order of $100,000, rounding errors introduced during the premium calculation could result in premium shifts of thousands of dollars. E.g. Javascript rounds to the nearest integer so in the event there is a deduction calculated within the premium, -20.5 would round to -20.
Another example is how territories impact Premium. High risk cities and counties are associated with different rates, which is typically handled by high risk cities/counties having specific rates and the rest of a state being assigned to a “remainder of state” risk category and rate. However, if each of the high risk territories has not been correctly identified or if the user interface isn’t referenced exactly correctly to how it’s captured in the data, the premiums generated will be incorrect. Additionally, there may be additional endorsements or coverages that should have been required but that would not be applied due to the mismapping of the territories.
These types of errors can result in the loss of revenue in the short term, and introduce serious risk into the portfolio in the long term.
Ratings Validation At Scale
Validating the ratings engine is tough to do efficiently, given the near infinite number of possible premium combinations. Even the most basic policy could be made up of 15 different customizations that make up the premium, and each of those customizations could have as few as a handful or as many as thousands of options. Thus, rebuilding the ratings engine is not a practical solution, and checking that every possible combination of inputs is triggering the correct premium would require millions of scenarios to be tested. Even if automated, such a massive suite of tests would require days to run (and once built, would be extremely cumbersome and time consuming to maintain). Not to mention each test would require not only a script to create the policy but also to calculate a correct premium upon which to compare to the ratings engine.
One way to tackle this challenge is to ground the tests in a solid understanding of the underwriting rules, and then, instead of testing everything, take an intelligent approach to test automation:
- Analyze each possible rating component to identify the most strategic place to test based on potential business value and risk of failure
- Mathematically reduce the number of permutations to test by utilizing a technique such as Orthogonal Array Testing (OATS)
Ultranauts has deployed this approach at a wide range of organizations with varying levels of digital maturity, with dramatic results. In a recent Ultranauts engagement, an innovation team inside a Fortune 100 insurer was gearing up to launch a commercial insurance underwriting platform, including an automated premium generator. In order to validate that legally-binding state filings were being implemented correctly prior to launch, the client needed to access a rating API endpoint that returned a total premium only, without visibility into the loss costs used to create that premium. An incredibly large number of permutations needed to be tested rapidly and the premiums returned needed to be validated, neither of which could be achieved through the typical industry vendors and frameworks.
90,000 Tests/Day
Ultranauts deployed a specialized quality engineering team with expertise in API test automation and data analytics, consisting of a QE lead, DevOps engineer and core group of quality engineers. Unlike most quality engineering or testing teams, the Ultranauts team invested the time to work closely with the client’s underwriting team and quickly began serving as a translator between the business and technical teams. As a result, the Ultranauts team was able to ingest the extensive underwriting rules and regulatory requirements and develop a test strategy focused on mitigating compliance risk by parameterizing the core rating elements while also establishing an expected result. The solution efficiently scaled up to run over 90,000 API tests per day, helping accelerate the development cycle time by 2X while preventing a multitude of mission-critical errors from reaching the customer.
Trust Your Ratings
If your ratings engine is inaccurate, non-compliant and/or out of sync with your filings, you could be building up a backlog of compliance issues without knowing it, which could result in short term financial impact and long term business risk.
If you’re concerned that your ratings engine can’t be trusted, you can start by tightening the collaboration between your development/quality engineering teams and your underwriting/actuarial teams. If they are already collaborating well but you’re still not sure, you can start to rest easy when the Test Suite for your underwriting platform (and ratings engine) includes a calculation of premiums to compare to the output of the ratings engine.
If you’d like a partner who can do the heavy lifting, Ultranauts can help. Our cognitively diverse teams, 75% of whom are autistic, combine deep technical skills with an aptitude for rapidly absorbing complex domain knowledge so we can hit the ground running. We will invest the time to work closely with your internal teams and thoroughly ingest the underwriting rules and regulatory requirements, serve as a translator between the business and technical teams when needed, and develop a scalable test automation strategy focused on mitigating technical debt and business risk.
