Case Study

Data Engineering and Business Intelligence

A business providing MGAs, program administrators, and InsurTechs with access to the reinsurance market was struggling to ingest and validate carrier data at volume. We built a Python-based ETL pipeline that now processes monthly data from more than 25 sources in minutes.

99%

See more results below ↓

Client

A business providing MGAs, program administrators, and InsurTechs with access to the reinsurance market

Lines

Homeowners, auto, commercial, specialty

What We Solved

Carrier data ingestion, validation, and reporting

ENGAGEMENT

Data engineering

The Challenge

Reporting that should have taken minutes was taking eight hours, and the cause sat upstream in how carrier data arrived and was processed.

Insurance companies routinely struggle with ingesting, validating, and processing carrier data, and this client faced the full set of those problems at once. Data formats were inconsistent, fields were missing, and errors made accurate analysis difficult. Dataset size and structure strained processing capacity. Integrating data from multiple carriers and confirming its accuracy was time-consuming. The ETL process itself was complex, resource-intensive, and prone to performance bottlenecks. And generating accurate reports, let alone deriving anything useful from them, was harder than it should have been.

The Solution

Paktolus implemented a robust Python-based ETL pipeline using BigQuery, dbt, Apache Airflow, and AWS.

Monthly data from 25 MGAs is now validated and ingested into the data warehouse with both speed and accuracy, and business reporting time has been reduced from eight hours to seconds.

Technologies

Python BigQuery dbt AWS Apache Airflow

Results

25+

MGAs’ data processed within minutes

99%

reduction in business reporting time, from eight hours to seconds

Where This Applies

Cession files, bordereaux, carrier statements, and loss runs are all the same problem wearing different names: data arriving in whatever shape the sender chooses, with a person absorbing the difference. The difficult part is rarely the transformation. It is the exception handling and data quality rules underneath it.

Explore: Paktolus for Reinsurers → Paktolus for MGAs → Systems Integration + Carrier Connectivity →

View More Case Studies

Automating Quote Submission Ingestion with OCR and AI

Emailed submissions in mixed formats, including handwritten, parsed and validated automatically for a mid-sized E&S insurer.

90% less manual data entry
READ THE CASE STUDY →

Digital Transformation

Seventeen separate systems unified into one platform for one of the largest insurance exchanges in the world.

70% efficiency gains
READ THE CASE STUDY →

Automated Commission Processing

Seventy carrier statement formats mapped into parsing templates, with commissions calculated and payouts generated automatically.

50% faster commission payouts
READ THE CASE STUDY →

How would results like these impact your business?

Twenty minutes, a candid conversation. If we’re not the right fit, we’ll say so.

Let’s Talk