How We Expanded a Real-time Decisioning Platform Without Creating Vendor Dependency
- By Viktoriia Serhiienko
CLIENT
A large US data and analytics company operating in the credit and identity intelligence space. The company held data on hundreds of millions of consumers and provided decisioning tools, risk scoring, and identity verification services to financial institutions, insurers, and government agencies. The engineering team supporting one of its core data products had around 40 people at the time of engagement.
CHALLENGE
The company had approved a significant expansion of its real-time decisioning platform — a system financial institution clients used to assess creditworthiness and identity risk at the point of application. The expansion covered new data sources, a redesigned scoring pipeline, and a new API layer for client integrations.
The internal team couldn’t absorb the additional workload. The problem was that previous vendor engagements had left them in a difficult position: work got done, but knowledge stayed with the vendor. When those engagements ended, the internal team was left operating systems they hadn’t built and didn’t fully understand.
For a company handling sensitive consumer data under FCRA, GLBA, and state-level data privacy requirements, that situation carried compliance and operational risk, not just inconvenience.
The requirement was specific: build the team to deliver the platform expansion, but structure the engagement so the internal team could operate, audit, and extend the system independently when REW stepped back.
SOLUTION
We had worked with this client before on a data pipeline reliability project and understood the regulatory environment and how carefully consumer data moved through their systems.
Before any development work began, we spent two weeks understanding where the internal team’s knowledge was strong and where it was thin. The gaps were in three areas: real-time stream processing at the required scale, API design for high-availability client-facing services, and infrastructure patterns for data residency and access control compliance.
The engagement ran two tracks in parallel throughout: delivery and transfer. Not delivery followed by transfer. That distinction mattered.
On the delivery side, we redesigned the scoring pipeline around Apache Flink for real-time stream processing, built the new API layer with versioning and deprecation policies from day one, and rebuilt the data access control layer to enforce attribute-level permissions, with specific fields within consumer records accessible only to services with documented, audited authorization.
On the transfer side, every significant design decision was documented before implementation as part of the process. Internal engineers were co-owners of each component — reviewing PRs, attending architecture discussions, and responsible for explaining the system to the compliance team on request. They weren’t observers.
The engagement ran thirty-six weeks. The last twelve covered joint production support before REW stepped back.
RESULT
The platform expansion went live on schedule. In the first month, the scoring pipeline processed over 40 million events per day with latency under 200 milliseconds, meeting the SLA committed to financial institution customers.
The compliance audit that followed was the first the team described as straightforward. Attribute-level access controls meant auditors could trace which services had accessed which data fields, when, and under what authorization.
Six weeks after REW stepped back, a financial institution client flagged a discrepancy in how identity match scores were being calculated. The internal team diagnosed the issue, identified it as a data normalization problem, and deployed a fix without external involvement in under 48 hours.
The client came back six months later asking us to scope a follow-on engagement extending the same access control architecture to two additional data products.
TECHNOLOGY STACK
- Apache Flink – real-time stream processing for the scoring pipeline, replacing batch-based processing
- Apache Kafka – event ingestion and distribution across pipeline stages
- Java + Spring Boot – API layer with versioned endpoints and built-in deprecation management
- AWS (EKS, RDS Aurora, S3) – containerized services, relational data persistence, and archival storage
- AWS Lake Formation – attribute-level data access controls and audit logging for consumer data fields
- Terraform + Atlantis – infrastructure as code with pull-request-based approval workflow
- Datadog – pipeline observability, latency monitoring, and SLA tracking
Has your engineering team ever found itself operating a black box after ending a vendor relationship? Most likely, it’s the result of an engagement structure where delivery and ownership transfer were treated as separate phases. The good news is that that’s a structural problem, not an inevitable dependency.
Sustainable systems require more than working software – they require internal teams that can operate, audit, and extend the platform independently. If that’s the outcome you’re aiming for, REW Technology can help design the engagement around it from the start.
Share:
