Negofin
Enterprise Data Lakehouse on AWS to unify business intelligence
Nublit designed a data lakehouse serverless on AWS that unifies the data from Wally, Oita, and Crediagil into a single source of truth, with end-to-end governance and security. The platform, deployed in 180 hours, lays the foundation for credit scoring and machine learning, and is now operational 24x7.
About the Client
Negofin is a Paraguayan financial services company that operates with multiple on-premise management systems (Wally, Oita, and Crediagil) distributed across its own data centers. With a growing volume of transactional and operational data dispersed across heterogeneous platforms, the company needed to consolidate all that information into a single modern analytical platform with traceability, governance, and the ability to scale towards risk models and artificial intelligence.
https://www.negofin.com.py/ Paraguay
Needs
Consolidating data from multiple systems into a single source of truth Wally, Oita and Crediagil operated independently, generating disconnected data without the possibility of integrated analysis. Each report required manual labor for extraction and cross-checking, with a high risk of inconsistencies.
Replacing a traditional Data Warehouse model with a modern and scalable architecture The conventional model implied high infrastructure costs, rigidity in the data schema, and limited flexibility. Negofin sought a serverless solution that would grow with the business without requiring server administration.
Ensure government, traceability, and security over financial data Since these are sensitive financial business data, granular access control, a history of dataset versions, secure credential management, and continuous monitoring were essential.
Building the foundation for advanced analytics, scoring, and machine learning Beyond reporting, Negofin sought to lay the foundations for credit risk models, scoring, and predictive analytics that require a well-structured and governed data platform from the source.
What was done?
Lakehouse architecture with Medallion model on AWS
Nublit designed and implemented a modern Lakehouse architecture based on the Medallion three-layer model: Bronze (raw data), Silver (refined and transformed data), and Gold (modelled and ready for analytical consumption). The storage was implemented on Amazon S3 Tables with Apache Iceberg, which provides transactional control, versioning of datasets, and the ability to recover historical versions.
Safe ingestion from on-premise systems via VPN Site-to-Site
Private connectivity was configured between Negofin’s datacenters and the AWS platform via Site-to-Site VPN, with data extraction from on-premise PostgreSQL through AWS Glue Connections. All data traffic travels over private channels without exposure to the internet.
ETL, transformation and orchestration serverless
The processing was implemented with AWS Glue for the ingestion and transformation jobs, DBT with Cosmos for structured modeling Bronze → Silver → Gold, and Glue Data Catalog as a centralized repository of metadata. The orchestration and scheduling of all pipelines were managed with Amazon MWAA (managed Airflow).
Analytical Queries 100% serverless with Amazon Athena
The consumption layer was built on Amazon Athena Engine v3, allowing ad-hoc SQL queries on the Iceberg data without the need to provision or manage compute infrastructure. The model is completely pay-per-query.
Government data with AWS Lake Formation and end-to-end security
AWS Lake Formation was deployed for granular control of table- and column-level access, AWS Secrets Manager for secure credential management, AWS KMS for encryption of data in-flight, and Amazon CloudWatch with automatic alerts for monitoring and traceability of all pipelines.
Infrastructure as Code with Terraform in three environments
The entire platform was deployed using Terraform in three different environments (DEV/TEST/PROD), ensuring reproducibility, consistency, and control of changes. The project was executed in 180 hours.
O&M 24x7 with specialized team in Data and Cloud
Once the platform was productive, Nublit took over the operation and maintenance with 24x7 coverage, including pipeline monitoring, incident management with defined SLAs, IAM management, FinOps on AWS consumption, and hours for evolutionary projects. The team combines Cloud Architecture profiles with Data Architect, Senior Data Engineer, and DataOps/Analytics Engineer.
Results
A single source of truth for the entire financial business:
Negofin consolidated the data from Wally, Oita, and Crediagil into a unified platform, eliminating silos and manual cross-process workflows.
Productive analytical platform, serverless, and without server administration:
The Lakehouse architecture operates in production by automatically scaling up as demand increases.
Data with history, versioning, and granular governance:
Apache Iceberg guarantees complete traceability of each dataset, with the possibility to consult historical versions and audit changes.
Base ready for scoring credit, risk models, and Machine Learning:
The Gold layer exposes clean, modeled, and governed data, ready for consumption by BI tools and predictive models.
Cost reduction compared to the traditional Data Warehouse model:
The pay-per-use model eliminated the need for initial investment in fixed infrastructure.