Digital Marketer Case Study
Onepoint helps global digital provider of financial data to implement complex ETL workflows at scale.
Our client is a global provider of digital marketing and client reporting tools. Founded in 2002 in London, they offer a broad range of fund marketing, investor servicing solutions, Business Continuity and Disaster Recovery plan.
With a team of over 100 highly skilled employees experienced in Asset Management Headquartered in New York, the client has regional business offices spread across London, Reno and Cape Town and India.
Business Drivers
- To have an unified data model to drive their future data services.
- To process high volumes of data.
- To improve the quality of data processing.
- To support a wide variety of file formats.
- And quickly adapt to changing business requirements.
The Requirement
Client wanted a strategic ETL – extract, transform and load – solution for their data processing needs.
ETL is a major component within the Client’s’ infrastructure. It’s a critical part of the services they deliver and as such they had chosen Talend to be their strategic platform for their ETL implementation. There were several challenges in order to achieve this.
The records in the standard files needed to be validated,parsed, transformed, enriched and then loaded into this unified model. The whole process needed to be seamless and carried out in an agile fashion. The requirement also extended to ensuring that the post-delivery handover did not entail a steep learning curve.
Given the vast amount of data that needed to be processed and the prospect of future growth, the platform needed to be horizontally scalable and highly available with complex scheduling based on certain dependencies and workflows.
The Solution
To fulfill the horizontally scalability required, a virtual server setup was implemented. This is a feature in Talend which allows a pool of servers to be created and the TAC server then balances the load of these servers. As new jobs need to be executed Talend selects the least utilized server and executes the new job on that server.
For high availability a clustered setup was proposed for the critical services which are required for an uninterrupted production service. The diagram below illustrates the new architecture for the client production platform.
Diagram 1: New architecture for the client production platform
The Outcome
The basic concepts of the design had been implemented had been implemented as part of a proof of concept. It proofed to support the flexibility and scalability required by our client. As such the design was chosen to be implemented for their new platform.
Added Technology Benefits
The newly implemented Talend technology brings with it additional benefits to ensure future business improvements to support:
- Ensure Data Quality
- Support the large volumes expected in production
- Provide the agility to quickly adapt to changing business requirements
