Japanese Electronics Case Study
Onepoint provides Japanese manufacturing giant with Talend Data Integration Expertise for their Global Web Centralization Project
Founded in early twentieth century, the client has been pioneer in breakthrough technology for nearly a century.
With a net sales of over US dollars 64 billion and over 200,000 employees they are one of the largest electronics product manufacturers in the world.
Headquartered in Osaka, Japan the client has over 500 companies spread out across the globe.
Business Drivers
- The client wanted to implement a global platform for publishing to the web. This would eliminate each country publishing at a local country level and would promote a unified strategy across the world.
- Ensure quality control by implementing a common reference data that would be managed collaboratively (i.e. created locally by countries but also with central oversight).
- To prepare multiple data pipelines that catered to different geographical regions.
- Being a global player they wanted a structure which supported ALL languages on the web.
- To enable faster generation of new and more flexible file and data formats whilst supporting existing file and data formats.
The Requirement
Talend which has a robust data integration in an open and scalable architecture was selected as being the core technology to achieve the goals set. The project was designed to have a two-stage approach with multiple steps so that there was considerable time saving.
1.The initial delivery was to take the product descriptions and transform them into a translatable format.
2.The files in this translatable format were then to be processed manually.
3.To ensure a proper handover the files were placed in an orderly fashion in a pre-specified folder structure.
4.The processed files were then to be automatically associated to multimedia contents.
5.The translated files and all multimedia files are then input in to the new Web Content Management System (CMS) solution which would then display products in all supported languages on the web.
The Solution
To achieve this, two main Talend jobs were deployed and scheduled using Talend Administration Console (TAC).
Translator Job
- A translator Talend job was introduced which extracted files in the original XML based format from the FTP server and converted them to the target XML format multiplying the file by the number of translation languages which could then be used by the translation team. At the end of this process there would be a transformed XML file for languages like e.g. German, French, Spanish with the untranslated product details.
- Each and every client product had an associated XML file created within a short span of time.
- The job was also designed to inspect each file and decipher which file needed product translation. Care was taken to ensure the job generated files with proper naming convention to include the country and language code so it would ensure easy readability on hand over.
To finish off this job cleaned-up the FTP server to remove the unwanted old files.
Figure 1: Translator Job diagram
Transfer Job
A transfer Talend job which mainly associated the multimedia content to the translated XML files and copied the associated content to an FTP folder accessed by the web CMS (content management system). Once again once the newly associated XML and multimedia files are copied to the CMS directories a clean-up is carried out on the FTP server.
Figure 2: Transfer Job diagram
The enumerated operations in the above diagram
1.XML files and multimedia files are read by the execution server.
2.The execution server deletes files from the FTP server.
3.Execution server uploads transformed XML files to the Translation FTP server.
4.Execution server reads the processed XML files from the Translation FTP server.
5.The combined Talend AMC and Console servers are used to prepare and run the jobs on the execution server using cron. Talend AMC basically provides a web application with which administrators can deploy, configure jobs.
6.The Execution server reports back to the Talend AMC server allowing administrators to monitor the job execution.
The Outcome
The Talend jobs designed were successfully tested on live environment to ensure each client product had the corresponding translated description and media.
As this was set to feed the main website there was expected to be a high frequency of product coverage which the jobs were designed to handle
Case Studies Category
Industry
Manufacturing
Solution
ETL, Data Integration
Technology
Talend
Service
Talend Enablement Services
