I am in charge of using Talend to transform data for my company.
We have many data streams from various sources that we must transform to a single output format. Each data stream has a file that dictates how the data is laid out. This file is always in the same format and has 6 columns that specify the field data type, the length, a flag option, the starting position, field name, and description.
Since these files are all laid out the same way, it should be pretty easy to create meta data automatically without having anyone manually create fields, name them, and give them a length/data type. I've looked around for something that could accomplish this but haven't found anything yet.
An example of the file would be something like this:
CHAR 20 0 1 LOANNO Loan Number
CHAR 40 0 21 FNAME Full Name
CHAR 40 0 61 SNAME Second Name
The use case would be a component that reads the file, skipping header lines at the top, then creating fields for each field. For the example above it would create 3 fields and name them LOANNO, FNAME, and SNAME. It would give them a length of 20, 40, 40 then create either delimited metadata or positional metadata using the starting position and field length.
I would then be able to select this meta data and use the tMap component to write expressions and map it to our company's output format.
Last edited by jeanjpotgieter (2014-02-24 21:26:18)
This is currently not possible with a component. For sure you can build your own Java code doing that.
Last edited by jlolling (2014-02-26 07:41:05)
Talend Certified working for cimt objects AG in Berlin