You are not logged in.

Unanswered posts

Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at We look forward to hearing from you there!

#1 2014-02-24 16:50:59

134 posts

jeanjpotgieter said:

Tool that creates file input metadata automatically from external file

Tags: [schema]

I am in charge of using Talend to transform data for my company.

We have many data streams from various sources that we must transform to a single output format. Each data stream has a file that dictates how the data is laid out. This file is always in the same format and has 6 columns that specify the field data type, the length, a flag option, the starting position, field name, and description.

Since these files are all laid out the same way, it should be pretty easy to create meta data automatically without having anyone manually create fields, name them, and give them a length/data type. I've looked around for something that could accomplish this but haven't found anything yet.

An example of the file would be something like this:

CHAR     20     0     1     LOANNO     Loan Number
CHAR     40     0     21   FNAME        Full Name
CHAR     40     0     61   SNAME       Second Name

The use case would be a component that reads the file, skipping header lines at the top, then creating fields for each field. For the example above it would create 3 fields and name them LOANNO, FNAME, and SNAME. It would give them a length of 20, 40, 40 then create either delimited metadata or positional metadata using the starting position and field length.

I would then be able to select this meta data and use the tMap component to write expressions and map it to our company's output format.

Last edited by jeanjpotgieter (2014-02-24 21:26:18)


#2 2014-02-26 07:36:07

3186 posts

jlolling said:

Re: Tool that creates file input metadata automatically from external file

This is currently not possible with a component. For sure you can build your own Java code doing that.

Last edited by jlolling (2014-02-26 07:41:05)

Talend Certified working for cimt objects AG in Berlin


Board footer

Talend Contributor Agreement - Talend Website Privacy Policy