*bump*

Hai John,

Thanks for the info provided. That will be really useful.

I also have a similar problem. I have designed a job to load data from a positional feed. It takes 20 mins to load the data.

The logic is as follows.

1. Read the data from the feed.
2. Do some validations against the data in the database.
3. Insert the data into the database tables

Can u please explain how parrallesim can be achieved here.

Thanks and regards,
Amirths

Amirths,

One strategy would be to either count or estimate the number of lines in your input file--  you can then define a set of context parameters to be used in the parallel child jobs. These parameters will hold the start and end lines for each copy of a child job that does most of the work.

in the child job using context parameters to set the values of the "Header" and "Footer" fields in a tFileInputPositional to partition the file into chunks.

The child then does the validation and inserts for its chunk of the input file.


This is just one way to parallelize a load-- there are other ways-- often what you choose depends on the paticulars of the job you are trying to complete.

Again, if you have more specific questions please post them (a new thread would be best). One of the best things about Talend is the community and their willingness to put in time helping strangers.