• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » [resolved] tFileInputDelimited accepting a variable to dynamically

#1 2009-05-29 22:22:25

jmbrown
New member
Registered: 2009-05-22
Posts: 4

[resolved] tFileInputDelimited accepting a variable to dynamically

Is there a way to send the tFileInputDelimited component variable to dynamically fill in the file that I need to process.
1. The file is stored in a directory.
2. The file names are stored in a database table.
3. I can get the name from the db but can figure out if there is a way to send that name to the tFileInputDelimited component.

Best regards.

I am currently a novice user.

Offline

#2 2009-05-29 22:50:32

bcourtine
Member
Company: Alcion Group
Registered: 1970-01-01
Posts: 64
Website

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

It is possible.

You can put this value in the globalMap of your job, and use it after.

If you have several files in your DB, the better way is probably to use a tFlowToIterate (which store the fileName the globalMap for each file).


Uploaded Images

Last edited by bcourtine (2009-05-29 22:50:51)


Java/J2EE and Talend certified consultant for Alcion Group.

Offline

#3 2009-06-01 17:48:48

jmbrown
New member
Registered: 2009-05-22
Posts: 4

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

Sorry it took so long to reply to this message. bcourtine this worked great!!! I decided to you use the tFlowToIterate component which worked great. The more I work with the various componets I find them easy to use.
Regards.

Offline

#4 2009-07-29 23:43:29

valeksa
New member
Registered: 2009-07-29
Posts: 3

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

bcourtine wrote:

It is possible.

You can put this value in the globalMap of your job, and use it after.

If you have several files in your DB, the better way is probably to use a tFlowToIterate (which store the fileName the globalMap for each file).

Hello,

I'm novice user so be patient with me smile

I have similar situation and resolve it in the same way.
Here is a picture of part of my job design.

I have a tFileInputDelimited component which holds a part of the name of the tFileInputExcel_3 file.
It is shown on a picture. It works as a charm.

My problem is, each of those excel files that I process in a loop, has several lookup excel files.
Those files are in the same directory and I can not move them.
I made a tFileInputExcel_2 component and it is not working. (bad file name).
I apply the same rule for creating a fileName as in a tFileInputExcel_3 component.

Is there a way for each record in FileInputDelimited file, to make a look up excel component with "dynamic" fileName.

bottom line:
1. in component tFlowToIterate_1 I have a variable "fileName"
2. in tFileInputExcel_3 component I dynamically create a file name with this variable "fileName". It works.
3. in tFileInputExcel_2 (lookup excel file) this variable "fileName" is null. It's not working.

In what way I can resolve this problem?

thanks in advance.

Best regards.


Uploaded Images

Last edited by valeksa (2009-07-30 02:19:40)

Offline

#5 2009-07-30 04:02:48

shong
Talend team
Registered: 2007-08-29
Posts: 10305
Website

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

Hello guy
Also use a tFileInputDelimited and tFlowToIterate to iterate all the file name, then use a tUnite to merge all look up records.
tFileInputDelimited-->tFlowToIterate--iterate-->tFileInputExcel_2-->tUnite-->tMap

Best regards

          shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#6 2009-07-30 21:11:01

valeksa
New member
Registered: 2009-07-29
Posts: 3

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

shong wrote:

Hello guy
Also use a tFileInputDelimited and tFlowToIterate to iterate all the file name, then use a tUnite to merge all look up records.
tFileInputDelimited-->tFlowToIterate--iterate-->tFileInputExcel_2-->tUnite-->tMap

Best regards

          shong

thanks for quick reply, but smilesmile

I don't get it. I do not have to merge the content (tUnite component) of "main" and lookup tables.

What I need is in loop of reading input file (tFlowToIterate_1) to make three things:
1. read main excel file (tFileInputExcel_3) with dynamic fileName (I done that)
2. in the same time for each record in main excel (tFileInputExcel_3) to look up for some values in lookup excel file. The name of this lookup excel file depends on filename from point 1. (tMap component)
3. write this record to DB (main component)

and I need this loop to iterate for 20 excel tables and 20 excel lookup tables (both excel tables have dynamic filename).

I have try to use tUnite component but I do not see in what way this component can resolve my problem...maybe I miss something obvious..i do not know

Once more,
thanks for valuable input. I have learn something new..

Regrads.

Offline

#7 2009-07-31 05:09:09

shong
Talend team
Registered: 2007-08-29
Posts: 10305
Website

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

Hello guy

in the same time for each record in main excel (tFileInputExcel_3) to look up for some values in lookup excel file. The name of this lookup excel file depends on filename from point 1.

I get your point now.
To reach this aim, you need put the main and lookup flow in a child job, use a tRunJob component to run this child job in the father job. I have created an example job, please see my screenshots.

Hope you understand well!!
Best regards

         shong


Uploaded Images


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#8 2009-07-31 09:41:22

valeksa
New member
Registered: 2009-07-29
Posts: 3

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

shong wrote:

Hello guy

in the same time for each record in main excel (tFileInputExcel_3) to look up for some values in lookup excel file. The name of this lookup excel file depends on filename from point 1.

I get your point now.
To reach this aim, you need put the main and lookup flow in a child job, use a tRunJob component to run this child job in the father job. I have created an example job, please see my screenshots.

Hope you understand well!!
Best regards

         shong

working great.

thats what I need...

Thanks for your time.

Regards.

Offline

#9 2009-07-31 09:46:33

shong
Talend team
Registered: 2007-08-29
Posts: 10305
Website

Re: [resolved] tFileInputDelimited accepting a variable to dynamically

Hello

working great.

thats what I need...

Thanks for your time.

Good news, wink it is very nice to help you and hope you enjoy Talend!!!

Best regards
     
          shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » [resolved] tFileInputDelimited accepting a variable to dynamically

Board footer

Powered by FluxBB