Talend community forum / My proccess gathers information from a catalog, how to?

mauricioabreua · 2012-07-21 16:40:02

Hello,

I've been spent some hours looking for a similar problem to mine and i could not.
My process is based on:
*Getting address information, users and passwords (ftp, database files) and files with patterns (such as file_miami_beach_20120402.rar, file_new_york_20120403.rar);
*Download these files to internal server;
*Execute pre-load procedures (shell, Oracle procedure);
*Load these files with Oracle Loader;
*Execute pos-load procedures (shell, Oracle procedure);
*Copy table to customer database;
*Send e-amail notifying the process status.

So, how do i gather information from a catalog and use it to connect to customers servers (such as SCP, FTP or external table too) and use this information through the process?

pedro · 2012-07-23 03:29:25

Hi

Welcome to Talend Community!
For process 1 and 2, you might use the component tFTPGet to donwload the rar files from the FTP server.
For process 3, use tPrejob to execute pre-load proceduces which depends on your onw job logic.
For process 4, usually please use the component tSystem to execute rar unarchive commands to unarchive these rar files. Then load files into oracle.
For process 5, use tPostjob.
For process 6, create a subjob for your requirement.
For process 7, use the component tSendMail.

I think you'd better download the document [TalendOpenStudio_Components_RG_xxx_EN] to get the instructions of those components.

Regards,
Pedro

mauricioabreua · 2012-07-23 18:06:14

Thank you.
I'm gonna try this tips.

mauricioabreua · 2012-07-23 19:01:52

I've downloaded the document you said above.
So yes, my ftp is working now, thanks.
But when i list directories, i got a error and there is a "1 exec finished" message over the component.
Can i output this files/directories list for a log or console?

mauricioabreua · 2012-07-23 19:43:10

So, my informations about ftp, passwords, users, directories and files to be searched are saved into the database (Oracle). How do i gather this information for future use?
Sorry, but i i did not find this info in any tutorials.

pedro · 2012-07-24 04:04:28

Hi

Which component do you use to list directories?
What's the error log?

Regards,
Pedro

mauricioabreua · 2012-07-24 15:38:31

Hello,

I'm using tFTPFileList.
Well, it worked after hours reading some posts in the forum.

How can i use my data from Oracle and use it in my jobs?

1 - Connect to Oracle
2 - Get info abou the process
3 - Use this info to log in FTP, get files from FTP (this list is generated by one query)
And so on...

I know i need to use tOracleInput -> tLogRow -> [need help after this]

Thanks!!!

alevy · 2012-07-25 03:08:35

tFlowToIterate is what you need

mauricioabreua · 2012-07-25 18:26:03

Sorry, i've tried to match this component with my problem, but i did not see how to connect a Oracle Query (returning rows) and using this data with tFTPGet component.

alevy · 2012-07-26 08:15:04

It's simply tOracleInput --row--> tFlowToIterate --iterate--> tFTPGet.

Use tFlowToIterate to populate globalMap variables with the values coming from Oracle and then use those variables in tFTPGet or you can use the row.column values directly.

It's not the best example but read the component help (see http://jira.talendforge.org/browse/DOCT-761 for corrections). There are also many posts on the forum on doing this sort of thing.

Last edited by alevy (2012-07-26 08:20:42)

Announcement

Unanswered posts

#1 2012-07-21 16:40:02

My proccess gathers information from a catalog, how to?

#2 2012-07-23 03:29:25

Re: My proccess gathers information from a catalog, how to?

#3 2012-07-23 18:06:14

Re: My proccess gathers information from a catalog, how to?

#4 2012-07-23 19:01:52

Re: My proccess gathers information from a catalog, how to?

#5 2012-07-23 19:43:10

Re: My proccess gathers information from a catalog, how to?

#6 2012-07-24 04:04:28

Re: My proccess gathers information from a catalog, how to?

#7 2012-07-24 15:38:31

Re: My proccess gathers information from a catalog, how to?

#8 2012-07-25 03:08:35

Re: My proccess gathers information from a catalog, how to?

#9 2012-07-25 18:26:03

Re: My proccess gathers information from a catalog, how to?

#10 2012-07-26 08:15:04

Re: My proccess gathers information from a catalog, how to?

Board footer