Talend community forum / Most recent file - PERL

Rodney · 2009-08-26 22:18:55

I use Perl version to gather files from FTP. How can I get the most recent file?

Example: today is 08262009

I have files in my FTP "talend-08232009 and "talend-08202009"

How do I write the FTPget and TFileList to find the most current file? i.e. "talend-08232009"

shong · 2009-08-27 04:35:42

Hello friend
1)Define a context var called filemask in the job.
2)Use a tFTPFileList to iterate each file, parse the file name and cut the file name to get the string Date, eg:talend-08202009-->08202009, parse the string Date to a Date, and then compare this date to the current Date, get the most recent date, and set this file name to context var.
3)On tFTPGet, set the file mask with context var, eg: context.filemask

Best regards

shong

Rodney · 2009-08-27 23:22:58

Hi Shong,

I am confused on how to set the context variable and parse the date. Do I have to do this in a routine?

shong · 2009-08-28 04:36:53

Hello Rodney
In fact, I am not a Perl programmer. I know how to do that in Java, I will send this topic to our perl developper.

Best regards

shong

plegall · 2009-08-28 11:54:44

The Perl tFTPFileList is really missing in this situation. So instead of proposing a very complex solution, I've quickly coded tFTPFileList for Perl projects. You can find it on Talend Exchange: [Talend Exchange item by plegall][Component, Perl] tFTPFileList (install it in your Talend Open Studio with the "Talend Exchange" view, it's all automatic).

I should have used a tSetGlobalVar instead of a tPerlRow, but unfortunately the auto-completed syntax $row[filename] is not available in this very place. I don't know why, I'll create a bug notification.

In the tPerlRow, we have:

Code:

$_globals{latest_file} = $input_row[filename];

The most tricky method is how I generate the date_iso from the filename in the tMap:

Code:

join '', ($row1[filename] =~ m/(\d\d)(\d\d)(\d\d\d\d)/)[2,0,1]

tSortRow is useful to have the latest file on top of the list. tSampleRow let me take only the first row of my flow.

Uploaded Images

Announcement

Unanswered posts

#1 2009-08-26 22:18:55

Most recent file - PERL

#2 2009-08-27 04:35:42

Re: Most recent file - PERL

#3 2009-08-27 23:22:58

Re: Most recent file - PERL

#4 2009-08-28 04:36:53

Re: Most recent file - PERL

#5 2009-08-28 11:54:44

Re: Most recent file - PERL

Code:

Code:

Board footer