• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » delete files more than 45 days (example) old?

#1 2008-05-21 18:36:30

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

delete files more than 45 days (example) old?

So, I figure once I get a list of these files, I can just use tFileDelete to delete them.  (I assume I would feed into tFileList and then make a connection to tFileDelete, like in the pic, but I'm not positive.)


What I don't know is how to read a directory and get a list of the (zip) files that are older that today minus x # of days.


Uploaded Images

Offline

#2 2008-05-22 09:23:13

shong
Talend team
Registered: 2007-08-29
Posts: 11169
Website

Re: delete files more than 45 days (example) old?

Hello

The tFileProperties component(under File-->Management family) can fit your need. With tFileProperties component, you can get the create time of the file.

Best regards

         shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#3 2008-05-22 19:46:10

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

what version is that in?  Cause, its not showing up for me in TOS 2.3.3r13892


Ah, ok, I see it for the Perl version, but I'm in Java...


Uploaded Images

Last edited by strpdsnk (2008-05-22 19:50:39)

Offline

#4 2008-05-22 20:18:12

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

this bug topic ([Bugtracker, bug 3545, fixed] new component tFileProperties) seems to indicate that the java module is completed in 2.4.0RC1.  are the fixes that are in 2.3.3 also in the latest 2.4.0 release?  (RC2)


Guess I'm about to find out.

Offline

#5 2008-05-22 20:41:42

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

ok, so... is there somewhere I can find documentation on tFileProperties?  cause its not in the most recent documentation available here http://www.talend.com/resources/documentation.php
and its not available in the help linked from TOS : http://127.0.0.1:50949/help/nav/2_2

or can someone tell me how to use this thing? please!

Last edited by strpdsnk (2008-05-22 21:02:40)

Offline

#6 2008-05-23 04:53:00

shong
Talend team
Registered: 2007-08-29
Posts: 11169
Website

Re: delete files more than 45 days (example) old?

Hello

The tFileProperties component describes the properties of one file, include:
abs_path: the absolute path
dirname: the direcotry name
basename: the file name
mode_string: read or write
size: the size of file
mtime: the create time of file, return millisecond (long)
mtime_string: the create time of file, return a date (String)

Here is a simple scenario(see the screenshot)

Best regards

         shong


Uploaded Images


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#7 2008-05-29 21:34:05

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

ok... now I can't figure out how I can filter on mtime_string and return either abs_path or basename to tFileList or tFileDelete

I mean, I think I can do some of the filtering in something like tMap, but I don't know if thats the right way to do it

Offline

#8 2008-05-30 04:37:14

shong
Talend team
Registered: 2007-08-29
Posts: 11169
Website

Re: delete files more than 45 days (example) old?

Hello

ok... now I can't figure out how I can filter on mtime_string and return either abs_path or basename to tFileList or tFileDelete

Here is a Java example, Please see the screenshot.

Code:

 /**
     * Returns the number of milliseconds since January 1, 1970, 00:00:00 GMT
     * represented by this <tt>Date</tt> object.
     *
     * @return  the number of milliseconds since January 1, 1970, 00:00:00 GMT
     *          represented by this date.
     */
    public long getTime() {
        return getTimeImpl();
    }

Best regards

         shong


Uploaded Images


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#9 2008-05-30 21:15:16

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

So I tried this process like you said, and it works perfectly... that is, until I try to change the # of days to anything greater than 24

I tried a bunch of different days, and all the ones <=24 worked, and all the ones >24 caused all the files to be deleted

Picture Information
1. contents of the folder before I run the clean up process
2. process with tFilterRow set to delete files created before 25 days
3. results of running the process defined in #2, where all files are deleted (whether they meet the filter criteria or not : before 5/6/08)
2. process with tFilterRow set to delete files created before 24 days
3. results of running the process defined in #4, where only the appropriate files are deleted (any files before 5/7/08)

Note:
tFileOutputDelimited_2 creates out_orig.txt                  (a list of all the files found in tFileList_3)
tFileOutputDelimited_1 creates out.txt                         (a list of all the files deleted by tFileDelete_1)
tFileOutputDelimited_3 creates out_notdeleted.txt        (a list of all the files rejected by tFilterRow_1)


I find it kind of odd that the advanced filter in tFilterRow_1 works where ### (in the quoted formula below) is <=24, but not greater...

((TalendDate.getCurrentDate().getTime()-(table1.mtime+86400000*###))>0


Uploaded Images

Last edited by strpdsnk (2008-05-30 21:16:23)

Offline

#10 2008-05-30 22:28:24

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: delete files more than 45 days (example) old?

Ok, I figured out how to get around the problem I posted above:
1. set ### (in the post above) to a context variable (ex: context.NumDays_Past)
2. set the Type of that context variable to Double
      a. You could use Float or Long, in place of Double, here - they both worked for me
      b. Don't use Integer or Short, they cause all the files in the folder to delete.
      c. Don't use Big Integer, it causes an error (Exception in thread "main" java.lang.Error: Unresolved compilation problem: The operator * is undefined for the argument type(s) int, BigDecimal)


(TalendDate.getCurrentDate().getTime()-(86400000*context.NumDays_Past))>table1.mtime

Offline

#11 2012-02-28 04:42:07

ainiesta
Guest

Re: delete files more than 45 days (example) old?

Hi, I am having error to this context.NumDays_Past

nullpointerexception.

what values you put in context.NumDays_Past?

Regards.

#12 2014-01-09 11:53:57

jzhao_talend
New member
Registered: 2013-12-26
Posts: 1

Re: delete files more than 45 days (example) old?

Hi,I have reproduce the problem.
This is cause by 60*60*24*1000*25 out of scope of Integer type(result is: -2134967296)
So the result alway false
Solution:change 60*60*24*1000*25 ---to---> 60*60*24*1000*25L(result is: 2160000000)
Change the result is long type.

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » delete files more than 45 days (example) old?

Board footer

Powered by FluxBB