• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Optimized TaggregateRow component when we have loads of column....

#1 2010-12-02 11:08:13

zelefan
Member
Registered: 2010-11-02
Posts: 13

Optimized TaggregateRow component when we have loads of column....

Hi all,
I am new in using TOS and I am very happy until now with the different possibility it allows us to work on our database.
I have a question on the "TaggregateRow" component as I can use it in aI guess the standard way but I wanted to know if there's a faster possibilty to aggrgate my colums. Actually, my problem is that I have a huge amount of table (around 600) and in average I have for each one 100 colums.
All my rows correspond to fifteen minutes and I need to aggregate all by suming all row by day.
I tried it for one table  and it works well but I took loads of time as I need to manually add the output columms name in front of the input columms names. I don't have to change the name of the columms in the output.
So I wanted to know if it exists lors for the Tmap components an automap functionnaly? If not, is someone have an idea to optimize my time for these aggregation?
I saw on the website that there's a specification about that but I don't know if it is already coded?
http://talendforge.org/wiki/doku.php?id … ons:agggui

Thank very much for your help and your time.
Regards,

Patrick


Uploaded Images

Last edited by zelefan (2010-12-02 16:20:15)

Offline

#2 2010-12-02 18:04:47

zelefan
Member
Registered: 2010-11-02
Posts: 13

Re: Optimized TaggregateRow component when we have loads of column....

No one have an idea concerning my issue? I am still trying to change all my output columns manually but I guess that I am gonna to give up my project with TOS. I have to many colums and table to work with to make it manually. On the top of that I have much more aggregation to come with the same number of tables and columns (from hour to day, from day to week and  from day to month)
I's a shame that the automap is not possible yet for aggregation.
I am waiting a little bit more to see if there's a return. If not, I will try to find another ETL if there's one that can do that that kind of thing.
Thank you in advance for your help. I would be pleased even if you juste can give me an idea as I am really blocked at the moment.
Regards,
Patrick VALLET

Offline

#3 2010-12-03 00:47:08

alevy
Member
Registered: 2009-11-20
Posts: 1559

Re: Optimized TaggregateRow component when we have loads of column....

I can sympathise as I've also found this very painful but there is no solution yet.  Feature requests have been submitted (10067 and 16327) potentially for v4.2.0.

Offline

#4 2010-12-03 09:46:00

zelefan
Member
Registered: 2010-11-02
Posts: 13

Re: Optimized TaggregateRow component when we have loads of column....

Thank you very much for your answer. I am going to continue with the current components and hope it would be in a future version.
Regards

Offline

#5 2010-12-03 10:24:09

emaxt6
Member
Registered: 2008-08-04
Posts: 325

Re: Optimized TaggregateRow component when we have loads of column....

For big amount of tables, all requiring the same transformations, maybe is worth the effort to use a custom solution ( maybe using the source produced by talend as basis ).
Take in account that Talend is a code generator and this provides a very big amount of flexibility.

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Optimized TaggregateRow component when we have loads of column....

Board footer

Powered by FluxBB