• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » performance issue on a tAggregate

#1 2008-11-08 14:57:13

Mumuri
Member
Company:
Registered: 2008-07-15
Posts: 24
Website

performance issue on a tAggregate

Hello

i ve got big problem of performance with a tAggregateRow

40 000 rows arrive on the tAggregateRow ,

for the rest of the component the performance are ok, but for this one it's very long

is there a solution to solve the problem (we are using 2.4.0)

Here is a sysout of the start and end hashes for each component

http://luffyserveur.free.fr/test.PNG

the component tAggregateRow  has the following setting
- group by 2 fields
- sum on 15 fields
- min on 15 fields

thanks in advance

Last edited by Mumuri (2008-11-08 15:03:09)

Offline

#2 2008-11-10 19:02:39

amaumont
Talend team
Registered: 2006-09-20
Posts: 471

Re: performance issue on a tAggregate

I will submit a new optimized (memory and cpu) version of the tAggregateRow this week into the ecosystem.

I will write a new post at this time.

Offline

#3 2008-11-12 13:28:12

Mumuri
Member
Company:
Registered: 2008-07-15
Posts: 24
Website

Re: performance issue on a tAggregate

thanks i will wait for him

Offline

#4 2008-11-12 15:57:26

amaumont
Talend team
Registered: 2006-09-20
Posts: 471

Re: performance issue on a tAggregate

You can now install it from http://www.talendforge.org/ext/index.php or from TOS Ecosystem view.

Warning, because of a current limitation of the Ecosystem you have to install the three following packages which represents one component :
- tAggregateRowOpt
- tAggregateRowOptIn
- tAggregateRowOptOut

I tested it on 2.4.0 but unfortunately it does not work on this version for any reason ... Yet it works on 2.3.3 (with a minor modification) and 2.4.1, 2.4.2, 2.4.3, 3.0.0 and 3.0.1.

Maybe you will be more lucky.

Last edited by amaumont (2008-11-14 11:59:30)

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » performance issue on a tAggregate

Board footer

Powered by FluxBB