• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » tAggregateSortedRow ignoring last grouping

#1 2008-02-26 03:22:21

cmj556
Member
Company: Lunexa
Registered: 2007-11-14
Posts: 15
Website

tAggregateSortedRow ignoring last grouping

I'm using tAggregateSortedRow to dedup a selection on one key.  (tUniqRow won't work because I need the last row instead of the first)  The component seems to be ignoring the last key group entirely.  For example, with the following input data, I get only two rows output:

a,1
a,2
a,3
a,4
b,5
b,6
c,7
c,8
c,9
c,0

The output is:
a,4
b,6

I'm using 2.3.0 Java on WinXP 32bit.  Any ideas?
   Chris.


Uploaded Images

Offline

#2 2008-02-26 06:40:56

shong
Talend team
Registered: 2007-08-29
Posts: 10353
Website

Re: tAggregateSortedRow ignoring last grouping

Hi

It is a [Bugtracker, bug 2511, fixed] Java tAggregateSortedRow forgets keys. but the bug isn't fixed yet.
To solve this issue, you can add a null row as the last row in your input data to get the right result.

Best regards

          shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#3 2008-03-03 02:53:03

psm2000
Member
Registered: 2008-02-25
Posts: 108

Re: tAggregateSortedRow ignoring last grouping

I am new to TOS and had the same issue. It is nice that there is a warning but as a newbie I've a question on this.

Should I use a NULL row file and use tUnite anytime I want to use this object? Will it cause any unintended issues? I think it is better than using the tExternalSortedRow as I coulnd not get it to work.

Thanks in advance.

Sean

Offline

#4 2008-03-03 03:36:05

shong
Talend team
Registered: 2007-08-29
Posts: 10353
Website

Re: tAggregateSortedRow ignoring last grouping

Hi Sean

Should I use a NULL row file and use tUnite anytime I want to use this object? Will it cause any unintended issues?

Currently, tAggregateSortedRow always ignore the  last groupin(of course, it is a bug and we will fix it). So now, if you want to use this componennt to get a right result, you should add a null row as the last row.

Best regards

           shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#5 2008-03-03 08:46:40

plegall
Member
Registered: 2006-09-19
Posts: 1586
Website

Re: tAggregateSortedRow ignoring last grouping

shong wrote:

Currently, tAggregateSortedRow always ignore the  last groupin(of course, it is a bug and we will fix it).

I'm not so sure about this :-). See my notes in [Bugtracker, feature 503, fixed] new component tAggregateSortedRow and [Bugtracker, feature 2511, fixed] Java tAggregateSortedRow forgets keys

Offline

#6 2008-03-03 17:51:45

psm2000
Member
Registered: 2008-02-25
Posts: 108

Re: tAggregateSortedRow ignoring last grouping

plegall:

I tried to use a NULL file combined with tUnite. It did not work. May be because I am not sure what a NULL row is in a flat file context. Also the Word doc in one of the bug trackers was difficult for me to follow (probably due to my newbie status with TOS).

In any case, tASR makes sense if the tSort is much better at sorting files than aggregate component (at this time I do not have any DB inputs but just excel and flat files). Else just regular aggregate can do the same job of Sort + tASR.

I am from Informatica world where SORT component is very good compared to AGGREGATE and as such tend to use SORT and mark the AGGREGATE as "Sorted Input" which is same as using tASR.

If the bugtracker on this is closed, I think it should be opened, right?

Thank you for your help here and with other issues.

Regards,
Sean

Offline

#7 2008-03-11 18:28:36

Vaiko
Member
Registered: 2008-01-16
Posts: 125

Re: tAggregateSortedRow ignoring last grouping

I just run into this issue. Do you have any feedback for which release you are planning the fix?


Uploaded Images

Offline

#8 2008-03-12 00:24:34

plegall
Member
Registered: 2006-09-19
Posts: 1586
Website

Re: tAggregateSortedRow ignoring last grouping

Vaiko wrote:

I just run into this issue. Do you have any feedback for which release you are planning the fix?

As you can read in [Bugtracker, bug 2511, fixed] Java tAggregateSortedRow forgets keys, slanglois has added the solution I suggested, ie a new property "expected number of rows". He has commited his code a very short time ago and the solution will be available in next milestone (not in next main release, not in 2.3.2 but in 2.4.0M1). The solution was implemented only for the Java component, not the Perl component yet.

This solution is not a smart solution, but we currently have no other smarter solution. Any suggestion is welcomed.

Offline

#9 2010-12-15 19:45:22

eeedmain
New member
Registered: 2010-10-27
Posts: 2

Re: tAggregateSortedRow ignoring last grouping

Wow... I have run into this bug in 4.1.0, and it is over two years later.  Is this ever going to be fixed?

I can't imagine a more important component!

Offline

#10 2012-01-27 21:01:53

Ekevoo
New member
Registered: 2012-01-27
Posts: 1

Re: tAggregateSortedRow ignoring last grouping

Good morning, gentlement from the past.

I come from 2012 where we have TOS 5.0.1 to let you know that these promises are empty and this incredibly simple bug still hasn't been fixed.

There are simple ways to work around it by using tAggregateRow and, for cases of selection aggregate functions such as "first" or "max", you can use tSortRow with tUniqRow, so there's nothing vital about this component.

It is, however, a shameful blunder to keep such a harmful bug available to the uninformed user. My co-worker lost a week fruitlessly on this thinking her selects were wrong, and I lost two days until I got suspicious of the component.

Offline

#11 2012-06-13 10:51:39

elvis
Member
Registered: 2011-12-22
Posts: 15

Re: tAggregateSortedRow ignoring last grouping

Anyone know if it's been fixed in 5.1.1?

Of course if tSortRow exposed the number of rows it has processed then it would make using tASR a lot more useful!

Last edited by elvis (2012-06-13 10:53:10)

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » tAggregateSortedRow ignoring last grouping

Board footer

Powered by FluxBB