• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » tDenormalize is not working as expected if used with tFor component.

#1 2008-03-31 08:29:20

JustETL
New member
Registered: 2008-03-05
Posts: 9

tDenormalize is not working as expected if used with tFor component.

Tags: [java, mysql, tos2.2.4]

I am using tFor component to execute a tMysqlInput query n times. and then using a tDenormalize component  before oupting to a file.

No matter how many times I iterate using tFor, only the last batch of rows gets sent to output file. To explain in detail, in tFor I have from=0,to=100,and step=20. In the tMysqlInput the sql is "select id,name,ssn,addr1,addr2 from emp limit "+(Integer)globalMap.get(tFor_CURRENT_ITERATION)+"20";

The out put file should contain 50 rows, instead it has 10 rows. These 10 rows belong to the last iteration.  So the sql with limit 100,20's results gets pumped to output file.

Is there any thing I can do to over come this prob.

Note: I tested by putting tLog beofore and after tDenormalize to see what goes in and out.

Offline

#2 2008-03-31 09:12:54

shong
Talend team
Registered: 2007-08-29
Posts: 10300
Website

Re: tDenormalize is not working as expected if used with tFor component.

Hello

There is a 'Append' option on the basic setting panel of output component, if you don't select this option, the last iteration data will cover the previous data. or, you can use tUnite component between tMysqlInput and tDenormalize to merge all the data.

Best regards

        shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#3 2008-03-31 09:26:14

JustETL
New member
Registered: 2008-03-05
Posts: 9

Re: tDenormalize is not working as expected if used with tFor component.

I did check the append check box. I even inserted a tLogRow before FileOutput to see if the rows gets sent to output component. They don't get sent at all.

Offline

#4 2008-03-31 12:33:34

mhirt
Talend team
Registered: 2006-09-19
Posts: 1638

Re: tDenormalize is not working as expected if used with tFor component.

Hello,

tMysqlInput the sql is "select id,name,ssn,addr1,addr2 from emp limit "+(Integer)globalMap.get(tFor_CURRENT_ITERATION)+"20";

Are you not missing a comma in +",20" ?

Offline

#5 2008-03-31 17:02:29

JustETL
New member
Registered: 2008-03-05
Posts: 9

Re: tDenormalize is not working as expected if used with tFor component.

The actual code does have comma. The one in the above post, I typed, without copying the above one.

Offline

#6 2008-03-31 21:10:55

JustETL
New member
Registered: 2008-03-05
Posts: 9

Re: tDenormalize is not working as expected if used with tFor component.

I think I found the source of problem. It looks like a bug, if some one would confirm, that would be great.

Here is what the generated Java code looks like(I am pasting only those lines that matter w.r.t to the current context):
          public void tFor_1Process() throws TalendException {
        try {
                         globalMap.put("tFor_1_CURRENT_ITERATION", 0);
            while (((Integer) globalMap.get("tFor_1_CURRENT_ITERATION")) <= 100) {     //The iteratiion for iFor component. This is where all the logic for a given iteration resides.
                               globalMap.put("tFor_1_CURRENT_ITERATION", ((Integer) globalMap.get("tFor_1_CURRENT_ITERATION")) + 20);
            }
            tDenormalize_1_DenormalizeInProcess();//This should be within the above for loop, but is outside the loop.
        } catch (Exception e) {
            throw new TalendException(this, e, currentComponent);
        }
    }
   
   //The actual code should be as follows:
    public void tFor_1Process() throws TalendException {
        try {
                         globalMap.put("tFor_1_CURRENT_ITERATION", 0);
            while (((Integer) globalMap.get("tFor_1_CURRENT_ITERATION")) <= 100) {     //The iteratiion for iFor component. This is where all the logic for a given iteration resides.
                                tDenormalize_1_DenormalizeInProcess();//This should be within the above for loop, but is outside the loop.
                               globalMap.put("tFor_1_CURRENT_ITERATION", ((Integer) globalMap.get("tFor_1_CURRENT_ITERATION")) + 20);
            }
        } catch (Exception e) {
            throw new TalendException(this, e, currentComponent);
        }
    }

Please let me know if I am doing something wrong.

Offline

#7 2008-04-01 11:58:18

mhirt
Talend team
Registered: 2006-09-19
Posts: 1638

Re: tDenormalize is not working as expected if used with tFor component.

You're right JustETL, this is a bug in TOS 2.2.4.
It's corrected in TOS 2.3 (2.2.5 will never be launched).
TOS 2.3.2 is available.


Regards,

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » tDenormalize is not working as expected if used with tFor component.

Board footer

Powered by FluxBB