#1 2010-08-02 20:10:15

Manuel.dion
Guest

Implementing complex join

Hi!

I'm pretty new with talend and I hit a problem. I dont see how I can implement the solution so here it goes...

In a normal lookup, you left join on a key and return several columns. To acheive my goal, i've to do the exact opposite : join on several non key column and return the key if there is a match. In other word, I want to find a pattern an extract the key if it exist. On one side you have several transactions (with several properties) to process. On the other you have several transaction pattern (with several propeties) and the goal is to find one patten that fit each transactions.

The SQL to accheive this is fairly simple:

Code:

select t.fk_transaction_code,
       s.fk_system,
       s.fk_customer,
       s.fk_partner
from   source.v_mapping_source s
       left join ods.v_mapping_target t on (s.fk_system   = t.fk_system            OR t.fk_system            = 0)
                                       and (s.fk_function = t.fk_function          OR t.fk_function          = 0)
                                       and (s.fk_customer = t.fk_netsuite_customer OR t.fk_netsuite_customer = 0)
                                       and (s.fk_partner  = t.fk_netsuite_partner  OR t.fk_netsuite_partner  = 0)

-The left join itself ensure a result for all row in the source
- The OR on each part of the join ensure that if a match is not found, use the default value for the property (wich is zero).

If I put in my join something like:

Code:

       left join ods.v_mapping_target t on s.fk_system   = t.fk_system
                                       and s.fk_function = t.fk_function
                                       and s.fk_customer = t.fk_netsuite_customer
                                       and s.fk_partner  = t.fk_netsuite_partner

I would only hit a result when all 4 propeties were set. the OR allow me to hit a result if one or more property is not set.

#2 2010-08-02 20:12:28

Manuel.dion
Guest

Re: Implementing complex join

My question is how to modify my tmap to hendle these OR clause in the lef join.
I'm going to look for the proper way to add an image of my tmap.

#3 2010-08-02 20:23:42

Manuel.dion
New member
Registered: 2010-08-02
Posts: 7

Re: Implementing complex join

Manuel.dion wrote:

My question is how to modify my tmap to hendle these OR clause in the lef join.
I'm going to look for the proper way to add an image of my tmap.

I put a red square in SQL would be :

Code:

row6.fk_partner = row17.fk_netsuite_partner.

What should I change in the tmap to have

Code:

row6.fk_partner = row17.fk_netsuite_partner OR row17.fk_netsuite_partner = 0

instead?

Any help would be appreciated.
Sorry for my english. Not my first language!


Uploaded Images

Last edited by Manuel.dion (2010-08-02 20:33:54)

Offline

#4 2010-08-03 01:30:34

JohnGarrettMartin
Member
Registered: 2009-01-07
Posts: 762

Re: Implementing complex join

one not-so elegant solution would be to do the join without the OR in the tmap, and then in the inner join reject output table apply an output filter that checks for the zero values. you can then recombine the flows or simply use two different components to do all your inserts.

Offline

#5 2010-08-08 05:35:57

alevy
Member
Registered: 2009-11-20
Posts: 1478

Re: Implementing complex join

Instead of joining those fields directly in the lookup (e.g. your red square), activate the expression filter for the lookup (white arrow with green plus) with the following:

Code:

(row6.fk_system==row17.fk_system || row17.fk_system==0)
&& (row6.fk_function==row17.fk_function || row17.fk_function==0)
&& (row6.fk_customer==row17.fk_netsuite_customer || row17.fk_netsuite_customer==0)
&& (row6.fk_partner==row17.fk_netsuite_partner || row17.fk_netsuite_partner==0)

Offline

#6 2010-08-09 19:53:38

Manuel.dion
New member
Registered: 2010-08-02
Posts: 7

Re: Implementing complex join

This is great. With this solution, you maid me understand so much stuff about the tmap.
Thanks a lot!!!

Offline

Board footer

Powered by FluxBB