Talend community forum / tFileInputXML -> tFlowToIterate -?> tJoin

Yves · 2011-02-25 17:33:44

Hi,

I have a number of patient IDs in an XML file. Since it contains patient notes, there are duplicates when extracting all IDs. For each unique patient I add a new entry in a lookup table that maps the patient's ID to a new meaningless ID. There's probably a few approaches to doing this, and here's mine:

tFileInputXML -> tFlowToIterate -?> tJoin -> (got it from here)

Now I want to do a tJoin for each ID from the XML with the lookup table. Each rejected inner join means the patient ID is not yet in the lookup table so I'll add it downstream. There is probably something I am not yet grasping (I'm new to Talend) but I can't connect the tFlowToIterate to the tJoin.

Thanks for any hint,
Yves

JohnGarrettMartin · 2011-02-25 17:38:49

You will need some way to extract the values from your flowToIterate and put them back in a flow. Take a look at the tFixedFlowInput component.

All of the column values from your xml file will be pushed into the global map by your tFlowToIterate-- take a look at the code tab to see exactly what they are.

Yves · 2011-02-25 17:40:58

Thanks, I'll give it a go.

Yves · 2011-02-25 18:38:51

Hi,

Your suggestion works but I'm not sure whether I really need to iterate since the result after tFileInputXML seems no different...

My purpose for using tJoin is to filter out duplicate patient IDs which already have an entry in the lookup table. Only for the inner join rejects I add a new entry that maps the ID to a new (meaningless) ID.

tMSSqlInput -[lookup]-> tJoin -[Inner join reject]-> tMSSQLOutput

Strangely, it's as if the lookup happens only once regardless the number of iterations.

Any hints?

Yves

JohnGarrettMartin · 2011-02-25 19:28:08

Lookups are loaded once per subjob

Yves · 2011-02-25 19:42:05

Aah. Good. This probably saved me more than a couple of hours!

Yves · 2011-02-25 20:16:54

Would you agree it is better to remove duplicates using tUniq <i>before</i> doing the tJoins then? Having each tJoin done in a subjob for the sake of incorporating the latest additions to the lookup table would unnecessarily complicate things, no?

Announcement

Unanswered posts

#1 2011-02-25 17:33:44

tFileInputXML -> tFlowToIterate -?> tJoin

#2 2011-02-25 17:38:49

Re: tFileInputXML -> tFlowToIterate -?> tJoin

#3 2011-02-25 17:40:58

Re: tFileInputXML -> tFlowToIterate -?> tJoin

#4 2011-02-25 18:38:51

Re: tFileInputXML -> tFlowToIterate -?> tJoin

#5 2011-02-25 19:28:08

Re: tFileInputXML -> tFlowToIterate -?> tJoin

#6 2011-02-25 19:42:05

Re: tFileInputXML -> tFlowToIterate -?> tJoin

#7 2011-02-25 20:16:54

Re: tFileInputXML -> tFlowToIterate -?> tJoin

Board footer