• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Perform a dynamic select along a flow inside a subprocess

#1 2008-04-03 00:35:37

g.lorent
Member
Registered: 2007-08-17
Posts: 50

Perform a dynamic select along a flow inside a subprocess

Tags: [java, sql, tos2.3.2]

Hi,

I still didn't find out how to do such a thing :
Say I am processing an input file and along the subprocess I need to perform a dynamic sql select to enrich my flow.

In such a case the sql lookups are loaded at the beginning of the file processing, so that it does not fit since it's kind of a 'static' sql select. Same issue with the new tJoin.
The base problem of lookups is that it is static data.

I found a workaround processing the end of the subprocess in a child job (using tRunJob) but it is not always feasible. Another workaround would be to split my input file in multiple input files of one line each, but I really try to avoid that.

The tDBRow Component could do the trick, but from the generated code I see that it does not deal at all with the incoming/outcoming flows. Moreover both flows have to be identical...

In fact I need something like :

Flow A --> MagicComponent performing some sql select (optionally using part of Flow A in the where clause) --> Flow B containing Some columns of flow A + some columns of the sql ResultSet

Any light appreciated

Offline

#2 2008-04-03 00:51:50

mhirt
Talend team
Registered: 2006-09-19
Posts: 1641

Re: Perform a dynamic select along a flow inside a subprocess

Hello,

This is funny because, I have just created a feature for DbRow components which will exactly resolve your problem.
You can monitor [Bugtracker, bug 3477, fixed] tDatabaseRow components : Optionally add query ResultSet in current schema

With current version, this can be done but sounds really tricky :
A --> tDBRow(select) ->tJavaRow -> tParseRecordSet -> B

Regards,

Offline

#3 2008-06-20 21:25:26

SMaz
Member
Registered: 2008-04-21
Posts: 186
Website

Re: Perform a dynamic select along a flow inside a subprocess

Could you please give an example of how to use this.

I've got a tDB2Row component which issues a SELECT COUNT(*) query on a table named in the input flow.
I want the results of that query to become part of the output flow of that component.
I've defined a "new" field in the incoming flow (as Object seems to be the only way it will work).

In the Advanced Settings tab, I've checked "Propagate QUERY'S recordset" and chose my "new" column name.
From reading elsewhere, it looks like I have to use tParseRecordSet to extract that data and insert it into the flow.

But, I cannot find any documentation on tParseRecordSet anywhere.  Nor any examples in the Blogs, tutorials, wiki or forum.

Here's my jonb


Uploaded Images

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Perform a dynamic select along a flow inside a subprocess

Board footer

Powered by FluxBB