Talend community forum / Split Columns to different flows

Lilth · 2012-04-13 13:48:19

Hi,

I am using TIS Pro 4.2.3 version. I have a requirement of splitting columns based on Headers from a CSV file into multiple data flows and update tables.

Say for ex, my CSV file is as below:
DM_Col1,DM_Col2,BIO_Col1,BIO_Col2,BIO_Col3
1,A,1,B,C,D
2,A,1,E,F,G
3,H,5,S,W,Q

Now based on the header 'DM' or 'BIO' I have to split into two data flows and update DM_Table and BIO_Table accordingly.

My columns can increase or decrease. So the number of data flow splits also can increase or decrease.

Can anyone throw some light on this topic?

Thanks in advance,
Lilth

vgalopin · 2012-04-13 19:49:26

Hi Lilth,

The only way I can think about is using Dynamic Schemas.

Basically the Dynamic Schemas are going to allow you to get data from your csv file, without defining a schema. Then you can use specific functions related to the Dynamic Schema to get the column name, and send the data to a flow or another based on that.

Lilth · 2012-04-16 06:29:37

Hi,

Thanks a lot for the reply. Helped a lot.

Regards,
Lilth

Lilth · 2012-04-16 11:46:56

Now I am stuck up with another problem. I am not able to delete dynamic elements using the function, removeDynamicElement(String columnName).

For the first row input, header and values are deleted but for the second or third rows values are not getting deleted.

Col1,Col2,Col3,Col4
Value1,Value2,Value3,Value4
Value5,Value6,Value7,Value8
Value9,Value10,Value11,Value12

I have to delete second column, i.e. Col2, but using removeDynamicElement(Col2) function only "Col2" and Value2 are getting deleted. "Value6" and "Value10" remain the same.

Is this a bug or is there an issue with my logic??

Regards,
Lilth

Announcement

Unanswered posts

#1 2012-04-13 13:48:19

Split Columns to different flows

#2 2012-04-13 19:49:26

Re: Split Columns to different flows

#3 2012-04-16 06:29:37

Re: Split Columns to different flows

#4 2012-04-16 11:46:56

Re: Split Columns to different flows

Board footer