• Index
  •  » Talend Enterprise Data Integration » General questions
  •  » Dynamic data type making truncation if the delimiter count is more

#1 2012-05-22 12:44:27

karthikj18
Member
Registered: 2011-08-11
Posts: 57

Dynamic data type making truncation if the delimiter count is more

Hi,
I have chosen the column as Dynamic and I am trying to get the column count based on the delimiter. But what I notice is the Dynamic type sets the column count based on the first row and if the next row consist of more delimiters than first  row then its getting truncated.

For example.

a1,b2,c3,d4    ----1st row
a1,b2,c3,d4,e5 ----2nd row

each row is Dynamic type based on the delimiter comma(,).

When I try to see this rows using tlogrow after a tfileinputdelimited..
its shows as follows :

a1,b2,c3,d4   
a1,b2,c3,d4
the 2nd row last field is being truncated.

Is there any solution for this?
Thanks.


Thanks,
Karthikeyan.

Offline

#2 2012-05-22 17:10:56

phobucket
Member
Company: Knoetry
Registered: 2010-07-27
Posts: 146
Website

Re: Dynamic data type making truncation if the delimiter count is more

You could import the the entire table using tInputFileFullRow,  put that into a tMap and add a field num_cols, the sort by num_cols descending.


tFileInputFullRow --> tMap (num_cols = (count of commas in field1 + 1)) --> tSortRow (by num_cols descending) --> rest of job with dynamic schema

Thanks,
Ben

Offline

#3 2012-05-23 08:44:07

karthikj18
Member
Registered: 2011-08-11
Posts: 57

Re: Dynamic data type making truncation if the delimiter count is more

Thanks for the reply.
Can you please help me how to convert string to dynamic in tmap?
Thanks.


Thanks,
Karthikeyan.

Offline

  • Index
  •  » Talend Enterprise Data Integration » General questions
  •  » Dynamic data type making truncation if the delimiter count is more

Board footer

Powered by FluxBB