You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2017-02-28 14:04:00

vishakhamehandru
Member
5 posts

vishakhamehandru said:

Merge two files with undefined schema

Tags: [merge column]

We use the community version of TOS.
My requirement is to transpose the following input:

[table][tr][td]
ID
[/td]
[td]
TEXT
[/td]
[td]
COMMENT
[/td]
[/tr]
[tr][td]

1

[/td]
[td]
Text A1
[/td]
[td]
Comment A1
[/td]
[/tr]
[tr][td]
2

[/td]
[td]
Text B1
[/td]
[td]
Comment B1
[/td]
[/tr]
[tr][td]
3

[/td]
[td]
Text C1
[/td]
[td]
Comment C1
[/td]
[/tr]
[tr][td]
4

[/td]
[td]
Text D1
[/td]
[td]
Comment D1
[/td]
[/tr]
[tr][td]
5

[/td]
[td]
Text E1
[/td]
[td]
Comment E1
[/td]
[/tr]
[/table]
To the following output:
[table][tr][td]
ID 1 Text
[/td]
[td]
ID 1 Comment
[/td]
[td]
ID 2 Text
[/td]
[td]
ID 2 Comment
[/td]
[td]
ID 3 Text
[/td]
[td]
ID 3 Comment
[/td]
[td]
ID 4 Text
[/td]
[td]
ID 4 Comment
[/td]
[td]
ID 5 Text
[/td]
[td]
ID 5 Comment
[/td]
[/tr]
[tr][td]
Text A1
[/td]
[td]
Comment A1
[/td]
[td]
Text B1
[/td]
[td]
Comment B1
[/td]
[td]
Text C1
[/td]
[td]
Comment C1
[/td]
[td]
Text D1
[/td]
[td]
Comment D1
[/td]
[td]
Text E1
[/td]
[td]
Comment E1
[/td]
[/tr]
[/table]
My approach is to use tPivotToColumnsDelimited. But there is a limitation of having only one aggregation column. Due to this I will need to use it twice and then somehow merge the files into one output.
This is what I have done so far.  I have used tPivotToColumnsDelimited, first using pivot column as “ID” and aggregate column as “Text”. Then using pivot column as “ID” and aggregate column as “Comment”. There is a bunch of “group by columns” as well.
This has resulted in two similar delimited files (csv). 
Ideally I could use tMap component to merge the two CSV files.  But I do not have a predefine schema, as the data in the files is not static.  So I am unable to define a schema for input source to tMap. 
Can someone please suggest a way forward? I’ll be grateful for your help.
Regards, Vishakha

Offline

#2 2017-02-28 14:12:05

vishakhamehandru
Member
5 posts

vishakhamehandru said:

Re: Merge two files with undefined schema

Sorry about the formatting of the sample tables. 
My requirement is to transpose rows to columns. I am using tPivotToColumnsDelimited. But this currently allows aggregation of one column.  I have to transpose the row data of three columns, where one column is the common pivot column for the other two columns.
So I have used this component twice, in the hope to merge the two resultant files (CSV) into one output later.

Ideally I could use tMap component to merge the two CSV files.  But I do not have a predefine schema, as the data in the files is not static.  So I am unable to define a schema for input source to tMap. 

Can someone please suggest a way forward? I’ll be grateful for your help.
Regards,
Vishakha

Offline

#3 2017-03-01 14:20:22

vishakhamehandru
Member
5 posts

vishakhamehandru said:

Re: Merge two files with undefined schema

Please can someone respond to my query above?

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy