• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Issue renaming fields of a CSV file dynamicaly

#1 2011-07-08 15:36:22

justinedr71
Member
Registered: 2011-05-03
Posts: 12

Issue renaming fields of a CSV file dynamicaly

Hello,

I have an issue for ages and I HAVE to resolve it before the end of next week...

I am working on Talend Open Studio 4.1.2. version with an Oracle DataBase (11g).

First, there is the insurance software, wich is installed in each insurance company, and works with company's own datas. Then, the part I have to work on, is the realization of two CSV files, created from these datas. I have a problem on the first one, the contract file.

Datas to create this file are available on different location (tables). I make my talend application, begining with a request that pick all needed datas up  to find each insurance contract with its associated premium (one per each cover subscribed by the insurance contract).

I have something like that :

contract_ID    client_ID    contract_version        cover_Name       cover_premium
1000000001              1                           1                      X1                        100 
1000000001              1                           1                      X3                        150   
1000000001              1                           1                      X4                          20
1000000001              1                           2                      X1                        100
1000000001              1                           2                      X2                        200
1000000001              1                           2                      X3                        150
1000000001              1                           2                      X4                          20
1000000001              1                           3                      X1                        100
1000000001              1                           3                      X3                        150
1000000002              2                           1                      X4                         100
1000000002              2                           2                      X2                         100
1000000002              2                           2                      X4                         100
1000000002              2                           3                      X2                         100
1000000002              2                           3                      X3                           50
1000000002              2                           3                      X4                         100
1000000002              2                           4                      X1                         500
1000000002              2                           4                      X3                           50
1000000002              2                           4                      X4                         100


the fact is, depending on the company, it could be many different kind of cover, parameters are defined at the begining of the software use, so I can't rename the heading fields of my file directly, I have to select all available cover in my client base, then create the heading fields in my file, and then fil them depending on the contract subscribed the cover or not (there is a 0 if it didn't)

What I want to do, is renaming heading field of my file with the good cover name, for example :

contract_ID    client_ID    contract_version       cover_X1       cover_X2         cover_X3        cover_X4
1000000001              1                           1              100                   0                  150                 20
1000000001              1                           2              100                200                  150                 20
1000000001              1                           3              100                   0                  150                  0
1000000002              2                           1                  0                   0                     0                100
1000000002              2                           2                  0                100                     0                100
1000000002              2                           3                  0                100                   50                100
1000000002              2                           4               500                   0                   50                100

For now, I can create this file :

contract_ID    client_ID    contract_version        cover_A    premium_A       cover_B    premium_B       cover_C    premium_C       cover_D    premium_D
1000000001              1                           1               X1               100                X3               150               X4                 20
1000000001              1                           2               X1               100                X2               200               X3               150               X4                 20
1000000001              1                           3               X1               100                X3               150
1000000002              2                           1               X4               100
1000000002              2                           2               X2               100                X4               100
1000000002              2                           3               X2               100                X3                 50               X4               100
1000000002              2                           4               X1               500                X3                 50               X4               100

But it is not correct... Because diferent cover are in the same column... And cover heading fields are not renamed (name of covers are in a separate columns, wich doesn't have to exist)

I hope I made myself clear... If you have any questions don't hesitate... I really need help, I posted on many forums, no-one could help me...
Thank you!

Justine

Offline

#2 2011-07-10 03:36:10

alevy
Member
Registered: 2009-11-20
Posts: 1477

Re: Issue renaming fields of a CSV file dynamicaly

It looks like you need to use the tPivotToColumnsDelimited component...

Offline

#3 2011-07-10 11:30:22

Justine
Guest

Re: Issue renaming fields of a CSV file dynamicaly

Thanks for answering.

Do you have any idea I can use it? Components succession?
Do I have to make another request to get all the available covers, and then reinject them in the file with the tPivotToColumnsDelimited?
I really don't know how to do that...

#4 2011-07-10 11:31:20

justinedr71
Member
Registered: 2011-05-03
Posts: 12

Re: Issue renaming fields of a CSV file dynamicaly

Thanks for answering.

Do you have any idea I can use it? Components succession?
Do I have to make another request to get all the available covers, and then reinject them in the file with the tPivotToColumnsDelimited?
I really don't know how to do that...

Offline

#5 2011-07-12 14:47:58

justinedr71
Member
Registered: 2011-05-03
Posts: 12

Re: Issue renaming fields of a CSV file dynamicaly

Thank you so much! It works smile smile.

But now, I have another problem, I just have to fill with a zero, values for covers not suscribed by the contract... Instead of leave the field blanck... How can I do that...?

Offline

#6 2011-07-13 01:18:49

alevy
Member
Registered: 2009-11-20
Posts: 1477

Re: Issue renaming fields of a CSV file dynamicaly

It appears the component will not do this for you.  You'll have to make a pass of the result file, filling in the missing values.

Offline

#7 2011-07-13 08:44:24

justinedr71
Member
Registered: 2011-05-03
Posts: 12

Re: Issue renaming fields of a CSV file dynamicaly

I thought about this solution, but as when I open de file I don't know the schema (it is dynamic, depending on numbers and names of different covers), I don't know how to do that...

I thought about something like "each time I found a null field, I replace it with the zero value". I tried in a tJavaRow and a tJavaFlex but they need column name... It doesn't work neither in a tMap, the component need the entry schema of rows...

Any idea?

Offline

#8 2011-07-13 08:54:24

alevy
Member
Registered: 2009-11-20
Posts: 1477

Re: Issue renaming fields of a CSV file dynamicaly

tFileInputFullRow --> tReplace --> tFileOutputDelimited

tReplace should be in Advanced mode with Pattern ",(?=,)|,$" and Replace ",0"

That will replace any instances of double-comma with ,0, and add 0 at the end if the last character is a comma.

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Issue renaming fields of a CSV file dynamicaly

Board footer

Powered by FluxBB