Talend community forum / [resolved] Rejected records

pgmuthukumar · 2012-06-08 10:08:22

Tags: [error, error management, file error, positional file, rejected records]

HI,
I am loading file use the TOS job:

tfileinputdelimited --(main)--> tOracleOutput

and my rejected_records table with columns (rejected_record, error_field). The rejected_record will be having the entire rejected record and the error_field will be having the field on which the issue is.

sample data:
abc,xyz,123,08/06/2012,1.2
cde,xyb,231,june,1.2

the fourth column is date in target table. however, we got the value as "june" instead of date. Talend successfully loads the first record. Now i need to implement the job in such a way the value in the rejected_records table will be as

'cde,xyb,231,june,1.2', 'june'

Thank you.

janhess · 2012-06-08 10:48:04

Right click on tOracleOutput and select Row Reject to link to your reject record table.

pgmuthukumar · 2012-06-08 11:12:12

HI,
the challenge here is to load the entire row to the "rejected_record" column and error column to the "error_field" as of now this handled by shell script.

pgmuthukumar · 2012-06-11 05:24:43

Hi,
Created a job as below to handled the rejected records.

tFileInputDelimited --(main)--> tOracleOutput
|
|(reject)
V
tmap --(main)--> tOracleOutput

1. the input file has got over 10 fields delimited by ,
2. error scenario: expected - date field in column 3 but file has got a string (that record needs to be rejected)

the issue observed is all values after the above error-ed field are flowing as null.

Is there a way to capture the actual values of each field (even though error fields) from file?

In the attached sample file
error at line 1: 12/312011 expected: 12/31/2011
error at line 3: 10044a expected: 100444

also attached the data in the rejected records table
Thank you.

Uploaded Images

pgmuthukumar · 2012-06-12 05:12:45

input please

avdbrink · 2012-06-12 08:08:58

Hi,

To make sure you get every value from the input file you could move the validation check of your data to a second component (use tSchemaComplianceCheck for example). This will make sure all data is read from the source file and put into a row. You can than, after the schema check use this rows .toString() method to get ALL data available in the source data.

Hope this helps.

Regards,
Arno

Idriss Coowar · 2012-07-26 15:28:18

hi,

I have a problem concerning the rejected records.

Description of my process :

- Input : read an apache log file (tApacheLogInput)

- tLogRow

- Output : tFileOutputDelimited

Issue : The process does not generate the rejected records.

Could someone give me a help about this issue ?

Thanks

janhess · 2012-07-26 15:41:44

Where do you expect the records to be rejected from?

Idriss Coowar · 2012-07-26 15:46:49

From my input file

pgmuthukumar · 2012-07-31 09:33:46

Thanks to avdbrink

janhess · 2012-07-31 10:28:47

Idriss Coowar wrote:
From my input file

For what condition?

Idriss Coowar · 2012-07-31 14:39:14

Hi,

For example, i read a file of 600 000 records and my process generates an excel output file of 567 422 records.

The difference represents 32 578 records which have been rejected.

So i would like to know how to generate a file containing these rejected records and why these records are rejected.

Thanks for your help.

Announcement

Unanswered posts

#1 2012-06-08 10:08:22

[resolved] Rejected records

#2 2012-06-08 10:48:04

Re: [resolved] Rejected records

#3 2012-06-08 11:12:12

Re: [resolved] Rejected records

#4 2012-06-11 05:24:43

Re: [resolved] Rejected records

#5 2012-06-12 05:12:45

Re: [resolved] Rejected records

#6 2012-06-12 08:08:58

Re: [resolved] Rejected records

#7 2012-07-26 15:28:18

Re: [resolved] Rejected records

#8 2012-07-26 15:41:44

Re: [resolved] Rejected records

#9 2012-07-26 15:46:49

Re: [resolved] Rejected records

#10 2012-07-31 09:33:46

Re: [resolved] Rejected records

#11 2012-07-31 10:28:47

Re: [resolved] Rejected records

Idriss Coowar wrote:

#12 2012-07-31 14:39:14

Re: [resolved] Rejected records

Board footer