Talend community forum / How to check if a record Exists in DB

devtripathi0112 · 2012-07-17 20:43:19

I have a folder of excel files
Talend package reads the folder and for each excel file in the folder it unpivots the content of the excel file and inserts in a MSSQL database.
Now the same row/record could exist in multiple files and thats why we keep getting duplicates in the database.
if someone could help me understand, if there is a way to check if the record exists in MSSQL DB, before inserting then i can skip duplicates and just get unique records. Please see the attached screenshot.
Thanks

Uploaded Images

devtripathi0112 · 2012-07-17 21:08:59

Can someone from the expert talend team point me in right direction please. I would highly appreciate it.

jonathanbowen · 2012-07-17 21:10:25

Can you add a unique key to your database table? If you then set the MSSQL component to be "Insert if not exist", this would stop duplicates being inserted.

devtripathi0112 · 2012-07-17 21:21:14

Thank You. Very Helpful. What should i do if i want to update the existing record instead of skipping it? Is there an option in the tMSSQL component for this as well?

jonathanbowen · 2012-07-17 21:27:55

OK - set the action on data to be "Update or insert". This updates existing entries or create it if non existing.

Announcement

Unanswered posts

#1 2012-07-17 20:43:19

How to check if a record Exists in DB

#2 2012-07-17 21:08:59

Re: How to check if a record Exists in DB

#3 2012-07-17 21:10:25

Re: How to check if a record Exists in DB

#4 2012-07-17 21:21:14

Re: How to check if a record Exists in DB

#5 2012-07-17 21:27:55

Re: How to check if a record Exists in DB

Board footer