• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Implementing rollbacks & other business logic with Talend Open Studio

#1 2011-09-21 23:50:19

Nathan G.
Guest

Implementing rollbacks & other business logic with Talend Open Studio

Hi all,

I am using Talend Open Studio to do some data syncing, but I have a question about how much capability this version has to do things like rollbacks and implement business rules.

For example, I may have 2 CSV files, one with parent records and one with child records.
If the parent record uploads/syncs successfully, but the child records fail to insert, then I would like to have it rollback so there aren't orphan records.

It looks like this is covered through "Business Rules" under the Implementation section on their feature matrix:
http://www.talend.com/products-data-integration/matrix.php

But it only comes with the commercial suite, not the open source version. Is there a way of implementing something like this in Open Studio?

Thanks,
Nathan

#2 2011-09-22 00:21:00

alevy
Member
Registered: 2009-11-20
Posts: 1478

Re: Implementing rollbacks & other business logic with Talend Open Studio

Yes, design your job so that the outputs used a shared connection and "Die on error", then call the rollback component with an OnSubjobError link.

Offline

#3 2011-09-22 00:38:19

Nathan G.
Guest

Re: Implementing rollbacks & other business logic with Talend Open Studio

Hi alevy,

Thank you for your reply.

I can try what you suggest, but wouldn't that only work to roll back the insert that failed?  What about other inserts that were successful, but that should then be rolled back because of the failure?  Sort of like setting up a transaction.

Thanks,
Nathan

#4 2011-09-22 07:35:53

shong
Talend team
Registered: 2007-08-29
Posts: 10297
Website

Re: Implementing rollbacks & other business logic with Talend Open Studio

Hi
It is very possible, but the job design depends on your request, do you want to rollback all the records include the parent records and child records? or either of them? Can you upload a screenshot of job you are working on now?

Best regards
Shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#5 2011-09-22 19:11:06

Nathan G.
Guest

Re: Implementing rollbacks & other business logic with Talend Open Studio

Hi shong,

I actually don't have the job set up yet so I can't take a screenshot of anything.

To answer your question though, in my example, if the parent record gets inserted successfully and the child records fail, I'd like both the parent and child records to be rolled back.  Essentially, the each parent and associated child record insert is a "transaction."

Thanks,
Nathan

#6 2011-09-23 10:25:08

janhess
Member
Company: Newcastle University
Registered: 2009-05-19
Posts: 1122

Re: Implementing rollbacks & other business logic with Talend Open Studio

You would need to design your job as a transaction so you commit every time you have a successful child insert.

Offline

#7 2011-09-25 07:23:47

alevy
Member
Registered: 2009-11-20
Posts: 1478

Re: Implementing rollbacks & other business logic with Talend Open Studio

In the connection component, uncheck "auto commit".  Nothing will be committed until you use a commit component and everything since last commit will be rolled back if you use a rollback component.  If you have multiple parents you need to commit each parent and child(ren) group separately.  Have a go and put up some screenprints and sample data if you have a problem.

Offline

#8 2011-09-27 18:42:42

Nathan G.
Guest

Re: Implementing rollbacks & other business logic with Talend Open Studio

Thank you for the suggestions!  I will give those a shot to see if they will work for me!

#9 2011-09-28 03:59:30

Dale
Guest

Re: Implementing rollbacks & other business logic with Talend Open Studio

If the job were exported as a war file, deployed to a server, and called via java classes, would turning off and on auto commit on the java connection also be a viable solution? Any idea on performance hit when committing such a small batch. Throughput is excellent for large
batches.

#10 2011-12-06 21:36:10

claude_ja
New member
Registered: 2011-11-17
Posts: 2

Re: Implementing rollbacks & other business logic with Talend Open Studio

janhess wrote:

You would need to design your job as a transaction so you commit every time you have a successful child insert.

Hi,
I'm trying to design my job in that way, with no success yet.

I don't understand how to trigger a commit (or a rollback) each time a row has been processed. With my actual job design (see attached picture) a rollback will avoid all inserts since the beginning of the job. How to modify my job in order the rollback avoid only the current insert ? Do I need to use iterate connections ? If yes, how to use them to get this kind of behavior ?

Thanks


Uploaded Images

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Implementing rollbacks & other business logic with Talend Open Studio

Board footer

Powered by FluxBB