You are not logged in.
Announcement
Unanswered posts
|
Pages: 1
HI,
We are running a system with one web server and three local (geographically distant) database servers. (eg. web server=> WEB; local servers=> ABC, DEF & XYZ). All the database structures are the same and are using MySQL 5.
We need to replicate the information on the local servers on the web server and vice-versa. The web server will hold all the data contained on the local servers. The local servers will be holding only the data which is relevant to it. (eg. Local server ABC will be holding only records for ABC).
We are using Talend Open Studio 3.0.0. I want to know:
1. Can I replicate the entire db with one job? Right now I am creating jobs for the replication of individual tables. and plan to create them for one local database server and then export them to the other database servers.
2. How are deleted records handled? If a record is deleted on the web server, then it must be deleted in the corresponding local server and vice-versa. Currently this is not happening. I was unsuccessful at even creating a separate job. Currently if records are deleted on the ABC they are not deleted from the WEB and then during replication form WEB to ABC these records are restored.
Please help as this is very urgent.
Thank you.
Last edited by mis (2009-05-03 09:50:40)
Offline
When you delete from the ABC server, do you have some kind of trigger that saves the "deleted" data to a temporary table or something like it ?
Maybe you need a new table to store deleted data information and come up with some kind of logic to maintain it.
Offline
Thank you aoliveira,
You see the data needs to be transferred between different installations of an enterprise software which contains about 216 tables.
I had originally thought of writing triggers for the delete operations on each of the tables beacuse the inserts and updates can be handled by Talend. The problem is, however, it's a multi-server environment and i am unable to connect to multiple databases within the trigger.
Even if you could help me connect 2 databases, on different servers, it would be fine.
Also, regarding storing of deleted data as mentioned by "aoliveira":
I can have a table "mno" with three columns
1. table_name
2. field_name
3. record_id
I can also write a trigger on each table such that when data is deleted from a table, the table name, field name and record id are inserted into the above table "mno".
My question is: "Can i design/write a job which will retrieve data from mno and then
delete from table_name where field_name = record_id
I'd really appreciate the help.
Thank you.
Last edited by mis (2009-05-05 06:54:21)
Offline
mis wrote:
Thank you aoliveira,
You see the data needs to be transferred between different installations of an enterprise software which contains about 216 tables.
I had originally thought of writing triggers for the delete operations on each of the tables beacuse the inserts and updates can be handled by Talend. The problem is, however, it's a multi-server environment and i am unable to connect to multiple databases within the trigger.
Even if you could help me connect 2 databases, on different servers, it would be fine.
Also, regarding storing of deleted data as mentioned by "aoliveira":
I can have a table "mno" with three columns
1. table_name
2. field_name
3. record_id
I can also write a trigger on each table such that when data is deleted from a table, the table name, field name and record id are inserted into the above table "mno".
My question is: "Can i design/write a job which will retrieve data from mno and thendelete from table_name where field_name = record_id
I'd really appreciate the help.
Thank you.
Yes you can.
Check the tMysqlOutput component. There's a property called Action On Data. This property gives you the option to DELETE things from the database. I think this should exist for almost all of the DBComponents, if not all.
I'm just not sure how you would be able to read from your proposed mno table and figure out which table to delete from, and update the tDBOutputComponent on the go....but maybe a tJava can help you there.
I'm looking forward to see the whole picture of your final solution.
Thanks,
Alvaro Oliveira.
Offline
aoliveira wrote:
Yes you can.
Check the tMysqlOutput component. There's a property called Action On Data. This property gives you the option to DELETE things from the database. I think this should exist for almost all of the DBComponents, if not all.
I'm just not sure how you would be able to read from your proposed mno table and figure out which table to delete from, and update the tDBOutputComponent on the go....but maybe a tJava can help you there.
I'm looking forward to see the whole picture of your final solution.
Thanks,
Alvaro Oliveira.
Thank you Alvaro,
I had seen and tried the DELETE option tMysqlOutput component. but it doesnt delete as there is no log of deleted records.
I will be writing a script that will read the mno table and form queries to delete the records. Any help with the script would be appreciated.
Once again. Thank you.
Offline
Pages: 1