We are designing data ingestion from source database into a data lake. We are transferring raw data as is, and want to verify that everything has been transferred correctly. I am thinking of using DQ for this purpose. Has anyone programmatically compared the DQ analysis results for two different databases? In this case source can be Oracle or SQL server and the target is Hive.
If we understand your job requirement very well, do you want to compare tables from two different sources to catch the changed data? Have you already checked talend data integration product with CDC feature?
Feel free to correct me if something is missing from our side.
What we can do is to make sure that Talend will be your best choice!