You are not logged in.
First of all I'm using version 5 of Talend opend studio
I have to be able to somehow validate the format of a file. If all records are correct file I then load into the registry database and if not do nothing. Most of the files I have to deal with more than 60,000 records.
I've seen that both the component and tschemaCompliancecheck fileinputdelimited to check the structure of the records will result in two outputs. Erroneous and correct row. From here no how.
Step one example I have an echo in the end only file where I have a record to give you an idea of what I've seen so far.
I am interested to do something general indiqueis when I have one or more records in the file.
Last edited by oscarmsanz (2011-12-16 11:22:23)
Hi Oscar Manso
You may create a job as the following images.
You said if all records are correct, you will load the file into database.
So I declare a context variable here and use a component tJava to set the value of this variable.
Then I use a run-if trigger to judge whether you load this file into database.
Hope this will help you.
Last edited by pedro (2011-12-16 13:03:31)
The problem I have that are very large files that I have to load and use thashinput thashoutput and I think it is the best solution,
The final problem is what is saved in memory and from experience with other tools ELTL can get to not having enough memory on the machine when it comes to keeping records,
Therefore need a better alternative if available.
Your solution and thought.
Oscar Manso Sanz
You are right. I hava corrected it and make a better workaround.
This time i just load the input file twice.
Hope someone can offer perfect workaround.
Anyone who knows a better solution than say so, because by using file with a large volume of data does not seem correpto unsar thashinput, because it can spilling the precious Blood thashouput computer memory
I know at other ETL tools like Oracle Wharehouse Builder I know for having cast a course, when you load a file only indicates the following.
File loading and file where the records are not loaded.
When you load the file is loaded as it is and if there is a mistake and want to download these data are usually delete these records database and a new file reloads.
I work for the client has asked me not to do this. Otherwise if the file has an erroneous structura not load it and while it loads well. Then if you get an error when doing insert into database will abort the entire file but that's another story.
Last edited by oscarmsanz (2011-12-16 13:23:08)