You are not logged in.
I am fairly new to Talend and I need some help with some implementation questions. I need to iterate through any files in a directory and process them. For each file, I need to make a decision row by row. In each row I process, there will be a card number column and value. For each row, I need to query our mysql database and see if that card number has an associated user id. If it does, I need to grab that associated user id and map it to an output file with the data from the original file. If it doesn't, I need to store all of the data from that row being processed into some sort of a batch file for later processing.
If you have any suggestions on what components would be best to use, that would be great. There seems to be quite a range of resources but not enough documentation to describe the situations they are best used. I was planning on using a tfilelist to iterate through each of the files in the proper directory. Then I plan to process each file using a tFileInputDelimited. After this, I run into difficulties with the implementation ideas. I think I should use tMySQLInput for the query, but I'd like to be able to run a very specific query like "SELECT user_id FROM user_table WHERE cardnumber = (variable with that row's card number)", but I can't figure out how to handle passing in a variable like that to the query and I also don't understand what happens after the query. If an entry is found, what happens and how do I handle passing that entry to the mapping component? If the entry isn't found, what happens? How can I dictate that row to be saved for later processing in a different batch file.
Anyway, I know this is a long post, but any help is appreciated. Talend seems like a great tool but even after paging through the documentation I feel like I don't have a grasp on it and clearly I don't.
It sounds like you made a good start with the filelist, fileinputdelimeted and the mysqlinput.
What you need is a tMap combining the fileinputdelimted and the tmysqlinput (use the latter as the lookup). When using an inner join on the mysql lookup you can catch inner joins for you main output and catch the inner join rejects and map them to a separate flow.
Key here is the tMap: google for some samples with an inner join and soon you'll be fine.
Thanks for the help.
When I run the sqlinput query, should I just grab the all of the relevant information in the table or can I pass in a variable to the query to narrow it down to just the single entry I care about?