You are not logged in.
Good Day All
I am trying to create a routine that stores my data in a hashmap as well as perform specialized functions on the data. This is required because my data is quite complex and needs to be stored in memory until it can be matched with an incoming records at a later stage.
I use a jJavaRow component witch adds the components to the hashmap though the routine. I then connect to a tJava component that will validate the data and then merge the data if necessary again though a routine function call. Finally I pass the data to another tJavaRow where the original data is merged with the original stream as appropriate.
The problem is that the java components are not being called in the order that they have been linked therefore. when it is run the validation and merging is attempted first (null pointer as the map has not been created), then the records are added to the hashmap and finally it will try and merge with the orgional stream. I suspect I am trying to get talend to function in a way that it was not designed. is there another approach to this problem or is this problem best suited for another solution?
Don't ever use tJavas in a flow.
Every Talend Component generates a BEGIN, MAIN and END block in java. At the beginning of a subjob all BEGIN blocks are generated, then the mainflow starts and uses the main blocks and after the mainflow has finished the END blocks are generated.
tJava's MAIN segment is empty! All code written in the tJava is performed in the BEGIN block.
tFileinputDelimited -(main)-> tJava -(main)-> tJavaFlex -(main)-> tLogRow
Generates a method:
This is quite confusing, tJava should only be used as a standalone subjob or at the beginning of a subjob.
As a solution, just use tJavaRow instead of tJava. If you need to initialize and/or clean up things, use tJavaFlex.
try to NOT use tjava* component if it could be process thanks Talend Component
you have hashmap , merge and so on ...
Think about "Maintenance" of your process and no need of expert Java ...
So use "hard coding" with "parcimony
Thanks for the answer, the only problem then is how would I execute the merge method after all records have been read with the tJavaRow? if I execute the validate and merge methods in a tJavaRow they would be executed multiple times. I suppose I would need to modify this to merge as the records come in, would this not slow down the process instead of being executed at the end?
the problem is that incoming data is received in multiple legs though a single source, each leg contains a fragment that will need to be merged into a single record. The legs are not received in order or within the same time span. Therefore un-merged legs would need to be kept in memory until they can be merged with fresh incoming data.
As I see it I would use a tJavaRow to do the following
1. add the legs into a memory Hashmap via a routine.
2. validate the record to ensure its correctness.
3. if it is the last needed leg, merge the new leg with data in the hash map.
from here I would need to pass the stream to another component that would write the merged record to a database and then aggregate this data. Finally update the hash to remove the records that have been merged.
I dont see how the components on offer could solve it "out of the box"
Your thoughts on the matter would be greatly appreciated
Last edited by BrianJohnson (2012-07-23 10:45:12)