• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Bufferizing data to be used in a tJava

#1 2008-05-13 17:06:37

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Bufferizing data to be used in a tJava

Hi everyone,

I'm working on a job where I want to bufferize some data extracted from a database before processing them in a tJava component:

tMySQLInput -----> bufferizing in an array (t[i][j], i and j are for indexes)
 
        |
        |
        |
        |

using the t[i][j] variable in a tJava

Context variable cannot be of an array type (or am I wrong?) and I didn't find any component that can be used that way.

Thanks for your help.

Regards.

Last edited by mlamrani (2008-05-13 17:08:52)

Offline

#2 2008-05-14 09:24:37

shong
Talend team
Registered: 2007-08-29
Posts: 10357
Website

Re: Bufferizing data to be used in a tJava

Hello

Context variable cannot be of an array type (or am I wrong?) and I didn't find any component that can be used that way.

Currently, there is not such a component can do that.
Why do you want to buffer some data bebore processing them in tJava? Can you show more detailed information and what are your expected result?

You can query and output the data to a Excel file, then extract data from specific Excel cells.
please read this [Forum, topic 970] Accessing specific Excel cells.
Hope it help you...

Best regards

         shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#3 2008-05-14 09:45:09

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

Hi Shong,

My problem is that I have data in my database that I need to sort. The data are circuits between two equipements (A and G for example), each one of these circuits is composed of mini-circuits between other equipements. My job extracts for each main circuits all the mini-circuits related to it, and I need to sort these mini-circuits:

Main circuit: A-----------------------------------------------------G
Mini-circuits composing it:A-----B, B-----C, C------D, D------E, E------F, F-----G

Sorting: 1st: A---B
            2nd: B---C
            3rd: C---D
            ...etc...

For that, I need each time I extract the mini-circuits, to store them In an array variable and process this array in a tJava to do the sorting. The most important thing is that I don't want to use temporary files.

I hope I was clear enough :p and thanks for your support.

Offline

#4 2008-05-14 16:46:38

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

From another point of view, you can see my problem as the sorting of a chained list that is extracted as a flow from a database.

Offline

#5 2008-05-14 17:47:11

mhirt
Talend team
Registered: 2006-09-19
Posts: 1658

Re: Bufferizing data to be used in a tJava

Window -> Preferences -> Talend -> Designer -> Check "Show hidden component" and then restart TOS.
You will wee some new component in Palette.

You can use tArray in Misc family.

Regards,

Online

#6 2008-05-14 18:00:06

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

Actually, I already tried this solution but I have an exception when I try to use in tJava the name I give to the array in the "destination" field. Is there an example of using tArray as an array for tJava ?

Offline

#7 2008-05-14 19:18:43

mhirt
Talend team
Registered: 2006-09-19
Posts: 1658

Re: Bufferizing data to be used in a tJava

Take example on tArrayIn.

It will be something like

Code:

java.util.List<Structarray_tArray_1> list_tArrayIn_1 = (java.util.List<Structarray_tArray_1>) 
                          globalMap.get("array_tArray_1");

Regards,

Online

#8 2008-05-15 09:21:56

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

Hi, I tried using the syntax like in tArray_in but I got this error:

Exception in thread "Thread-1" java.lang.Error: Unresolved compilation problems:
    Structarray_tArray_1 cannot be resolved to a type

Then I tried this in my tJava:
java.util.ArrayList list_tArrayIn_1 = (java.util.ArrayList) globalMap.get("array_tArray_1");

for (int i = 0; i < list_tArrayIn_1.size(); i++) {
    System.out.println(list_tArrayIn_1.get(i));
}

and I got:
project.job.job$MINICIRCUITStruct@1a9334
project.job.job$MINICIRCUITStruct@b64435
project.job.job$MINICIRCUITStruct@37fb1e
project.job.job$MINICIRCUITStruct@1479feb
project.job.job$MINICIRCUITStruct@15212bc
project.job.job$MINICIRCUITStruct@14c1103
project.job.job$MINICIRCUITStruct@f11404
project.job.job$MINICIRCUITStruct@1592174
project.job.job$MINICIRCUITStruct@97a560
project.job.job$MINICIRCUITStruct@1ea5671
project.job.job$MINICIRCUITStruct@1d15445
project.job.job$MINICIRCUITStruct@1f3aa07
project.job.job$MINICIRCUITStruct@704baa
project.job.job$MINICIRCUITStruct@77a7f9
project.job.job$MINICIRCUITStruct@1b4fad5
project.job.job$MINICIRCUITStruct@bfea1d

I tried casting to string but I got a java exception. Any thougth about getting the values of the arraylist ?


Uploaded Images

Last edited by mlamrani (2008-05-15 10:20:36)

Offline

#9 2008-05-16 00:17:32

mhirt
Talend team
Registered: 2006-09-19
Posts: 1658

Re: Bufferizing data to be used in a tJava

I realize that tArray is probably a bit compliated to use for such a process


Why don't you write a routine with two methods :
A store method
A sort method


For each row {
   call the store method with a tJava Row
}

On Subjob OK

cal sort method

This can be done esaily with only 2 tJava ?

BTW What is the type of sort that you can't make with tSortRow ?

Last edited by mhirt (2008-05-16 00:19:10)

Online

#10 2008-05-16 11:50:50

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

Is it possible to use a method that is defined in a tJava in another tjava ?

For the sorting, I want to match the extremities of the mini-circuits, and the tSort sort rows depending on 1 column ...

Offline

#11 2008-05-18 16:14:49

mhirt
Talend team
Registered: 2006-09-19
Posts: 1658

Re: Bufferizing data to be used in a tJava

Hello,

You can use Code -> Routine to develop / import your own classes an reuse them in all your jobs.
See documentation for more details.

Online

#12 2008-05-19 14:23:03

mlamrani
Member
Registered: 2008-03-07
Posts: 34

Re: Bufferizing data to be used in a tJava

Actually I found a solution for the use of the array stored in the tArray. I put this code in the tJava:

java.util.ArrayList<MINICIRCUITStruct> mc = (java.util.ArrayList) globalMap.get("array_tArray_1");

MINICIRCUITStruct [] a = new MINICIRCUITStruct [topo.size()];
for (int i = 0; i < mc.size(); i++) {
    a[i] = mc.get(i);
}


and then I use the array "a" as I wish.

For the database insertion or update I use this:

java.sql.Connection conn = null;
conn = (java.sql.Connection) globalMap.get("conn_tMysqlConnection_1");
java.sql.PreparedStatement pstmt = null;

pstmt = conn.prepareStatement("<your query here>");
try {
    pstmt.execute();
} catch (Exception e) {
    System.err.print(e.getMessage());
}

if (pstmt != null) {
    pstmt.close();
}


Thanks for your help.


PS: I didn't find the path for code -> routines

Last edited by mlamrani (2008-05-19 14:24:27)

Offline

#13 2008-05-24 13:57:59

mhirt
Talend team
Registered: 2006-09-19
Posts: 1658

Re: Bufferizing data to be used in a tJava

Happy to hear that it's running..
The Code/Routine can be fond very easily


Uploaded Images

Online

#14 2008-09-08 14:48:59

jamesB
Member
Registered: 2008-03-19
Posts: 47

Re: Bufferizing data to be used in a tJava

hi to all !
i'm having the same problem for bufferizing data!!!
i followed this example but without success

my data are extracted from a/some file/s , i elaborate this data, and then i want to use in other elaboration the last result...

i have
A----B     tFileExcelInput---tMap----tArray
B----C     tJava---tLogRow   (because the tArrayIn it doesn't work),


the tJava code

Code:

java.util.ArrayList<out_IStruct> mc = (java.util.ArrayList) globalMap.get("array_tArray_1");
out_IStruct[] a = new out_IStruct[mc.size()];
a = (out_IStruct[])mc.toArray(new out_IStruct[0]);

The result is:
||||||||||||||||||||

it seems that the tLogRow it doesn't filled up


if i make a System.out.println(a[0]);
i have the right result   

newColumn=I, newColumn1=CT, newColumn2=CRIS, newColumn3=NIO, newColumn4=CRSNA, newColumn5=null, newColumn6=null, newColumn7=null, newColumn8=null, newColumn9=90, newColumn10=90


Whatever you are, be a good one..

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Bufferizing data to be used in a tJava

Board footer

Powered by FluxBB