• Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Accessing Values from SetGlobalvar and tBufferOutput and Input

#1 2011-09-25 11:29:49

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Accessing Values from SetGlobalvar and tBufferOutput and Input

Hello,

i have the following problem, which is discussed here several times, but after reading these threads i cannot find a solution.

Description:
I want to share values between jobs.
As i read here, it is not possible to share these values with SetGlobalVar.
Some solutions here works with the tBuffer component.
So I put the values from Global into tBufferOutput.
If i understand it correctly, i can read this buffer in other jobs with tBufferInput, too!.
But i have no luck doing this.

Any idea?

Another solution is to put the values in context.
But how do i copy a globalvar into a context var?
Should i do this with tJava component?
Could i access these values all over my jobs?

Here are my screenshots of the main and childjob.
Hope it helps.

Best wishes
Jens


Uploaded Images

Last edited by berufsschule (2011-09-25 11:32:01)

Offline

#2 2011-09-26 04:11:04

alevy
Member
Registered: 2009-11-20
Posts: 1488

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

You cannot share globalMap between jobs.  Context variables can easily be passed to child jobs by checking "Transmit whole context" in tRunJob.  But passing values back to a parent job is harder.

Problems with your screenshots are:
1. The row coming out of a tSetGlobalVar is just a duplicate of the row going in to it (if any), not any globalMap variables you might have set in it.  You need to use tFixedFlowInput to create a flow of your globalMap variables, which you can then pass to tBufferOutput.
2. To retrieve the values from the buffer in the parent, you should have a row directly from tRunJob.  (Don't forget to set the schema.)  You then need either tSetGlobalVar or tContextLoad or tJavaRow (as appropriate) to convert that flow back to globalMap or context variables in the parent.

Converting globalMap into context variables (or vice versa) has to be done manually - tJava is fine.

There are indeed many posts about passing data between parent and child jobs and back so I suggest you review them again.

Offline

#3 2011-09-26 18:06:09

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

alevy wrote:

You cannot share globalMap between jobs.  Context variables can easily be passed to child jobs by checking "Transmit whole context" in tRunJob.  But passing values back to a parent job is harder.

Problems with your screenshots are:
1. The row coming out of a tSetGlobalVar is just a duplicate of the row going in to it (if any), not any globalMap variables you might have set in it.  You need to use tFixedFlowInput to create a flow of your globalMap variables, which you can then pass to tBufferOutput.
2. To retrieve the values from the buffer in the parent, you should have a row directly from tRunJob.  (Don't forget to set the schema.)  You then need either tSetGlobalVar or tContextLoad or tJavaRow (as appropriate) to convert that flow back to globalMap or context variables in the parent.

Converting globalMap into context variables (or vice versa) has to be done manually - tJava is fine.

Hello,
i tried the following with tJava, but it doesn´t work. Perhaps i should use tJavaRow?
Want to save an input row to context variables and then dump it with tContextDump.
Here is the Code inside tJava. The Input Rowname is hoechstes_datetime.
I think, that the name of the rows are not accessible in java that way.


context.put("dirty_datetime_last",hoechstes_datetime.dirty_datetime_last);
context.put("dirty_datetime_new",hoechstes_datetime.dirty_datetime_new);
context.put("aggregation_last",hoechstes_datetime.aggregation_last);
context.put("integration_last",hoechstes_datetime.integration_last);
context.put("date_from",hoechstes_datetime.date_from);
context.put("date_until",hoechstes_datetime.date_until);
context.put("kw_von",hoechstes_datetime.kw_von);
context.put("kw_bis",hoechstes_datetime.kw_bis);
context.put("tag_von",hoechstes_datetime.tag_von);
context.put("tag_bis",hoechstes_datetime.tag_bis);
context.put("monat_von",hoechstes_datetime.monat_von);
context.put("monat_bis",hoechstes_datetime.monat_bis);
context.put("jahr_von",hoechstes_datetime.jahr_von);
context.put("jahr_bis",hoechstes_datetime.jahr_bis);
context.put("stunde_von",hoechstes_datetime.stunde_von);
context.put("stunde_bis",hoechstes_datetime.stunde_bis);
context.put("anzahl",hoechstes_datetime.anzahl);

Also i tried the suggestion with tJavaRow from shong http://talendforge.org/forum/viewtopic.php?id=17823
context.dirty_datetime_last = input_row.dirty_datetime_last;
context.dirty_datetime_new = input_row.dirty_datetime_new;
context.aggregation_last = input_row.aggregation_last;
context.integration_last = input_row.integration_last;
context.date_from = input_row.date_from;
context.date_until = input_row.date_until;
context.kw_von = input_row.kw_von;
context.kw_bis = input_row.kw_bis;
context.tag_von = input_row.tag_von;
context.tag_bis = input_row.tag_bis;
context.monat_von = input_row.monat_von;
context.monat_bis = input_row.monat_bis;
context.jahr_von = input_row.jahr_von;
context.jahr_bis = input_row.jahr_bis;
context.stunde_von = input_row.stunde_von;
context.stunde_bis = input_row.stunde_bis;
context.anzahl = input_row.anzahl;

But this throws an error, too.
Exception in thread "main" java.lang.Error: java.lang.Error: java.lang.Error: java.lang.Error: Unaufgelöste Kompilierungsprobleme:
    dirty_datetime_last kann nicht aufgelöst werden oder ist kein Feld
    dirty_datetime_new kann nicht aufgelöst werden oder ist kein Feld
    aggregation_last kann nicht aufgelöst werden oder ist kein Feld
    integration_last kann nicht aufgelöst werden oder ist kein Feld
    date_from kann nicht aufgelöst werden oder ist kein Feld
    date_until kann nicht aufgelöst werden oder ist kein Feld
    kw_von kann nicht aufgelöst werden oder ist kein Feld
    kw_bis kann nicht aufgelöst werden oder ist kein Feld
    tag_von kann nicht aufgelöst werden oder ist kein Feld
    tag_bis kann nicht aufgelöst werden oder ist kein Feld
    monat_von kann nicht aufgelöst werden oder ist kein Feld
    monat_bis kann nicht aufgelöst werden oder ist kein Feld
    jahr_von kann nicht aufgelöst werden oder ist kein Feld
    jahr_bis kann nicht aufgelöst werden oder ist kein Feld
    stunde_von kann nicht aufgelöst werden oder ist kein Feld

Any suggestions?

Jens


Uploaded Images

Last edited by berufsschule (2011-09-26 19:25:26)

Offline

#4 2011-09-27 05:51:52

shong
Talend team
Registered: 2007-08-29
Posts: 10310
Website

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Hi
It is a compilation error, have you defined the schema of tJavaRow?

you should use tJavaRow replace tJava if you want to access the columns. I suggest you to use 'onsubjobook' in the job, it makes the job more readable, for example:
tMysqlConnection
   |
onsubjobok
   |
tFileInputDelimited--main--tContextLoad
   |
onsubjobok
   |
next processing

Best regards
Shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#5 2011-09-27 10:39:20

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

shong wrote:

Hi
It is a compilation error, have you defined the schema of tJavaRow?

you should use tJavaRow replace tJava if you want to access the columns.

Best regards
Shong

Hello
thanks for the reply.
yes i have defined a schema for the input of tjavarow, but not for output, because i don´t have any rows to output.
tjavarow should only save the colums from the inputrow to the correspondending context.variables.

here is the screenshot for the schema.

best wishes
jens


Uploaded Images

Offline

#6 2011-09-27 10:57:55

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

shong wrote:

Hi
It is a compilation error,
Best regards
Shong

Hi,
it´s a bit strange, because the context.variable seems to be not known. Hmmh.
Something wrong with tContextLoad?
The log says, that the variables are loaded and known.

10:55:20|ddFHcP|ddFHcP|ddFHcP|4064|TOS_WETTERDATEN|Copy_of_Uni_Pruefung|_jBqKUOa6EeCETqKxVR9mCQ|1.10|Default|tContextLoad_2|begin||
Warning: Parameter "dirty_datetime_last" is a new parameter of tContextLoad_2
Warning: Parameter "dirty_datetime_new" is a new parameter of tContextLoad_2
Warning: Parameter "aggregation_last" is a new parameter of tContextLoad_2
Warning: Parameter "integration_last" is a new parameter of tContextLoad_2
Warning: Parameter "date_from" is a new parameter of tContextLoad_2
Warning: Parameter "date_until" is a new parameter of tContextLoad_2
Warning: Parameter "kw_von" is a new parameter of tContextLoad_2
Warning: Parameter "kw_bis" is a new parameter of tContextLoad_2
Warning: Parameter "tag_von" is a new parameter of tContextLoad_2
Warning: Parameter "tag_bis" is a new parameter of tContextLoad_2
Warning: Parameter "monat_von" is a new parameter of tContextLoad_2
Warning: Parameter "monat_bis" is a new parameter of tContextLoad_2
Warning: Parameter "jahr_von" is a new parameter of tContextLoad_2
Warning: Parameter "jahr_bis" is a new parameter of tContextLoad_2
Warning: Parameter "stunde_von" is a new parameter of tContextLoad_2
Warning: Parameter "stunde_bis" is a new parameter of tContextLoad_2
Warning: Parameter "anzahl" is a new parameter of tContextLoad_2

Any suggestions?
Jens

Offline

#7 2011-09-27 11:18:41

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

berufsschule wrote:

shong wrote:

Hi
It is a compilation error, have you defined the schema of tJavaRow?

you should use tJavaRow replace tJava if you want to access the columns.

Best regards
Shong

jens

Perhaps i find out a solution:
I change the code in the following way
context.put("dirty_datetime_last",input_row.dirty_datetime_last);
context.put("dirty_datetime_new",input_row.dirty_datetime_new);
context.put("aggregation_last",input_row.aggregation_last);
context.put("integration_last",input_row.integration_last);
context.put("date_from",input_row.date_from);
context.put("date_until",input_row.date_until);
context.put("kw_von",input_row.kw_von);
context.put("kw_bis",input_row.kw_bis);
context.put("tag_von",input_row.tag_von);
context.put("tag_bis",input_row.tag_bis);
context.put("monat_von",input_row.monat_von);
context.put("monat_bis",input_row.monat_bis);
context.put("jahr_von",input_row.jahr_von);
context.put("jahr_bis",input_row.jahr_bis);
context.put("stunde_von",input_row.stunde_von);
context.put("stunde_bis",input_row.stunde_bis);
context.put("anzahl",input_row.anzahl);

Now this component seems to work, but the following component tContextDump throws an error:
Exception in component tContextDump_1
java.lang.NullPointerException
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tContextDump_1Process(Copy_of_Uni_Pruefung.java:848)
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tMysqlInput_1Process(Copy_of_Uni_Pruefung.java:5329)
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tWaitForSqlData_1Process(Copy_of_Uni_Pruefung.java:1992)
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.tMysqlConnection_1Process(Copy_of_Uni_Pruefung.java:1521)
2011-09-27 11:15:33|KsGnBY|KsGnBY|KsGnBY|4916|TOS_WETTERDATEN|Copy_of_Uni_Pruefung|_jBqKUOa6EeCETqKxVR9mCQ|1.10|Default|tContextDump_1|begin||
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.runJobInTOS(Copy_of_Uni_Pruefung.java:9464)
    at tos_wetterdaten.copy_of_uni_pruefung_1_10.Copy_of_Uni_Pruefung.main(Copy_of_Uni_Pruefung.java:9331)

What could i do wrong here, because the component has no special details...

??

Last edited by berufsschule (2011-09-27 11:20:15)

Offline

#8 2011-09-28 11:22:05

shong
Talend team
Registered: 2007-08-29
Posts: 10310
Website

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Hi berufsschule
I can't reproduce the problem, not sure why it throws NPE here. Can you upload a screenshot of job?

Best regards
Shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#9 2011-09-28 11:47:59

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

shong wrote:

Hi berufsschule
I can't reproduce the problem, not sure why it throws NPE here. Can you upload a screenshot of job?

Best regards
Shong

hello,
here is a screenshot.
hope it helps.
i posted the code from tjavarow some mails above.

Best wishes
Jens


Uploaded Images

Last edited by berufsschule (2011-09-28 11:49:22)

Offline

#10 2011-09-28 17:08:27

shong
Talend team
Registered: 2007-08-29
Posts: 10310
Website

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Hi
Have you redesign the job as I suggested yesterday?

you should use tJavaRow replace tJava if you want to access the columns. I suggest you to use 'onsubjobook' in the job, it makes the job more readable, for example:
tMysqlConnection
   |
onsubjobok
   |
tFileInputDelimited--main--tContextLoad
   |
onsubjobok
   |
next processing

Best regards
Shong

Best regards
Shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#11 2011-09-28 17:31:43

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

shong wrote:

Hi
Have you redesign the job as I suggested yesterday?
Best regards
Shong

hello,

yes i did.
screenshot below.
i changed the flow into an subjob and changed tjava into tjavarow.

What could i do further?

Any suggestions?

Best wishes
Jens


Uploaded Images

Offline

#12 2011-09-30 09:17:31

shong
Talend team
Registered: 2007-08-29
Posts: 10310
Website

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

Hi
Change the job to:
tMysqlConnection
   |
onsubjobok
   |
tFileInputDeliited--main--tContextLoad
   |
onsubjobok
   |
tWaitForDate--itertate-->....

Best regards
Shong


Email:shong@talend.com
Choose Talend, Enjoy Talend!
New & Event: Talend Help Center
Talend-->the leader of open source data management and application integration solutions!

Offline

#13 2011-09-30 12:06:37

berufsschule
Member
Registered: 2011-07-13
Posts: 26

Re: Accessing Values from SetGlobalvar and tBufferOutput and Input

shong wrote:

Hi
Change the job to:
tMysqlConnection
   |
onsubjobok
   |
tFileInputDeliited--main--tContextLoad
   |
onsubjobok
   |
tWaitForDate--itertate-->....

Best regards
Shong

Hi,

thanks for the tipp, but same result :-((
here is the code from tjavarow (i changed nothing from the above code)
context.put("dirty_datetime_last",input_row.dirty_datetime_last);
context.put("dirty_datetime_new",input_row.dirty_datetime_new);
context.put("aggregation_last",input_row.aggregation_last);
context.put("integration_last",input_row.integration_last);
context.put("date_from",input_row.date_from);
context.put("date_until",input_row.date_until);
context.put("kw_von",input_row.kw_von);
context.put("kw_bis",input_row.kw_bis);
context.put("tag_von",input_row.tag_von);
context.put("tag_bis",input_row.tag_bis);
context.put("monat_von",input_row.monat_von);
context.put("monat_bis",input_row.monat_bis);
context.put("jahr_von",input_row.jahr_von);
context.put("jahr_bis",input_row.jahr_bis);
context.put("stunde_von",input_row.stunde_von);
context.put("stunde_bis",input_row.stunde_bis);
context.put("anzahl",input_row.anzahl);
context.synchronizeContext();

and here is the screenshot.

Any clues?

Best wishes
Jens


Uploaded Images

Last edited by berufsschule (2011-09-30 12:08:26)

Offline

  • Index
  •  » Talend Open Studio for Data Integration » Usage, Operation
  •  » Accessing Values from SetGlobalvar and tBufferOutput and Input

Board footer

Powered by FluxBB