• Index
  •  » Talend Enterprise Data Integration » General questions
  •  » Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

#1 2008-09-17 19:17:08

pozunmv
Member
Registered: 2008-06-06
Posts: 36

Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

I am working on migrating a job from TIS2.3.3 to TIS2.4.1 and have noticed an issue with the way the job is executin in the new version.  The job in 2.3.3 has a single execution when it gets run, however in 2.4.1 the job gets executed twice.  The entire flow of the job is linked together so I am not unerstanding why its running with this behavior.   The job when its executed in 2.4.1 misses the entire first subjob and starts out on the second sub job... once that has been executed it then goes back and starts the job beginning from the first subjob.  This multiple runnings of the job causes files to be generated in the wrong places because the job it self builds what the path to the file should be.

Ive added some logging to standard out so I could see whats going on.   The logging comes from the tJava_1 component in the attached screenshot of the job.   The last part of the directory gets added on in this component and is populated by a context variable which gets populated in the first subjob.  The results of my adding the stdout logging is below:

Code:

Starting job j_TxAttachment at 13:07 17/09/2008.


Directories:
Archive: c:/SWM_expedia/datafeed/Archive/null/
Export: c:/SWM_expedia/datafeed/Files/null/




Directories:
Archive: c:/SWM_expedia/datafeed/Archive/null/A0000000CW/
Export: c:/SWM_expedia/datafeed/Files/null/A0000000CW/


Job j_TxAttachment ended at 13:08 17/09/2008. [exit code=0]

What between 2.3.3 and 2.4.1 would cause this type of radical change in behavior?  The subjobs are all connected so it should have only one path of execution it can take, but its starting out in the middle and then running from the beginning.   Has anyone else experienced issues like this in their migrations and if so how were you able to resolve them?


Uploaded Images

Last edited by pozunmv (2008-09-17 19:19:11)

Offline

#2 2008-09-17 20:06:41

strpdsnk
Member
Registered: 2008-03-25
Posts: 328

Re: Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

is the problem when you try to run it from TIS or from the Job Conductor on the web?

If its the former:
1. Check your tRunJobs and make sure they are all set to run the correct job, with the correct version and context you want used

If its the latter:
1. check your Tasks in your Job Conductor and make sure the proper Job name is selected to run.
2. if you're using triggers, make sure the triggers don't overlap.  (like having two entries for file trigger that would pull the same file)
3. go look in the Apache logs (mine are located, on the server apache is installed on: C:\Program Files\Apache Software Foundation\Tomcat 5.5\Talend\logs for Talend WUI logs and C:\Program Files\Apache Software Foundation\Tomcat 5.5\logs for Apache specific logs)
  a. Open the files with Notepad (not Wordpad, will pop up an error if you're looking at the most recent file that the WUI is still referencing)

Last edited by strpdsnk (2008-09-17 20:14:15)

Offline

#3 2008-09-17 20:17:00

pozunmv
Member
Registered: 2008-06-06
Posts: 36

Re: Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

I am running it from TIS itself (havent migrated it onto the Job conductor yet).  I have yet to get to the point of running the batch which calls all of these jobs individually, I am simply running this job by itself (we created the jobs so they could either be run in batch mode, or by themselves), so there is no tRunJob component to check.

Ive simply opened up this one job in TIS and used the run tab to tell it to go.  The subjob that is being skipped in the first execution is calling a joblet to either get a batchid or to simply defualt in the batchid that is already set as a context var.  In this case the joblet would get a new batchid because I do not have one set in the contexts. During execution in 2.3.3 this step was always done first and then the rest of the job followed.   Now its skipping this step, running the rest of the job, then going back running that first subjob and then finishing by running the rest of the job.

Offline

#4 2008-09-18 15:06:28

pozunmv
Member
Registered: 2008-06-06
Posts: 36

Re: Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

Well I went ahead and tried to add in the prejob and post job orchestration components to see if they would help me out, I was reluctant because I have 50 some jobs setup this way which I will need to change, and im trying to make the changes very minimal (I already have a couple other basic settings I am changing for other reasons).  But even when adding in the orchestration the job does not run entirely correctly. 

It runs the prejob and postjob, then it runs the actual meat of the job that is not tied to the pre and post job orchestration tasks, then it runs the pre and post job steps again.  With this it at least creates the exports in the correct locations, but its still not running correctly because the pre and post jobs still run twi twice.  In these parts of the job we do some basic setup and then some more detailed logging to a generic logging database we have setup.

From what AI can see there is absolutely no reason why parts of the job are getting executed multiple times, also I thought the the orchestration components were setup to make sure that they only run those steps at the beginning or at the end depending on whether they are tied to pre or post.

See the output from my modified job, I added some logging to stdout once again so I can see in the console output exactly what is happening.

Code:

Starting job j_TxAttachment at 08:11 18/09/2008.
[statistics] connecting to socket on port 4095
[statistics] connected
!!!!!!!!!!RAN PreJob!!!!!!!!!!!
!!!!!!!!Ran Post Job!!!!!!!!!


Directories:
Archive: c:/SWM_expedia/datafeed/Archive/A0000000D0/
Export: c:/SWM_expedia/datafeed/Files/A0000000D0/


!!!!!!!!!!RAN PreJob!!!!!!!!!!!
!!!!!!!!Ran Post Job!!!!!!!!!
[statistics] disconnected
Job j_TxAttachment ended at 08:12 18/09/2008. [exit code=0]

Also I am having issue getting to the image uploads in the frum using Firefox 3.0.1, I have to change to the IE rendering engine to get the image uploads to work now.  I never used to have issues with FF and this forum, has something recently changed which would break imageupload when using FF?


Uploaded Images

Last edited by pozunmv (2008-09-18 15:14:05)

Offline

#5 2008-09-18 15:46:11

pozunmv
Member
Registered: 2008-06-06
Posts: 36

Re: Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

If I uncheck the Multithreaded execution box in the job settings I also get a different exectuion path

Code:

Starting job j_TxAttachment at 09:39 18/09/2008.
[statistics] connecting to socket on port 3965
[statistics] connected
!!!!!!!!!!RAN PreJob!!!!!!!!!!!


Directories:
Archive: c:/SWM_expedia/datafeed/Archive/A0000000D2/
Export: c:/SWM_expedia/datafeed/Files/A0000000D2/


!!!!!!!!Ran Post Job!!!!!!!!!
!!!!!!!!!!RAN PreJob!!!!!!!!!!!
!!!!!!!!Ran Post Job!!!!!!!!!
[statistics] disconnected
Job j_TxAttachment ended at 09:40 18/09/2008. [exit code=0]

Offline

  • Index
  •  » Talend Enterprise Data Integration » General questions
  •  » Migrating jobs from TIS2.3.3 to TIS2.4.1: different execution paths

Board footer

Powered by FluxBB