Talend community forum / Strangeness when running TOS deployed process under bash script...

Opus · 2012-07-11 03:36:42

Tags: [database, deployment, development, excel, oracle]

Hello,

I have developed a number of simple Talend jobs using TOS Community Edition on a Windows PC.

These jobs do nothing more serious than loading data into an Oracle table from an Excel spreadsheet.

I then export those jobs as 'autonomous' processes and put them on a Linux server.

I execute the jobs on the Linux server using a simple menu/option bash script. The actual line that executes the Talend job looks like this:

nohup ./JOBS/TRANSACTION_LOAD_0.1/TRANSACTION_LOAD/TRANSACTION_LOAD_run.sh &

The 'strangeness' occurs when I submit the same job twice OR I submit one job and then submit another. In all cases I wait for the first job to finish before submitting the second job.

The 'strangeness' is that the first job executes in a flash, i.e. less than 1 second, HOWEVER, the 2nd job, even if it is the same job with the same data will take upwards of 90 seconds to complete (although it still runs successfully).

Can anyone explain why this would be occurring and if there is any way of correcting it?

Thanks in advance...

pedro · 2012-07-11 03:48:13

Hi

Welcome to Talend Community!

It's werid. Did you try to recur this issue after restarting your OS?
I think mainly this is due to memory issue.

Regards,
Pedro

Opus · 2012-07-11 05:47:23

Hi Pedro,

Thanks for the welcome but actually...I'm not new, I just seem to have collected a couple of different accounts along the way...

Thanks for the suggestion as well but I do not believe that re-starting the server will make a difference in this case.

My initial thought was that the Oracle connection is not being released after the first job and the second job may be waiting for an Oracle connection but I'm not sure about how to test this. In any case I would have thought that the second job (running under a different PID) would not place such a constraint on the job unless Oracle had run out of available connections, which is unlikely in this case...

Can you confirm that a second job would not be affected by the first job leaving a connection hanging (if Oracle is not under any stress with regard to connections....

Cheers

Opus · 2012-07-11 05:49:22

P.S. I should have mentioned that these jobs also use Talends built-in stats, flow and log catching capabilities, which are set at the Project level.

pedro · 2012-07-11 05:59:02

Hi

Code:

the Oracle connection is not being released after the first job and the second job may be waiting for an Oracle connection

That's another possibility.
But before we confirm it, please upload a screen shot about your job.
If there is tOracleCommit or tOracleClose, I think it may not be caused by it.
It seems we'd better fix it steps by steps.

Regards,
Pedro

Opus · 2012-07-11 08:21:19

Hi Pedro,

Screenshot as requested.

When I said the job(s) was(were) simple I was probably being a bit simplistic myself.

To explain...The job(s) basically read a table from an Excel Spreadsheet and:
1) Update insert and new/changed records.
2) Test a temporal (historical) copy of the table to see if any records have been physically deleted from the spreadsheet and delete those records from the table.
3) Use the SCD component to udpate the temporal table with the changes, inserts and deletes.

Cheers

Last edited by Opus (2012-07-11 08:24:15)

pedro · 2012-07-11 08:49:32

Hi

I can't find a screenshot...

Regards,
Pedro

Announcement

Unanswered posts

#1 2012-07-11 03:36:42

Strangeness when running TOS deployed process under bash script...

#2 2012-07-11 03:48:13

Re: Strangeness when running TOS deployed process under bash script...

#3 2012-07-11 05:47:23

Re: Strangeness when running TOS deployed process under bash script...

#4 2012-07-11 05:49:22

Re: Strangeness when running TOS deployed process under bash script...

#5 2012-07-11 05:59:02

Re: Strangeness when running TOS deployed process under bash script...

Code:

#6 2012-07-11 08:21:19

Re: Strangeness when running TOS deployed process under bash script...

#7 2012-07-11 08:49:32

Re: Strangeness when running TOS deployed process under bash script...

Board footer