You are not logged in.

Unanswered posts

#1 2010-07-03 06:23:14

captainlaw
Member
81 posts

Out of Memory Error

Tags: [error, mssql]

Hello,
I'm importing data from one SQL DB to another SQL DB using tMSSqlInput.  However, I am receiving the following error.
Error Message -
"Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space..."

Is there a way around this error? 
Is there a way to specify batch rows to insert at a time?  Would that be a work around?

Any help is appreciated!

Offline

#2 2010-07-03 11:57:46

lraulier
Member
13 posts

Re: Out of Memory Error

hi captain smile

did you try to change your jvm arguments.

Try a search into the forum with java heap errors ... a lot of messages around this kind of 'problems'  !
perhaps your solution is there !

hope it help

regards
laurent

Last edited by lraulier (2010-07-03 11:58:30)


individuals are more important than any process (Grady Booch)

Offline

#3 2010-07-03 12:26:29

lijolawrance
Member
375 posts

Re: Out of Memory Error

In the generated batch file after export scripts

increase the jvm size -Xms256M -Xmx1024M


Regards
Lijo Lawrance

Offline

#4 2010-07-04 22:13:07

camsellem
Talend Team


Re: Out of Memory Error

Can you post a screenshot of your job plz ? I dont think the exception is coming from the MSSQLinput component since the jtds drivers used by our component has already a value (look at this -> http://www.talendforge.org/bugs/view.php?id=8095&nbn=9)

Are you using a tMap ? tAggregateRows ? tSortRows ?

Offline

#5 2010-07-06 22:15:08

captainlaw
Member
81 posts

Re: Out of Memory Error

Hello,
Looks like altering jvm is in the Talend batch file.  How would I do in with Talend Design Studio?  I am simply running the job in Design Studio, getting the memory error. 

I've attached the components I am using, and it is erroring out between tMSSqlInput and tHashOutput.  tHashOuput, I have "Data write model" to "Persistent to file", and the hash file is located in D drive, which I have set "Max memory heap size(MB)" to 6000.

Any other thoughts based on the components I am using?
Let me know if you have problem viewing my attached image.

Offline

#6 2010-07-06 23:38:00

captainlaw
Member
81 posts

Re: Out of Memory Error

BTW, I found jvm settings under Preference, Run/Debug.  I reset -Xmx256M to -Xmx1280M.  However, I still received out of memory error. 
"Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2882)"

Any help is appreciated.

Offline

#7 2010-07-07 09:15:22

bboutros
Talend Team


Re: Out of Memory Error

Hi,

Can you post a full snapshot of your job.
You seem to be using Hash components.
This components are advanced ones and I think that you don't need them.

benjamin

Offline

#8 2010-07-07 09:42:43

reiko
Member
39 posts

Re: Out of Memory Error

Hi,
I had a similar issue when reading data from mysql tables. Only workaround for me was to use a loop and a context variable and read the total x million rows from tmysqlinput in buckets of 1 million rows (as defined in the where clause of the input component: ID between x and y).
maybe this'll help!

cheers, reiko

Offline

#9 2010-07-07 10:23:16

bboutros
Talend Team


Re: Out of Memory Error

Hi,

For Mysql, you should go in the advanced properties of the tMysqlInput and select Enable Stream, otherwise everything will be loaded in memory.

regards,

Benjamin

Offline

#10 2010-11-12 10:20:21

PerisRaja
Guest

Re: Out of Memory Error

I have checked TMSSqlinput advanced properties but I did not find any stream related link.

#11 2011-11-24 18:16:01

geuder
Member
112 posts

Re: Out of Memory Error

Thanks, Ben!

"advanced properties of the tMysqlInput and select Enable Stream"

This(!) helped me.

/Geuder

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy