Talend community forum / FileInputXML job runs out of memory

ojouny · 2009-05-07 11:34:48

Tags: [database, error, java, java error, out-of-memory, PostgreSQL, xml]

Hello

I simply want to read an file of 500MB to inject it in a postgreSQL table but my job fails with an out-of-memory exception.

I try to change the generation mode of the FileInputXML with DOM, Xerces and SAX but it makes no differences.

Can you help me ?

Regards,

ojouny · 2009-05-07 11:44:02

ojouny wrote:
Hello

I simply want to read an file of 500MB to inject it in a postgreSQL table but my job fails with an out-of-memory exception.

I try to change the generation mode of the FileInputXML with DOM, Xerces and SAX but it makes no differences.

Can you help me ?

Regards,

I use TOS V3.1.0 r24382

aoliveira · 2009-05-07 15:34:33

How's the memory usage of your Java VM ?

Java application can have a setting on how much memory they can consume. I *think* that the standard is 256MB, so maybe you need to update that Java argument so that you can use more RAM.

For Talend, the arguments appear to be in this file TalendOpenStudio-win32-x86.ini

There are several parameters included in this file already, the one I supposed you have to tweak is the '-Xmx500m' one.

The 500m stands for the amount of memory your application can use. try updating that to maybe '-Xmx1024m' and try to run your service.

I hope it helps.

ojouny · 2009-05-07 15:56:30

I exported the job to run it out of TOS.

The parameter in the generated batch are : java -Xms256M -Xmx1024M .....

1Gb is largely enough or should be as I don't want it to get more memory.

Is there other options for processing the XML input file ?

Announcement

Unanswered posts

#1 2009-05-07 11:34:48

FileInputXML job runs out of memory

#2 2009-05-07 11:44:02

Re: FileInputXML job runs out of memory

ojouny wrote:

#3 2009-05-07 15:34:33

Re: FileInputXML job runs out of memory

#4 2009-05-07 15:56:30

Re: FileInputXML job runs out of memory

Board footer