You are not logged in.
Announcement
Unanswered posts
|
Pages: 1
The following error is being thrown whe I try to load a large csv file:
Exception in thread "main" java.lang.Error: Unresolved compilation problem:
The code of method tFileInputDelimitedProcess(Map<String,Object>) is exceeding the 65535 bytes limit
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.tFileInputDelimitedProcess(ShadowFileInputToDelimitedOutput.java:12219)
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.runJobInTOS(ShadowFileInputToDelimitedOutput.java:24747)
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.main(ShadowFileInputToDelimitedOutput.java:24645)
Preview error. Some settings must be changed.
Note: Preview errors are generally due to a wrong encoding setting.
org.talend.designer.runprocess.shadow.ShadowFilePreview.preview(Unknown Source)
org.talend.repository.ui.utils.ShadowProcessHelper.getCsvArray(Unknown Source)
org.talend.repository.ui.wizards.metadata.connection.files.delimited.DelimitedFileStep2Form$PreviewProcessor.nonUIProcessInThread(Unknown Source)
org.talend.commons.ui.swt.thread.SWTUIThreadProcessor$1.run(Unknown Source)
Thanks
David......

Getting the same error message about exceeding the 65k using the tFileInputJson component. 65k? Talend folks, please, you need to fix this.
Thanks
Offline
Hi dforrestca
This is a well-known problem, you can find some explanation in topics:
http://www.talendforge.org/bugs/view.php?id=6766
http://www.talendforge.org/bugs/view.php?id=14512
Best regards
Shong
Offline

A well known problem is definitely a bug, yes? Thank you for the response. Anyway, the other postings pointed to mostly refer to splitting up a job to solve the issue. We are having a problem with the 65k limit reading in a json data stream with the the Tfileinputjson component. Please see the attached pic of a job setup to split the input stream. The main glitch with this is I think tJoin was intended for database input as the lookup feature seems to be not working for us.
Anyone have an example they wish to share of combining to file inputs into one? Tried using the json doc "_id" as a matching element, but, still seems not to join them up.
Any help appreciated,
Thanks
Offline
Hi
The problem is there are so many columns in the tFileInputJason component, the generated code exceed the 65k byte limit.
Please see the attached pic of a job setup to split the input stream. The main glitch with this is I think tJoin was intended for database input as the lookup feature seems to be not working for us.
Do you want to merge the two input stream or join them?
Best regards
Shong
Offline

Hi Shong,
Thanks again for reply. I think we want to join? What we would love to do is read the json stream in all at once, but, that 65k barrier is killing us. SOoo... I thought to split the json schema into two logical parts and read the entire URL by both of them in separately and then join then back together as one. I hope we can find a better way to do this long term, but, need to have it working now. Its fantastic that we can pull a row from a couchdb database by URL using the tInputjsonFile and we need this to work with a whole stream. Part of the issue too is that talend throws errors when there is json data without a mapping from the json pathing associated to the schema. In other words, let's say the stream contains Elements A,B,C,D, E, F. With one tFileJson the schema and json pathing are to A,B, and C, the second tFileJson has D,E,F. The first one will throw the "json data structure....may not be right" error because elements D,E,and F are not in the schema/path and the second fFIleJson throws the error because elements A,B,C are not in the schema/path. Please, there must be a workaround we can do until you guys resolve the 65k issue. We could also restructure the generated code with a little refractoring.
Please advise...
Thanks and regard,
dforrestca
Offline
Hi,
Being new I can't load the prevew to setup the tfile component to be able to split error I am getting in the preview is:
Exception in thread "main" java.lang.Error: Unresolved compilation problem:
The code of method tFileInputDelimitedProcess(Map<String,Object>) is exceeding the 65535 bytes limit
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.tFileInputDelimitedProcess(ShadowFileInputToDelimitedOutput.java:12219)
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.runJobInTOS(ShadowFileInputToDelimitedOutput.java:24747)
at importmasterdata.shadowfileinputtodelimitedoutput_0_1.ShadowFileInputToDelimitedOutput.main(ShadowFileInputToDelimitedOutput.java:24645)
Preview error. Some settings must be changed.
Note: Preview errors are generally due to a wrong encoding setting.
org.talend.designer.runprocess.shadow.ShadowFilePreview.preview(Unknown Source)
org.talend.repository.ui.utils.ShadowProcessHelper.getCsvArray(Unknown Source)
org.talend.repository.ui.wizards.metadata.connection.files.delimited.DelimitedFileStep2Form$PreviewProcessor.nonUIProcessInThread(Unknown Source)
org.talend.commons.ui.swt.thread.SWTUIThreadProcessor$1.run(Unknown Source)
Offline
What would be nice is a tutorial of a work around for anyone that gets this error.
Thanks
Offline

Hi Dcostelloe,
Shong has another posting out there on another topic that brings three spreadsheets together in one stream.. http://www.talendforge.org/forum/viewtopic.php?id=11978
Using that as an example, we went forward and have been getting the desired output, though it still needs a little tweaking.... see below
Offline
Ah yeah looks like there are multiple files, I only have one large file which I can not Split out around 1000 columns with 29924 rows which is causing the issue with limitation. I can't split out the file as this is how the data is provided.
Will try the suggestion in the forums again.
Offline
This is what I tried:
Took 50% on Part1
50% on Part2
Last edited by dcostelloe (2011-07-03 16:17:07)
Offline
Pages: 1