Talend community forum / crypic errors when deploying job

DavidSevier · 2008-09-05 22:23:30

Greetings again,

I'm now at the point in my project where I need to start testing things on the production server instead of my desktop. I'm running into an issue, though, which I think is related to the custom perl routines that I've added in.

My first issue is that the web scheduler/administrator tool isn't generating any error logs for my job. When I attempt to run it from there, all I get is "Unexpected error, contact your administrator", with a blank log.

I'm assuming that the problem has to do with missing libraries, although without an error log it's hard to say for sure. When you deploy out to a remote execution server, does Talend automatically bundle along any required custom modules that you created in Talend? If not, where should I stick them on the server? Talend's lib directory? Somewhere already in the @inc, like site_perl?

Also, is there a more detailed logfile somewhere that would tell me what error was thrown when I tried to do the deploy?

Thanks!

Last edited by DavidSevier (2008-09-05 23:03:38)

strpdsnk · 2008-09-05 22:55:49

These are where the log files are for me (on the Server the webapp is installed on)
Apache: C:\Program Files\Apache Software Foundation\Tomcat 5.5\logs
TIS: C:\Program Files\Apache Software Foundation\Tomcat 5.5\Talend\logs

Last edited by strpdsnk (2008-09-05 22:57:57)

DavidSevier · 2008-09-05 23:51:29

Thanks for the reply.

Turns out that there was a problem with some env variables on the box that was interfereing with talend, which is why we got suck a cryptic error.

Those have been freed up, and I've gotten farther along. At least, I've gotten an error message.

### Job STARTED at 2008/09/05 14:47:52 (jobId=WfrumdY6, jobExecutionId=krkK0o4H) ###
Can't locate SOEUtilities.pm in @INC (@INC contains: lib /usr/lib/perl5/5.8.5/i386-linux-thread-multi /usr/lib/perl5/5.8.5 /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.1/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl/5.8.4 /usr/lib/perl5/site_perl/5.8.3 /usr/lib/perl5/site_perl/5.8.2 /usr/lib/perl5/site_perl/5.8.1 /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.4/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.2/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.1/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl/5.8.2 /usr/lib/perl5/vendor_perl/5.8.1 /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at lib/routines/EQ2DATAFEEDS/EQ2QuestFeedRoutines.pm line 2.
BEGIN failed--compilation aborted at lib/routines/EQ2DATAFEEDS/EQ2QuestFeedRoutines.pm line 2.
Compilation failed in require at EQ2DATAFEEDS.job_QuestDataFeed_0.1.pl line 19.
BEGIN failed--compilation aborted at EQ2DATAFEEDS.job_QuestDataFeed_0.1.pl line 19.
### Job ENDED WITH ERROR at 2008/09/05 14:47:52 (jobId=WfrumdY6, jobExecutionId=krkK0o4H) ###

This confirmed that, indeed, Talend is not porting over the pm modules that I created within Talend for this project. Seems a bit odd. But I'm happy enough to manually copy them over. I just need to know where to copy them to.

Based on the '@INC contains: lib' part I've tried copying them into Talend's lib directory, and a lib directory under the user Talend is running under. Neither works. Is there some way to set the @INC for talend specifically? I can always do a 'use lib' with perl, but if I do that I'd like it to be at a global level, rather then per module/script.

I'd just copy them into site_perl somewhere, but I don't have permissions for that on this box.

Last edited by DavidSevier (2008-09-06 00:09:47)

DavidSevier · 2008-09-06 01:04:18

Update:

Well, I got around that issue by specifically updating the profile of the user Talend runs as with a local lib directory and copying my modules in there. I'd still love to know if there is a way to get Talend to automatically deploy those modules as well.

New problem, maybe related.

I get an error saying that Time/HiRes.pm cannot be found in the @inc. I'm not using this module specifically anywhere, but it seems to be in the .pl file produced by Talend for this job. Anyone able to shed some light on why this would be needed?

It shows up in this section of the generated code:

# print $_licence;

use Getopt::Long;
use Time::HiRes qw/gettimeofday tv_interval time/;
my %opt = ();

plegall · 2008-09-06 23:00:37

It's right to say that Perl exported script doesn't bring the external Perl module dependencies. For example, if you use tFileInputExcel, you will have to install Spreadsheet::ParseExcel on the deployment server. It is "the standard way" to do things with Perl, the advantage is that exported scripts are tiny (in comparison to Java exported script). But there "more than one way to do it" and we may in the future propose to "export as PAR" and then the exported executable would bring everything required (including the Perl interpreter if I remember well).

Time::HiRes is required because it's often used for statistics or execution time. But...

Code:

$ perl -e 'use Module::CoreList; print Module::CoreList->first_release("Time::HiRes");'
5.007003

... means that this module was added in the code with Perl 5.7.3 (that is to say in 2002) but I see that you use Perl 5.8.5 (please tell us what the "perl -V" command line gives you). It's also a very old release (2004, 4 years ago), please ask your admin to update to 5.10.0 or at least 5.8.8.

About "SOEUtilities" which is a specific module you wrote I think, you can either copy/paste its content in a Talend routine (and it will be automatically included in the exported script), or copy the file in any of the @INC directories the "perl -V" command gives you. The "lib" directory listed by the error trace is the lib directory relative to the script execution directory. "lib" is added to @INC when writing "perl -Ilib" in *_run.sh, so don't take it into account.

DavidSevier · 2008-09-08 22:13:06

I'm not sure what's going on with SOEUtilities. I created it in talend under the Code->Routines section. I have some others I made that way which work fine, but some (including this one) don't. However, if I copy into my site_perl/lib directory everything works fine.

Any idea why Talend is so confused about this? All the PM files are in the same place on the hard drive. In this case, it's: C:\Program Files\Talend\TIS_EE\TIS_EE-All-r16077-V2.4.1\lib\perl\routines\EQ2DATAFEEDS

There doesn't seem to be any obvious reason for the issues.

For example, the SOEUtilities module only contains a single simple function at this point.

Code:

use Exporter;

use vars qw(@EXPORT @ISA);

@ISA = qw(Exporter);
@EXPORT = qw(
    StripWhitespace
);

sub StripWhitespace
{
  my $input = shift;

  $input =~ s/^\s*//;
  $input =~ s/\s*$//;

  return $input;
}

1;

That's the entirety of the module at this point. Eventually I'll add more and more helper routines to it, but for now there isn't enough in there to really be causing any problems.

Any ideas?

plegall · 2008-09-09 00:10:09

This is a bug. I don't reproduce it neither with TOS 2.4.1, nor with TIS 2.4.1. Can you create a bug notification in the bugtracker please?

plegall · 2008-09-09 14:51:17

thanks for opening [Bugtracker, bug 5033, fixed] Perl modules built within TIS not always found by Talend

Announcement

Unanswered posts

#1 2008-09-05 22:23:30

crypic errors when deploying job

#2 2008-09-05 22:55:49

Re: crypic errors when deploying job

#3 2008-09-05 23:51:29

Re: crypic errors when deploying job

#4 2008-09-06 01:04:18

Re: crypic errors when deploying job

#5 2008-09-06 23:00:37

Re: crypic errors when deploying job

Code:

#6 2008-09-08 22:13:06

Re: crypic errors when deploying job

Code:

#7 2008-09-09 00:10:09

Re: crypic errors when deploying job

#8 2008-09-09 14:51:17

Re: crypic errors when deploying job

Board footer