You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2013-03-13 15:16:17

l.mouna
Member
40 posts

l.mouna said:

tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

hello everyone,

i tried this tuto:

https://help.talend.com/display/TalendO … 6+tPigJoin

i searched on the net how to fix the error but i didn't find how.
any help please!
Thank youuu
________________________________________________________________

3/03/13 14:44:50 INFO executionengine.HExecutionEngine: Connecting to hadoop file system at: file:///
13/03/13 14:44:51 INFO pigstats.ScriptState: Pig features used in the script: HASH_JOIN
13/03/13 14:44:51 INFO rules.ColumnPruneVisitor: Columns pruned for tPigJoin_1_LOOKUP: $1, $2, $3, $4
13/03/13 14:44:51 INFO mapReduceLayer.MRCompiler: File concatenation threshold: 100 optimistic? false
13/03/13 14:44:51 INFO mapReduceLayer.MRCompiler$LastInputStreamingOptimizer: Rewrite: POPackage->POForEach to POJoinPackage
13/03/13 14:44:51 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size before optimization: 1
13/03/13 14:44:51 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size after optimization: 1
13/03/13 14:44:51 INFO pigstats.ScriptState: Pig script settings are added to the job
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: Setting up single store job
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=0
13/03/13 14:44:51 INFO mapReduceLayer.JobControlCompiler: Neither PARALLEL nor default parallelism is set for this job. Setting number of reducers to 1
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 1 map-reduce job(s) waiting for submission.
13/03/13 14:44:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/03/13 14:44:51 ERROR security.UserGroupInformation: PriviledgedActionException as:admin cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 0% complete
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: job null has failed! Stop running all dependent jobs
13/03/13 14:44:51 INFO mapReduceLayer.MapReduceLauncher: 100% complete
13/03/13 14:44:51 WARN mapReduceLayer.Launcher: There is no log file to write to.
13/03/13 14:44:51 ERROR mapReduceLayer.Launcher: Backend error message during job submission
java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
    at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
    at java.lang.Thread.run(Unknown Source)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
13/03/13 14:44:52 ERROR pigstats.SimplePigStats: ERROR 2997: Unable to recreate exception from backend error: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
    at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
    at java.lang.Thread.run(Unknown Source)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)

13/03/13 14:44:52 ERROR pigstats.PigStatsUtil: 1 map reduce job(s) failed!
13/03/13 14:44:52 INFO pigstats.SimplePigStats: Detected Local mode. Stats reported below may be incomplete
13/03/13 14:44:52 INFO pigstats.SimplePigStats: Script Statistics:

HadoopVersion    PigVersion    UserId    StartedAt    FinishedAt    Features
1.0.0    0.10.0    admin    2013-03-13 14:44:51    2013-03-13 14:44:52    HASH_JOIN

Failed!

Failed Jobs:
JobId    Alias    Feature    Message    Outputs
N/A    tPigFilterColumns_1_RESULT,tPigJoin_1_JOIN_RESULT,tPigJoin_1_LOOKUP,tPigJoin_1_RESULT,tPigLoad_1_RESULT    HASH_JOIN    Message: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\admin-1787914753\.staging to 0700
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:682)
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:655)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
    at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:856)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
    at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
    at java.lang.Thread.run(Unknown Source)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
    C:/Hadoop/exemple/outputFile.csv,

Input(s):
Failed to read data from "C:/Hadoop/exemple/refFile.csv"
Failed to read data from "C:/Hadoop/exemple/inputFile.csv"

Output(s):
Failed to produce result in "C:/Hadoop/exemple/outputFile.csv"

Job DAG:
null


13/03/13 14:44:52 INFO mapReduceLayer.MapReduceLauncher: Failed!
_________________________________________________________

Offline

#2 2013-03-17 20:52:14

l.mouna
Member
40 posts

l.mouna said:

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

I'm trying to run it with the "Local Mode"...

any answers please?

Offline

#3 2013-03-20 00:33:20

cdynes
Talend Team


cdynes said:

Re: tPigLoad, tPigJoin, tPigFilterColumns et tPigStoreResult

Hi

I don't use Hadoop on windows. Perhaps http://stackoverflow.com/questions/1050 … in-windows  might offer some insights.


Ciaran

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy