You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2017-01-12 04:34:35

jpmauss
Member
25 posts

jpmauss said:

[resolved] tJava BD Batch Spark Job

In a Big Data batch Spark job, you can write custom java code in tJava for your input rdd.  Is it possible to input multiple rdds to the same tJava component so the custom code can by written to leverage different data sets imported earlier in the job?  If so, could someone share an example.

Last edited by jpmauss (2017-01-12 04:35:52)

Offline

#2 2017-01-12 21:50:35

jpmauss
Member
25 posts

jpmauss said:

Re: [resolved] tJava BD Batch Spark Job

After testing out a few different options, I figured out one that should work well.  java/spark code to read textfile from hdfs to a new rdd within the custom code so I can leverage that data set and the rdd that is an input.  Make sure to add java rdd imports in advance settings.

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy