You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2016-07-27 13:21:38

mardiris
Member
1 post

mardiris said:

can I batch load csv files in Hive?

Hello community,
can I use talend in order to load 1000s of csv files into Hive?
if yes, does it matter if the delimiter is , or ; and whether the text fields are within "" or not?
Can the schema be auto detected and the respective tables automatically created?
Is there the functionality of configuring a hotfolder process (the source system is saving csv files in a folder and at regular intervals it is checked whether new files exist in the hotfolder, if yes, then these files are imported into Hive)?

Offline

#2 2016-07-30 20:15:39

jlolling
Member
3186 posts

jlolling said:

Re: can I batch load csv files in Hive?

You can do this for sure but at first you never load files into Hive. You load files into the HDFS and you can use Hive to interpret such files as database files. Hive is not a data store, it is a method to run SQL on files on the HDFS with meta data configured in Hive.


Talend Certified working for cimt objects AG in Berlin

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy