You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2017-04-03 09:23:36

mitulvyas
Member
16 posts

mitulvyas said:

Dynamic CouchDB schema mapping to RDBMS

Hi All,

I am developing an ETL job whereby a variety of different JSON schemas are extracted from CouchDB and then converted to RDBMS. This will be a flat JSON documents after 2 level (which needs to be extracted). Each key in the JSON is mapped 1 to 1 to a column in a table. Right now, I am manually creating a JSON file in metadata for each schema, and manually mapping it to the corresponding table. However, in a few months, the amount of different JSON schemas received will increase exponentially, so manually creating a metadata file and mapping for each schema will not be feasible.

Is there a way I can build a job that generically takes any JSON schema I throw at it, and automatically maps the value to a column in a table (where the name of a column = the JSON key), maybe using tSetDynamicSchema, or write/extract dynamic fields?

Adding the information to the problem, each different type of schema goes into a different table. The data from another configuration paramters defines what the target table is, which is currently achieved by setting the relevant key/value at the configuration DB. I can take the variable (containing the table name) to the context variable and push the data to the relevant table.

P.S.: I'm trying to implement the solution on Big Data Sandbox 6.2. and the Target System is MSSQL.

A quick revert with example will be much appreciated.

Thanks,
Mitul

Offline

#2 2017-04-04 13:12:26

mitulvyas
Member
16 posts

mitulvyas said:

Re: Dynamic CouchDB schema mapping to RDBMS

Hi, Could You please Respond?

Thanks.
Mitul

Offline

#3 2017-04-04 15:39:15

dwilmer
Talend Team


dwilmer said:

Re: Dynamic CouchDB schema mapping to RDBMS

This sounds like a case for a POC.  Please reach out to your area Talend Rep for details and further qualification.  Otherwise, please post this to the more general Big Data Forum.
Thanks!

Offline

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy