Talend community forum / Embedded Talend?

mikem2005 · 2008-06-21 04:15:34

Is it possible to use Talend embedded inside another application (similar to how CloverETL allows embedded mode)? The goal would be to programmatically generate an ETL pipeline and then have Talend run it.

Thanks!

shong · 2008-06-23 07:24:04

Hello

tSystem component executes one or more command as you do on CMD. This component can typically used for companies which already implemented other application that they integrate into their processing flow through Talend.

Best regards

shong

mikem2005 · 2008-06-24 15:49:51

Instead of having Talend call another process using tSystem, can the reverse be done? Can I have my system configure and run a Talend pipeline - so Talend is embedded into my application?

Thanks!

fyaghob · 2008-06-24 16:02:53

You can export your job as a Java POJO package (or as an Axis WebService). I'm using this packages for my projects (in java) everyday.

Last edited by fyaghob (2008-06-24 16:04:28)

mikem2005 · 2008-06-24 17:17:21

Exporting as a Java POJO sounds great for running the job, but doesn't that still require the UI to output the POJO?

Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?

Thanks!

GéomatiLux · 2008-06-24 17:47:19

Hi,
Talend Export Job Scripts feature allows you to deploy and execute a job on any server, regardless
Talend Open Studio.
The job scripts export adds to an archive all the files required to execute the job, including the .bat and
.sh along with the possible context-parameter files or relative files.
The export type include also Java POJO, Axis Webservice (WAR) and Axis,Webservice (Zip).

Best look

mikem2005 · 2008-06-24 23:44:19

Perhaps my question is not clear. I understand that I can export an ETL pipeline I've created in the UI and then run that job embedded in another Java application.

My question:
Can I create a new job, configure it, and export it using *only* the Talend API, not using the Talend UI at all?

fyaghob · 2008-06-24 23:54:21

mikem2005 wrote:
Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?

I mean you need own task generator, for recompute all in the space ...
Create a job for a new data transfer, export the job to POJO package and embede generated package to your own app. If you need transfer something from somewhere to someplace, you need "somegenerator". But I mean, with well configured and prepared Talend job, you can use only one package and "something" define via properties and contexts.

coombsj · 2009-10-08 00:16:20

mikem2005, did you ever get your question answered? I'm looking for the same thing. As far as I can tell the only way to create a new job is through the TalenD UI.

There doesn't seem to be an API exposed for other applications to create jobs.

pravu · 2009-10-08 06:39:24

Looks like Pentaho kettle uses an API. May please try to find out .

Please let us know in case it is useful.

Thanks and Regards,
Pravu Mishra.

Morbo · 2009-10-08 22:14:46

Well, TOS is opensource, it'll be a pain, but you can try to look at the jar files & their API.
A talend job is basically a xml file (see .item files) serialized from an EMF model (see the eclipse project).
TIS (the non free version of TOS) has a console which can then compile the project.

pravu · 2009-10-09 06:18:40

Hi Morbo,

The question is not about design of Talend and using Eclipse modelling framework(EMF) or java classes without having intergace implementation.

The question is about whether there is/are APIs which can be used independently from out side of TOS. Please let us know in case you have some idea about the API/s exposed by talend.

There are many third party APIs those are used by talend. Might be there are APIs developed by talend on top of third party API. Do you have any idea about it?

Any way talend is an excellent software.

Thanks and Regards,
Pravu Mishra.

Morbo · 2009-10-09 18:24:37

Hello,
I don't know much, I just did some digging on some area.
All I'm saying is that's it's already possible to manipulate the .item file in a regulated manner (thanks to EMF) and without TOS. I'm made a basic gui launcher wich uses this to read the item file, deduce the context parameters to expose.
See this about the EMF model of talend http://www.talendforge.org/forum/viewtopic.php?id=6606
And this was a start of discution about talend use of GEF : http://www.talendforge.org/forum/viewtopic.php?id=7237 but I didn't go further.
Feel free to share your investigations :p.
Regards

Announcement

Unanswered posts

#1 2008-06-21 04:15:34

Embedded Talend?

#2 2008-06-23 07:24:04

Re: Embedded Talend?

#3 2008-06-24 15:49:51

Re: Embedded Talend?

#4 2008-06-24 16:02:53

Re: Embedded Talend?

#5 2008-06-24 17:17:21

Re: Embedded Talend?

#6 2008-06-24 17:47:19

Re: Embedded Talend?

#7 2008-06-24 23:44:19

Re: Embedded Talend?

#8 2008-06-24 23:54:21

Re: Embedded Talend?

mikem2005 wrote:

#9 2009-10-08 00:16:20

Re: Embedded Talend?

#10 2009-10-08 06:39:24

Re: Embedded Talend?

#11 2009-10-08 22:14:46

Re: Embedded Talend?

#12 2009-10-09 06:18:40

Re: Embedded Talend?

#13 2009-10-09 18:24:37

Re: Embedded Talend?

Board footer