You are not logged in.

Unanswered posts



Important! This site has been replaced. All content here is read-only. Please visit our brand-new community at https://community.talend.com/. We look forward to hearing from you there!



#1 2009-08-27 21:55:29

dadumas
Member
219 posts

dadumas said:

[resolved] tMap needs to support multiple lookups to the same table (alias)

Tags: [database]

In a Data Warehouse Star schema scenario it is a common requirement to be able to lookup multiple surrogate keys based on the same physical lookup table.  Example: A (main) order record may contain the following columns: OrderDate, ProcessedDate, ShippedDate, etc - all needing separate surrogates from a date dimension in the data pipeline.  Therefore the Date Dimension needs to be used as a lookup multiple times.  In OLAP terms this is called a role-playing dimension.  In database terms this is just an alias.

So, rather than duplicating the Date dimension table multiple times as mutiple tInput lookup components, and using up memory, it would be great to use a single tInput lookup component, and in the tMap, be able to alias that single input lookup multiple times, joined from the desired main input.  That makes for a simpler design, and should save valuable memory.

Dave

Offline

#2 2009-08-28 09:04:37

nkxxDV
Member
31 posts

nkxxDV said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

+1

Thank you for saying out loud !

Offline

#3 2009-09-30 16:25:00

cantoine
Talend Team


cantoine said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

Hi,

It's already exist into our tELTMap you can refer multiple INPUT TABLE for the same using Alias.

Best regards;

Offline

#4 2009-09-30 16:40:38

dadumas
Member
219 posts

dadumas said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

cantoine wrote:

Hi,

It's already exist into our tELTMap you can refer multiple INPUT TABLE for the same using Alias.

Best regards;

Any way to include that  functionality in the non ELT (regular) tMap?

Dave

Offline

#5 2009-10-07 18:11:57

dadumas
Member
219 posts

dadumas said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

cantoine wrote:

Hi,

It's already exist into our tELTMap you can refer multiple INPUT TABLE for the same using Alias.

Best regards;

Wanted to ask again:

Any way to include that  functionality in the non ELT (regular) tMap?  We are not using ELT.

Dave

Offline

#6 2009-10-08 09:09:26

amaumont
Talend Team


amaumont said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

We intended to manage this possibility as you can see in this Wiki page : http://www.talendforge.org/wiki/doku.ph … :hash_file

tHashOuput and tHashInput components exist but they are not official, they don't take account StoreOnDisk mode, and they don't work well when they are directly connected to a tMap.

Yet, I will speak about this subject at our next meeting.

Offline

#7 2009-10-08 15:56:10

dadumas
Member
219 posts

dadumas said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

amaumont wrote:

We intended to manage this possibility as you can see in this Wiki page : http://www.talendforge.org/wiki/doku.ph … :hash_file

tHashOuput and tHashInput components exist but they are not official, they don't take account StoreOnDisk mode, and they don't work well when they are directly connected to a tMap.

Yet, I will speak about this subject at our next meeting.

That would be great.  It is extremely common to have a date dimension aliased many times for lookups.

Offline

#8 2010-01-15 21:01:23

recruta42
Guest

recruta42 said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

Is there any solution for this? I have to load 3 times the date and hour tables !!!

#9 2010-01-15 21:09:21

dadumas
Member
219 posts

dadumas said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

recruta42 wrote:

Is there any solution for this? I have to load 3 times the date and hour tables !!!

Im tMap, you can create a lookup based on the same table more than once, but, essentially, you are incurring the extra memory for each duplicated lookup table, nit to mention cluttering up the job display .  Also, if the solution adds inferred dimension members on the fly, and that dimension is on eof the dups, then I would doubt that the others would get get updated. - not sure if Talend dynamically updates lookup cache as records are added.

I am used to SSIS (I like Talend better), where the same lookup table can be aliased several times.  Also, added inferred dimension members are dynamically added to the lookup cache as well.

Hopefully talend will jump on this one.

Dave

Offline

#10 2010-01-16 02:51:09

recruta42
Guest

recruta42 said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

And could you explain to me how can I do that on Tmap component ? Thanks in advance

#11 2010-01-18 10:04:36

recruta42
Guest

recruta42 said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

recruta42 wrote:

And could you explain to me how can I do that on Tmap component ? Thanks in advance

Anyone ?

#12 2010-08-19 18:14:03

dadumas
Member
219 posts

dadumas said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

cantoine wrote:

Hi,

It's already exist into our tELTMap you can refer multiple INPUT TABLE for the same using Alias.

Best regards;

Cantoine,

This actually already works using a regular tMap,  I successfully used the same table multiple times in a single tmap.  Since Talend automatically aliases each lookup, it just treats each instance like another table - works great.

This would make a great webinar. 

Dave

Offline

#13 2010-08-31 19:57:18

DrewBlay
Member
1 post

DrewBlay said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

Maths Games

and for example, one subject from the list:
interactive Times Tables Games

Offline

#14 2016-11-28 07:56:14

schadha212312
Guest

schadha212312 said:

Re: [resolved] tMap needs to support multiple lookups to the same table (alias)

I am migrating from datastage to talend. I have a requirement where i am performing 200 reference lookups in 3 lookup stages in a set of 70-70-60 per lookup stage. All the reference lookups for each lookup stage are coming from one source, How should this be implemented in Talend as tmap maps one to one amongst its components.

Board footer

Talend Contributor Agreement - Talend Website Privacy Policy