You are not logged in.
My business entity has the key set on the product name (string). I get errors when the product name includes dashes, punctuation etc (see attached pic). How can this be solved?
You should rather create a simple technical ID, that won't be a source of problems. Have a look at AUTO_INCREMENT or UUID simple custom types, to indentify your entities instances in a unique way.
Setting labels as key lead to this kind of problem, as the value of the key is used to technically name the xml document in the database.
Last edited by csonnefraud (2012-04-16 18:01:06)
Thanks a lot for your reply! I've used IDs before for entities, so I'll go with this then.
I was just wondering ... I am just starting this MDM project. I have to sync the MDM data with already existing data on various databases. Generally speaking it seems like the product name can be used as unique identifier (as it is enforced as such on most of the databases).
What is the common approach in such cases? To create an id and product name on MDM and then to only sync based on product name (unique identifier)? The MDM id will not be available on other databases (it could be possibly added, but I'd rather not do this as it might be quite a long process).
I've already had the same kind of use case, with this form of entity :
- ID (PK, AUTO_INCREMENT)
- Functional Key (Unique)
As you may know, it's not possible to set a unique constraint on an attribute of the datamodel. Only the primary key is unique, by definition.
What I did to ensure that my functional key was also unique was to setup a beforeSaving process on the entity, to guarantee that a duplicate on that attribute would throw an error.
This was done using a simple job that checks the functional key, using :
tMDMReceive --> tMap** --> tUniqueRow --(Uniques)-> tMap --> tBufferOutput (Validation)
-- (Duplicates) -> tMap --> tBufferOutput (Error)
** But in my opinion, joins using labels almost always lead to matching issues. That's why data quality components that use a fuzzy join logic (tRecordMatching for instance) should be preferred in that case.
To ensure a good coherence between your MDM technical IDs and your other databases, you may build & maintain cross-references then.
Hope that helps,
Last edited by csonnefraud (2012-04-17 10:29:04)
Many thanks Cyril for your detailed reply! This is what I was looking for. Yes, in my case the product name has to be unique. Your solution looks very interesting and I will try to implement this.
You also wrote:
"To ensure a good coherence between your MDM technical IDs and your other databases, you may build & maintain cross-references then."
Can you please explain this in more detail?
Sorry for the late reply. Cross-referencing enable you to store relations between your MDM entities, and the related records from your OLTP applications. Cross-references are crucial in a MDM project, as this is where are consolidated the links to your other applications. This leverages the 'single-version of the truth' in a MDM project.
Let's say you hold products in your MDM. It will answer the question : What is the reference 'product' in Talend MDM related to that 'product' in my SCM app ?
For instance, this is the kind of schema a cross-referencing entity will use, holding only primary keys :
IdMDM (PK), IdApp_1, IdApp_2, IdApp_n
Such an entity enables you to link your MDM entities instances to synonymous information stored elsewhere.
You'll have to lookup this entity to find the reference between the technical Id from MDM, and the technical Id from your app.
No more joins on labels, that may bring data quality issues
Talend Enterprise MDM gives you the ability to store & manage cross-references within a dedicated application.
Thanks a lot Cyril for your answer! This sounds very interesting. I'd like to understand how this is set up. Is there a dedicated component for this? I did a Google search but couldn't really find much about this topic. Can you please point me to some info/resources?
You mentioned that this is EE only ... but interestingly enough I found some information on this doing a Google search: Talend is offering a TOS MDM course which includes cross referencing:
http://www.talend.com/services/talend-m … dition.php
Last edited by diddy (2012-04-24 19:24:09)