How to organise for MDM?

Have you ever thought through how to build an organisation for master data management?

One of the key issues I always find at customers who are struggeling with their quality of master data is that their organisation is not really prepared to deal with master data in a sustainable manner.

In my view Gartner has done quite a good job in proposing a generic organisation for master data management. Actually this organisation implies that you are introducing a new business process - Master Data Management.

Please look at the following orgchart - this is how I remember the generic organisation Gartner is proposing:

The generic MDM organisation

One of the key ideas is that business has to lead the MDM organisation. A quick description of the different roles & responsibilities:

1. The Information Governance Board should consist of executive level sponsors. They should  set and enforce information management policies. In this board you should have representatives from the business - that is key - but you also should have a representation from your IT executives in here.
2. The MDM Team is responsible to manage the MDM program and who authors and maintains the master data. If you are also implementing GDSN or any other means to collect item data from your suppliers the supplier onboarding and communication is also a key task for this team.
3. Data Stewards sit in the business unit and they are responsible for data quality monitoring, improvement and issue resolution. They are NOT maintaining the data themselves but work closely with the MDM Team to get the master data quality on to the level the business needs it.
4. The MDM Infrastructure Team is a kind of virtual team responsible for all aspects of the IT implementation needed for the MDM business process.
5. In the GDSN context I only want to highlight the importance of the Data Modeling and Information Architecture role. If participating in the GDSN this role is key because it is the link to the GSMP process and defining change requests and also anticipating and adopting the changes coming through the GSMP / GDSN.

Please take this organisation template as what it is - only a template. In real life you have to look at your current organisation and see how you can adopt the different roles and responsibilities within your organisation.

But also be aware - you want to introduce a new business process which you did not have in the past.

And if your organisation is multinational or even global I will discuss in one of my next posts what impact that has on the MDM organisation.

Don't brands want to communicate transparently to the consumer?

Do you remember one of my earlier postings Brand owners loose sales due to mobile scanning?

Actually my main message was that brands have to invest into MDM to be able to publish their product information in a high quality way to all those app providers to have control over the branded product information publically available.

I thought that it is mainly a technical and effort issue!

But I had to learn that there are also other issues.

Last couple of days there was the SA2 Worldsync's Annual User Congress 2011where we promoted a lot the new mobile century. We had great presentations from the mobile space (eg. from Mirasense and from barcoo) which made it clear that there is a massive demand for extensive product information from the consumer. And those app providers are fulfilling this demand - either with or without the support from the brand owners.

We got quite some good feedback from some major brand manufacturers for our activities to deliver trusted, manufacturer product information to those platforms. See one of our press releases here.

But we also got quite some negative feedback from some other major brand manufacturers.

Quote: "That is not in our interest."

It is not in your interest to communicate with your customers?
It is not in your interest to provide correct and honest information to your customers?
It is not in your interest to use the new media?

One argument I heard was "We cannot give allergene information to the consumer via such platforms because then we could be held liable for it. And we could be sued if the data is incorrect and thereby somebody gets harmed.".

But you are putting allergene information on the packaging and on your website.

Are your processes so much more reliable for putting information on the packaging or on your website then publishing it to the GDSN?

Or are there other reasons behind your reluctancy to provide transparent information to the consumer?

Actually I haven't got it yet. I would be happy to get some feedback on that topic ...

Even with GDSN - retailers product master data is still crap!?

Just recently GS1 Belgium did  a "Mini Data Crunch Survey 2011" where they did a survey what the impact of bad data on the belgian grocery market is. This is a survey which stands in a line with what GS1 UK started with their "GS1 UK Datacrunch Report 2009".

I think GS1 Belgium really put together a very nice report - and the results are - I would like to say "as expected" - very bad. The data between suppliers and retailers is significantly out of sync. And there is a huge cost to the market for that.

What I liked most is that GS1 Belgium not only looked at suppliers and retailers who were not yet doing GDSN but also at suppliers and retailers who are heavy GDSN users. Surprisingly by looking at the figures they have measured regarding "missing data" and "mismatches supplier-retailer" you really cannot tell who is doing GDSN and who is not. The percentage of incorrect data ranges from 49% to 67% - and keep in mind they only researched the data of 4 suppliers at 4 retailers and even that with only 100 products:

I would have expected that the retailers who are using GDSN have significant better data then those who are not using GDSN at all.

If that is not the case, what are the reasons for those issues?

GS1 Belgium does not directly address this subject, but indirectly they are proposing an answer with their analysis what is happening behind the firewalls of suppliers and retailers.

Obviously suppliers and retailers still have to do a lot of manual work on both sides of the table even if they have implemented an automated GDS process.

From my perspective the two most important recommendations of GS1 Belgium are:

1. Use an automated exchange of data between suppliers and retailers. Here GS1 Belgium for a good reason refers to their GDSN datapool offering. But I think that is only the one part needed. The other part is that retailers have to implement internal automatic integration processes to really integrate the data into all their business applications.
   
   Just recently I found out at a major german retailer who is already doing GDSN for quite some time that they are not integrating the GDSN data into their purchasing system!? How do they expect to benefit from GDSN for their order and invoice processes???
   
2. Introduce a so called "data team" on suppliers and on retailers side to be the one and only team to maintain the data.
   

They are also proposing a lot of other very reasonable actions to be taken like establish data ownership, data quality dashboards, etc.

But I really think you have to establish a dedicated organisation and then you have to integrate the data into your business applications in an automated way. Actually straight forward, isn't it?

But my expierence is that suppliers and retailers are really struggeling at this point and typically do not implement this consequently ...

Let me repeat my mantra: Suppliers and Retailers simply have to implement a MDM program and make their GDSN initiative part of that MDM program. This is the only way to implement GDS successfully and gain the business benefits out of it.

Retailers: GDSN Changes - the productivity killer?

Lately I talked to one of the bigger european retailers who has recently fully adopted the GDSN processes to receive item data from his suppliers.

Meaningless changes!?

He was complaining that they were receiving so many changes and that he needed more staff to get control of this stream of changes although he thought that most of those changes were not meaningful to them!?

How can a change of an item from a supplier be not meaningful to a retailer? At first sight I really questioned this statement and we started investigating what was happening.

First of all I sat down for a couple of days with the data stewarts who are in charge to approve all changes to really see what was happening.

50 - 1,000 changes / day

My first observation was that they really had to approve a huge amount of changed items each day. They had to manage a minimum of 50 changes per day but also one day it was more than 1000 changes which they were not any more able to approve with the existing staff in that day and changes began to pile up.

Up to 10 minutes for approval of a single item

My second observation was that it was quite painful to look at each and every item and to figure out what the change was. Although they have implemented a quite fancy looking comparison screen it takes the data stewart up to 10 minutes to do the item review and to decide whether to synchronize, review or reject the item.

90% of changes meaningless

My third observation then was the eye opener. Approx. 90% of the changes were meaningless for this retailer. How come? Actually every retailer is only using a subset of the attributes which are available in the GDSN. None uses all 1,600 attributes available. All retailers I have seen use between 100 and 250 attributes. But suppliers have to maintain actually the superset of all attributes required by their retail customers.

So what is happening is that supplierA is maintaining Attr1 - Attr200, while retailerB is using Attr1 - Attr100 and retailerC is using Attr90 - Attr200.

If supplierA is changing Attr1 of an item this change gets send automatically to retailerB and retailerC. But it is only relevant for retailerB. retailerC is not using Attr1, so the change is not relevant to him. But according to the GDSN rules he has to synchronize this version of the item to stay synchronized with the supplier. Sending back a CIC review or reject would not make any sense because that is just not what he wants to do.

Solution 1

What is the solution? The first solution approach is to evaluate automatically whether a change is relevant to a retailer and if it is not to automatically synchronize (meaning store the item and send back a CIC synchronized) the item change. Only if the change is relevant put the item into the manual approval workflow.

With this solution this retailer could already save up to 50-60% of the needed effort in his approval process.

Solution 2 - and my recommendation

My recommendation is to take it even one step further. Why do you have to look at each and every item change even if it is relevant to you? Do you really think that your data stewart can judge for all your items whether a slight change in the measurements or in the ingredients or in the packaging or where ever is correct without pulling the concrete product and remeasuring?

My experience is that the whole manual approval process mostly is pure waste of time.

Instead turn the process around. Define rules when an item change cannot be automatically approved (eg. if the description changes, if measurements change more then 10%, etc.). If an item change passes all those rules it should be synchronized automatically. Otherwise you put it to the manual approval process.

And then measure what is your ratio during manual approval between sending a review or reject to the supplier and synchronized. If you have more than 50% getting an immediate synchronized than your approval rules are still to strict and you should further relax them.

Now you should be down to 10% or even less of the original effort.

Btw. dealing with ADD's is typically a little bit more complicated because you typically have a manual enrichment process ...


UPDATE: 
There is a very interesting discussion on LinkedIn regarding this Blogpost, see here.

What are the core components of a MDM system?

Actually we are talking  a lot about MDM systems – but hey - are you aware what  a full blown MDM system should provide?

In my opinion the following modules are essential to a MDM system:

• A very flexibel Data Model. This is key and actually the core. You want to manage master data and you have to be able to put the data model into the system which meets your requirements.
  
  For me managing the data model is NOT a development task but instead a pure customization task which should be taken over by the people who are operating and administrating the MDM system. In the best case the MDM Team itself is able to adjust the data model to their needs.
  
  The flexible Data Model should be accompanied by a validation rules engine. What you want to do when defining your data model you also want to define the data quality rules your master data should comply to. And those data quality rules are typically more complex then just a primitive type check or the check for mandatory attributes.

• A Content or may be better “Item” Management Module which allows the different departments actually to manage (enter and maintain) the master data itself. And man – do not mix that module up with a web content mangement system (CMS) that is something completely different.
  
  This module should be fully driven by the data model (meaning again no programming if you add an attribute) and allow to customize different views for different users or user groups.
  
  It should be also integrated with some workflow engine because you might want to establish editorial and approval workflows for master data management.
  
  
• A Data Quality Module. This module should provide you the capabilities to monitor the data quality you have in your system. Thereby this should be integrated with your validation rules engine so that you can report on which items do not comply to your rules and stuff like this.
  
  It is also great if the MDM system already provides simple data quality analysis functionalities like pattern analysis or filling rate analysis independent from your defined data quality validation rules.
  
  
• A Data Integration Module. This is essential because you want to integrate your master data into the diverse other systems you have. What you need is batch upload and download functionality plus you need realtime integration capabilities eg. via  a Enterprise Service Bus.
  
  
• A Workflow Engine should be part of the MDM system. Actually as data models differ from company to company also the editorial workflows but also the approval workflows differ from company to company. This gives you a great flexibility to model your processes into the system and it helps that you not have to adjust yourself to the processes defined within the tool.
  
  
• Webservices should be supported for integration purposes. As master data is needed in most areas of the business having public webservices often helps to solve integration requirements very quickly.
  
  
• For sure you have to look at Technology and Architecture that this fits into your Company policies.
  
• Performance, Scalability and Availability are also key requirements for a business critical application like a central MDM system

• You should also look at operating and security requirements and whether the system meets them.

• And last but not least a MDM system should also provide at least a minimum of reporting capabilities.

And as you know my focus is always not only MDM but also how MDM connects and enables GDSN. So if your requirement is that your MDM system should also support your GDSN initiative what is required then?

1. Best would be if your MDM vendor already provides you with a out-of-the-box GDSN connector which is certified – or at least know to work with – your favorite data pool. But as there are not that many MDM solutions out there with GDSN connectors out-of-the box you should look at option 2.
   
   
2. To support GDSN with your MDM you need:
1. Your data model has to be aligned with the GDSN data model. Either you have your own and you can map that to the GDSN data model (be aware of codelist issues, attribute type issues and stuff like that) or you directly adopt the standards and use them also internally.
   
2. You have to build workflows which are supporting the item confirmation process.
   
3. Your MDM systems integration capabilities should support the data formats and protocols your data pool supports

I hope this helps you to structure your evaluation process of a MDM system!

What is “100% Data Quality”?

Isn’t your goal “100% Data Quality”? And don’t you expect (at least if you are  a retailer) your suppliers to deliver via GDSN 100% correct item data?

But what does this simple term “100% Data Quality” really mean?

Imagine that you are a retailer. You are receiving item data from your suppliers via GDSN. You are feeding that data directly and exclusively into your warehouse system. And your warehouse is fully automated so you are really dependent to receive from your suppliers the correct measurements. So you are very proud of your warehouse logistics and you are managing that in the most efficient manner which gives you serious benefits over your competition. But because you are so focused on efficient logistics you have not yet implemented any ecommerce strategies and therefore you have very little demand in any additional product information like nutritional information or things like that.

So for you “100% Data Quality” does mean – get me the measurements and my other logistics information (like packaging hierarchies) correct!

Now imagine you are a e-commerce retailer. What you need to sell is good images and very detailed product feature descriptions. If you are selling food you need nutritional and allergen information. In some countries you need this already for legal reasons.

Now “100% Data Quality” means logistic information including measurements + feature information + images.

And now try to imagine what “100% Data Quality” means to a small supplier who is confronted with forms with hundreds of attributes to fill out. He is happy when he gets his data passed the damn data pool validations!

Got my point?

Data quality – and my examples so far only touched on the completeness of the data – is “relative”. It is dependent on the requirements of the recipient. If you are using the measurements in your systems and your business processes depend on that information, you need it to have “100% Data Quality”. If you do not use it you can still have 100% Data Quality for your purposes if the measurements are incorrect or even empty.

Same applies for all dimensions of data quality: correctness, completeness, in time delivery and validity.

Imagine the perfect world all retailer requirements regarding data quality are equal because they are fully using the full set of product information in all their processes. But in todays world implementation levels of retailers are different and therefore they today might have very different requirements depending on the business processes where they are using the data today.

My recommendation to retailers would be, give a clear communication and documentation to your suppliers what you consider to be “100% Data Quality” and build into your system validations to ensure that quality level including feedback for suppliers if their data does not meet those requirements.

To suppliers I recommend ask your retail customers what they understand under the term “100% Data Quality”. Then start implementation accordingly.

MDM and GDSN hard to connect? The original post ...

Actually when I started writing my last blog post on the topic why MDM and GDSN are hard to connect my original intention was not to write about a flaw in the design of GDSN. This just happend to come to my awareness while I was thinking through the whole "How to connect a MDM system to GDSN?".

My original thought was that GDSN actually allows - and in my view even "requires" if you want to get all the benefits out of it - interaction between the supplier (data source) and the retailer (data recipient) when you are synchronising item data. This interaction is mainly the feedback from the retailer to the supplier what he has done with the data on item level via the CIC (Catalog Item Confirmation) message.  With the CIC message the retailer can ask the supplier to review the send data because it might not comply with the retailers data requirements or he simply can indicate that he has accepted the data and synchronized it with his backend systems.

This type of interaction between data source and data recipient is very unique to GDSN. I am not aware of any other item data exchange standard which has a similar technique.

In a MDM system the typical approach for data integration is on the data source side more a "fire and forget" approach. That means you are maintaining your master data in the MDM system until it is approved and then you are publishing it for the consuming systems. Typically a MDM system does not expect any feedback on its publication. If you are lucky you are using some kind of EAI tools for the data distribution and that is handling at least technical protocols regarding the success of the data distribution. Those protocols are then typically managed by some technical teams.

But the GDSN CIC process is a business process! So the CIC feedback has to be dealt with on the supplier side by the business people who are normally maintaining the item data. So your MDM has to provide some kind of user interface for its users to see CIC's which have been received and to process them efficiently.

On the data recipient side it is a little bit different. There your MDM system has validation rules to ensure the data quality of your item data. If the data you are receiving from a supplier does not comply with those rules your MDM system has to be capable to send back a CIC review message automatically. On top of that your MDM system also has to enable its business users to either accept or send back a review manually because not everything can be validated automatically.

And as those workflows are very unique to GDSN there is no generic solution for it in MDM systems. If you look at MDM systems which have implemented a standard connector to GDSN they all have implemented a separate module (a "GDSN console" or something alike) to deal with those special processes.

It is even getting worse if you also take price synchronisation into account. In GDSN price sync is a different message set and a different choreography from item sync. It is in a way similar (similar message names, similar idea of the choreography) but the details are then quite different. For example is the "Price Sync Confirmation" process not optional like in the item sync process but it is mandatory. Additionally you also first have to synchronise and accept a business relationship before you can start synchronising prices.

Right now I am not aware of any MDM system that has implemented support for the GDSN Price Synchronisation!

And that is doubting the whole value of a GDSN implementation in my view. If I am only able to synchronise item data and no price data my business problem is only solved half. I then still have to find a solution how I can synchronise prices between trading partners. And they really want to get rid of those error prone, not versioned, not processable Excel sheets!

So if you want to connect your MDM to the GDSN (either because you are the software manufacturer of the MDM or because you have introduced a MDM system into your business) be aware of the business processes GDSN implies. They have to be supported.

Also be aware that only the whole GDSN package of Item sync + Price sync really unlocks the whole value of GDSN for you.

Just to be fair I should mention here that price sync adoption today is really low globally - to not say zero if we except Australia. But Australia is a good proof that it can be done and really brings the value expected!