How to implement a Data Quality Management system ...

... for product master data!?

Did you ever think through it? Did you ever try to implement one? Actually I was happy to be part of a team at a large retailer to implement a Data Quality Management system.

In principle it is very simple:

But the problems already start with the very first bubble "Measure the current Data Quality". What is Data Quality? What are the the KPI's (KPI = Key Performance Indicator) to measure Data Quality?

First of all you really have to define what Data Quality means to you. And this really depends on your business processes which use the product data and the requirements those business processes have.

In my case I am pretty sure you could very quickly imagine that the core processes are purchasing and logistics which are the main drivers at the retailer I was working on the Data Quality Management system.

And for gods sake - our team was not the very first one who has to implement a Data Quality Management system for a retailer. We did a little bit research and found very quickly the "Data Quality Framework (DQF)" which is compiled by GS1 to support companies to get ready for GDSN.

The DQF is very helpful to implement a Data Quality Management system for product information if your product information is mainly targeted on supporting business processes in purchasing and logistics. If your requirements go beyond that you have to extend it.

The DQF consists of KPI's, a guide how to implement a Data Quality Management System, and a Self-Assessment procedure.

Most interesting to us were the KPI definitions:

1. Overall item accuracy: Ok, this indicator is not that surprising - it is the percentage of items that have correct attributes values for the attributes in scope.
2. Generic attribute accuracy: That is already more interesting. This is describing the percentage of items that have correct values for some more generic attributes. GS1 defines the following attributes to be in scope for this KPI:
1. GTIN - that is the key for identification of an item therefore it is really key ;-)
2. Classification Category Code - As the classsification code is relevant for reporting it is a very important attribute within retail industry
3. TradeItemDescription - to me this is really a difficult attribute in retail. At all retailers I have been so far, the buyers always insisted that item descriptions are a means to differentiate from competitors and therefore have to be handcrafted at the retailer or have at least to comply with the retailers rules how the description has to be build. Just as a sidenote - I think that is wrong and the item descriptions in no way drive revenue, but I might be wrong here.
   Therefore we decided to leave that attribute out in our reporting.
4. Net Content - is important for shelf tags and therefore one of the really important informations.
3. Dimension and weight accuracy: Depth, Width, Height and Gross Weight are the key attributes here. And those attributes are not only key for distribution centers but also for your transport/route  planning and therefore have very strong and immediate impact on logistics.
4. Hierarchy accuracy: This is absolutly relevant because in different business processes you are using different units of the same item. E.g. you might order an item on palett level but you stores order it on case level or even each level at your distribution center. If you then do not have the packaging hierarchy correct then you are in serious trouble!
5. Active / Orderable: You should not order item units which are not active at your supplier or just not orderable units. This immediately disrupts every automatic, electronic process and therefore has to be avoided.

So with those KPI's you are covering very much all the requirements from business processes in purchasing and logistics.

But the question now is: How to measure accuracy for those attributes?

A retailer has two approaches he can take:

1. Compare the data to the data provided by the supplier.
2. Do a self-assessment and go to your DC and really measure the physical products to gain that information.

In our project we are doing both. We have implemented a system where we are comparing the supplier data to our data according to the above KPI's on an ongoing basis. As the supplier data provided through the data pool does not cover 100% of the business we are also calculating how much of the business is covered by this report.

On top of this we are doing a self assessment. The reason for this is mainly to figure out what quality the supplier data has.

From our experience a Data Quality Management system based on the GS1 Data Quality Framework is  a solid basis manage your MDM program. It gives you the means to document and communicate the progress your MDM program achieves.

---
Update 12.12.2011:

You made it till the end of this post? ;-)
Ok, then I have some more fun stuff for you. I just stumbled over this quite old video from GS1 Netherlands on data quality. But I think it is still to the point and at least it is fun watching and listening:

Why GEPIR sucks ...

Have you ever tried to use GEPIR?

Lately I needed the address of a Distribution Center of one of the major german retailers. As usual I first tried google but there was no address to be found. I found a lot of information on the DC - press releases mentioning it and stuff like that - but no address.

Then I happily remembered GEPIR. GEPIR is a global service from GS1 and GS1 promotes it as "a unique, internet-based service that gives access to basic contact information for companies that are members of GS1. These member companies use GS1's globally unique numbering system to identify their products, physical locations, or shipments." .

And as you know each and every retailers uses GLN's (Global Location Numbers - the GS1 numbering system to identify physical locations) to also identify their Distribution Centers (DC's).

Ok - I quickly went to http://www.gepir.de (you can also use the more international version http://www.gepir.org or any of the other nationally provided URL's, they all access the same data), typed in the name of the retailer - and was first very surprised and then quite disappointed.

There were only 17 hits and none of them was a Distribution Center!?

I quickly checked multiple of the major german retailers, EDEKA, Metro, REWE - none of their Distribution Centers gets listed by GEPIR.

And all of them have more then one thousand GLN's to identify their DC and store locations!

GS1 is selling each and every GLN and they are not able to provide the correct master data to their GLN's!???

How come?

Did I get their vision wrong for GEPIR? Is GEPIR not meant to provide all the master data to the GS1 numbers? If that is not the vision of GEPIR what shall be the value otherwise??

I think the vision of GEPIR is to provide all the master data identified by the GS1 identifiers. Regarding the GLN I expect GEPIR to have all valid GLN's and the physical addresses they refer too. And I would also expect GEPIR to show what type of location that is. Is it a DC? Is it a store? Is it a location used for ordering? Or is it  a location used for invoicing? But that is a second issue with GEPIR.

I think GS1 has a simple master data management issue as many companies do have. I think there is a vision what GEPIR should be (see above), there might be even a strategy how to achieve that vision (implement a globally shared database) but they did the typical mistakes you can see in many companies regarding master data management.

They did not establish any metrics. For example they should measure what the coverage of the GLN's within GEPIR compared to the sold GLN's is. Then they would know that the coverage in the best case is only 50-60% and that they would have to take some measures to improve the situation.

They did not establish any sustainable organisation and processes. Who is responsible to get new GLN's into GEPIR? How about updates of addresses?

And finally their IT infrastructure does not seem to be really tailored to best support their vision. I even heard that couple times GLN's from a single, local GS1 organisation was sold two times. What a mess. And what type of IT infrastructure is not able to ensure the uniqueness of such a simple number like the GLN?

So finally GEPIR currently ends up as many MDM and PIM projects. There is a great tool. Even with an iPhone app. But it really sucks because the master data is either not available or the quality is so poor that you do not want to rely on it.

Care for the content! Care for your master data! Care for your master data quality!

Ok GS1, I really would recommend that you start a Master Data Management Program as you are expecting from your members to do data synchronisation. A possible point to start with you can find here: What is a MDM program?

GEPIR is really out of sync right now!


----
Update 03.12.2011:
There are some really interesting comments to this post on linkedin, which you can find here.

Why is GDSN ignored by PIM vendors?

Lately I had a very intensive discussion  on GDSN with one of the major PIM software producer. They were really interested to see whether GDSN could bring them any additional business and if there were relevant customers being in the need of GDSN support in a PIM solution.

They got very detailed presentations on GS1 activities in the Retail & CPG and in the Healthcare space. And they were also presented the actual GDSN growth figures and actually it was tried everything to sell them the relevance of GDSN for their PIM business. They even talked to major retailers using GDSN on their experience and advice and also to some data pools.

Finally they took some time to evaluate what was presented to them and draw some conclusions for their business. Actually I was quite surprised with their key conclusions:

1. Datapools only have relevance in Europe and in US. This really puzzled me because at least Australia is one of the most advanced GDSN communities with the highest adoption rates. But looking at most of the other countries - even within Europe - GDSN is not that much adopted and still manual data exchange (even on paper) is the most spread way of data synchronisation. Although the adoption rate is continuously growing as you can see from the Global Registry Statistics.
   
   
2. The data is not maintained sufficiently and the data quality is considered to be very low.
   In this case this is really a two-sided issue. First I think bad data quality in GDSN data pools is really one of the badest myths at all. In all cases where I did comparisons between retailer product data and the data in a data pool the retailer data was much worse then the data in the data pool. The data in the data pool actually always was in quite a good shape. This is also supported by all recent studies like the GS1 UK datacrunch report or the report from GS1 Belgium.
   The second side of the issue in this case is that their customers are mainly also in the ecommerce space and what you need there is the marketing and sales information on products and this is today really not available in the GDSN.
   It really seems to be a challenge within the GSMP process to get a standard developed for marketing and sales relevant (or let's use the new hype term "B2C") data. If you look for example at Consumer Electronics. The first hints on additional attributes and an additional standard I have found in 2008. And now in 2011 we are getting calls to action to work on the requirements for Consumer Electronics ... For me that does not look convincing too ...
   
   
3. Data enrichment is not covered within the GDSN
   What they meant by "data enrichement" is to add B2C data to the data if that is not available directly from any internal source at the supplier.
   I think here they are going wrong. In my perception the PIM of a retailer should offer a supplier portal where suppliers can enrich their data manually. Also data pools like SA2 Worldsync are starting to offer such functionalities based on their data pools.
   As GDSN is "only" a protocol between data pools (and in some respect between a data source and a data recipient and its corresponding data pool) it does not deal with "manual" data enrichment processes by concept.
   
   
4. They consider GDSN only as  a infrastructure side topic
   Here I think they are really making a big mistake. GDSN is not only about publishing the data to  a data recipient (retailer) but it is also about communication with the retailer (think of the CIC messages which allow retailers to send feedback back to suppliers).
   In my point of view every serious PIM system has to support the full GDSN choreography. And this also means to have specific UI's for the corresponding user roles and being aware that there is a new type of user role which has also to be established in the customers organisation.
   
   
5. As they consider retail as one of their major markets, they also consider supplier data onboarding for retailers as one of their key tasks - but they do not consider GDSN to be one of their main means to do so
   As their customers are very much in the ecommerce space and they are dealing a lot with sales and marketing oriented product information this decisions seems to be very much linked to the above topic 2 and 3 and that this type of data is just not available in the GDSN.
   Although I think that onboarding of supplier data is always following the same principle. It is independant whether I need supplier logistics data or some feature information. Only the content differs.

What is my take away?

GDSN seems to be struggeling from my point of view with two challenges:

1. Market perception: The market does not perceive GDSN as a major, global and successful way to synchronize product information. And this despite the huge community behind GS1 Standards and GDSN.
   I think that is really a misperception mainly because of the huge GS1 community. I am not aware of any comparable standardization organisation which is so user driven and has such a huge supporting community. But here also the second challenge comes into the game.
   
2. GDSN adoption and perception by the implementing companies: Although there is such a huge supporting community the real success stories where the implementing companies really have achieved the promised savings are still rare. And this is not mainly because GDSN is the wrong approach but because companies either have not implemented it at all or have not implemented it properly as part of a MDM program.

In one of my next postings I will discuss some alternatives to GDSN and how a combined approach of GDSN with some alternatives might help user companies to achieve their goals and thereby also help GDSN to improve its market perception.

Have you ever thought through how MDM differs in a multinational company from MDM in a only nationally focused company?

Do you remember my blog post how to organize for MDM? There I described how you should organize your MDM activities in principle. If you are working for a big multinational company you might have thought that this is by far to simple to solve your challenges. 

And you are right!

In a multinational company you have on top of the generic MDM specific tasks additional challenges which you have to address within your organization, your processes and your IT infrastructure (esp. in your metadata model).

Even if you start looking in the data of multinational products you can find very serious challenges for MDM.

Global, regional and local products: In an multinational environment you typically find global products which are sold and relevant to all regions or countries. But you also find products which are region specific (eg. only for the European or Asian market) and also very local products which might be available and target only on one country. Typically you want your local organizations to be able to list and sell the local products while the global products are more or less defined by a central business unit.

Global, regional and local elements in global products: If you have global products which are available in different regional or even local markets you typically have some product information which is really only targeted on local requirements. One good example for those types of requirements is the requirement for the “green dot” information in germany (which by the way became obsolete again due to changed legislations). This information was only relevant if you wanted to sell a product to germany. But even if it was the same product globally, for germany you specifically had to add the information whether the product had the “green dot” certificate or not.

Multi-branded products: Multiple brandings can occure for the same product already from the industry side, but it is even more common in retail where you have two retail brands owned by one company and therefore sharing a big part of their assortment. But when presenting to the market they want to describe the same product typically in different ways to the market to clearly differentiate their products from each other.

How does all of this impact your organization and processes?

If you are a multinational company you will have a strategy which will also cover the parameters “need for integration” and “need for local responsibility”. These two parameters define quite well your internationalization strategy. The same you have to apply to your master data management.

In the end you have to setup a distributed organization for your MDM. A quick shot might look like this:

Leaving SA2 Worldsync …

I am leaving SA2 Worldsync. I have worked now for over 6 years in the area of master data management and global data synchronization with SA2 Worldsync and its predecessors.

And I think it was a great time and esp. a great team to work with. And I think we achieved a lot. SA2 Worldsync has become one of the globally dominant data pool players and also one of the leading PIM vendors at least in the retail market.

Thinking back I started all this master data management stuff right back in 1997 when founding Cataloom AG. There we developed successfully one of the first PIM (Product Information Management) / MDM (Master Data Management) Systems. 

And I still remember very vividly myself coding the first database schemas and graphical user interfaces (still very awkwardly looking) for cataloom 0.1.

We were very quickly successfully fulfilling the requirements of our customers like AXA, BASF, Deutsche Bahn, Emaro (the famous eprocurement marketplace joint venture of SAP and Deutsche Bank), Siemens Medical and many others. It was a real fun time, working with a small but enthusiastic team, having all this great ideas and concepts which are still in the product and which are still the base for its success.

In 2005 PIRONET NDH has taken over Cataloom. My personal journey into the world of GS1 and data synchronization started when PIRONET NDH also took over a majority stake in SINFOS the predecessor of SA2 Worldsync. 

Replacing the proprietary SINFOS technology with my previous Cataloom technology and building this great Webforms with online validations was one of my first responsibilities during that time.

Merging with the Agentrics data pool business unit was the next milestone. And this merger was mainly driven by our superior technology. Esp. I think the usability of our Webforms is still leading edge technology and the team keeps developing and improving it.

After that merger I became responsible for looking into our retail customers understanding their master data management processes and to help them to get those processes organized to be able to also leverage the SA2 Worldsync services better.

I think in the last two to three years I worked with most of the leading retailers globally (from Europe over Japan to the US) to understand their challenges in master data management and to work with them on improving those processes, their organization and sometimes also their IT infrastructure.

Right now SA2 Worldsync is not only supporting world’s largest retailers and brand manufacturers but also some of the most advanced GS1 and GDSN communities like GS1 UK and GS1 Australia by providing our technology to them. And not to forget to mention that the SA2 Worldsync WS|PIM product today is run by some of the largest retailers – and right now there are more to come.

I think SA2 Worldsync has been settled right now and achieved a really leading position in the master data management and global data synchronization business.

That is the time to leave for me.

Why?

I am the more entrepreneurial kind of guy. I have to start up something new. Change the world :-) 

I think Cataloom and SA2 Worldsync had some impact. At least the world changed a little bit. Product Information and data synchronization processes became a little bit more electronically. But I think there is still a long way to go to have all business transactions electronically.

That is where I still see a huge potential! So be prepared to see something new coming up which hopefully will really have some impact!

And in the meantime – if you have some requirements regarding master data management consulting, just let me know. I am helping companies globally to put in organisation and processes and IT infrastructure to manage their master data more efficiently.

PS: Stay tuned - this blog will be continued and now I might even have more time to share my thoughts on MDM and GDSN!

How to organise for MDM?

Have you ever thought through how to build an organisation for master data management?

One of the key issues I always find at customers who are struggeling with their quality of master data is that their organisation is not really prepared to deal with master data in a sustainable manner.

In my view Gartner has done quite a good job in proposing a generic organisation for master data management. Actually this organisation implies that you are introducing a new business process - Master Data Management.

Please look at the following orgchart - this is how I remember the generic organisation Gartner is proposing:

The generic MDM organisation

One of the key ideas is that business has to lead the MDM organisation. A quick description of the different roles & responsibilities:

1. The Information Governance Board should consist of executive level sponsors. They should  set and enforce information management policies. In this board you should have representatives from the business - that is key - but you also should have a representation from your IT executives in here.
2. The MDM Team is responsible to manage the MDM program and who authors and maintains the master data. If you are also implementing GDSN or any other means to collect item data from your suppliers the supplier onboarding and communication is also a key task for this team.
3. Data Stewards sit in the business unit and they are responsible for data quality monitoring, improvement and issue resolution. They are NOT maintaining the data themselves but work closely with the MDM Team to get the master data quality on to the level the business needs it.
4. The MDM Infrastructure Team is a kind of virtual team responsible for all aspects of the IT implementation needed for the MDM business process.
5. In the GDSN context I only want to highlight the importance of the Data Modeling and Information Architecture role. If participating in the GDSN this role is key because it is the link to the GSMP process and defining change requests and also anticipating and adopting the changes coming through the GSMP / GDSN.

Please take this organisation template as what it is - only a template. In real life you have to look at your current organisation and see how you can adopt the different roles and responsibilities within your organisation.

But also be aware - you want to introduce a new business process which you did not have in the past.

And if your organisation is multinational or even global I will discuss in one of my next posts what impact that has on the MDM organisation.

Don't brands want to communicate transparently to the consumer?

Do you remember one of my earlier postings Brand owners loose sales due to mobile scanning?

Actually my main message was that brands have to invest into MDM to be able to publish their product information in a high quality way to all those app providers to have control over the branded product information publically available.

I thought that it is mainly a technical and effort issue!

But I had to learn that there are also other issues.

Last couple of days there was the SA2 Worldsync's Annual User Congress 2011where we promoted a lot the new mobile century. We had great presentations from the mobile space (eg. from Mirasense and from barcoo) which made it clear that there is a massive demand for extensive product information from the consumer. And those app providers are fulfilling this demand - either with or without the support from the brand owners.

We got quite some good feedback from some major brand manufacturers for our activities to deliver trusted, manufacturer product information to those platforms. See one of our press releases here.

But we also got quite some negative feedback from some other major brand manufacturers.

Quote: "That is not in our interest."

It is not in your interest to communicate with your customers?
It is not in your interest to provide correct and honest information to your customers?
It is not in your interest to use the new media?

One argument I heard was "We cannot give allergene information to the consumer via such platforms because then we could be held liable for it. And we could be sued if the data is incorrect and thereby somebody gets harmed.".

But you are putting allergene information on the packaging and on your website.

Are your processes so much more reliable for putting information on the packaging or on your website then publishing it to the GDSN?

Or are there other reasons behind your reluctancy to provide transparent information to the consumer?

Actually I haven't got it yet. I would be happy to get some feedback on that topic ...