SDMX

What is SDMX?

SDMX provides a metamodel for describing data in any statistical domain. The origins of the SDMX Information Model can be traced directly back to the model for the Generic Statistical Message (GESMES) which is a UN/EDIFACT standard, but in reality the core of the model is an evolution over time dating back to the 1970's. Following the launch of SDMX in 2001 this model has been enhanced significantly and is implemented fully in XML (SDMX-ML) and specific parts of it in UN/EDIFACT (a sub set of the GESMES message supporting time series), and JSON (to support data dissemination over the web).

The SDMX initiative is sponsored by seven organisations:

Bank for International Settlements (BIS); European Central Bank (ECB); Eurostat; International Monetary Fund (IMF); Organisation for Economic Co-operation and Development (OECD); United Nations Statistics Division (UNSD); World Bank.

SDMX supports many statistical activities and the processes supporting these activities:

• data collection – data registration and data retrieval, data validation
• data reporting and data mapping
• data dissemination – data discovery, data query, data portal
• structural metadata repository for metadata management, persistence, query, and retrieval
• reference metadata reporting and dissemination, and linking metadata to data points

Why Use SDMX?

Support for Multiple Use Cases

First, it is important to know that SDMX can deliver more than just a common format for data collectors and data reporters to use for data and metadata exchange, even though the acronym would suggest otherwise.

Second, don’t worry about the syntax representation of the data and metadata, there are plenty of tools and open source code that hide this complexity and thus enable you to use SDMX to solve your problems. Such as how to consume SDMX in your favourite statistical tool such as SAS or R, or Excel, or how to create a dissemination web site, or how to build a robust data collection system.

And finally, look at the SDMX Information Model and see how this can support your use cases. When you follow this model (we explain this model later) we are convinced that you will see that it meets many your needs. Then with the tools and open source you will find you can implement your systems far quicker than you would have imagined.

Established Model

SDMX is, at its heart, an Information Model implemented in specific syntaxes (mainly XML). SDMX is responsive to new technologies as new syntax representations can be constructed easily as the base of these is the Information Model. For example, a new JSON format has been developed and this is gaining popularity with web developers.

We have implemented data collection, data validation, and data dissemination systems for a number clients. We have been building SDMX tools since 2005, and we have developed and released the SdmxSource (open source). Like many other organisations we have discovered that it is not the syntax, but the Information Model that is the power behind SDMX. We are pleased to share this knowledge in the explanations that follow to show how systems with the “SDMX Inside” can enable you to build solutions with less cost , less resources both initially and on-ongoing, and in less time than would otherwise be the case.

The Information Model is syntax and format agnostic and consequently the majority of processes and functions can be developed around the model, and not around the syntaxes. This is the essence of SDMX Source (www.sdmxsource.org) which comprises an open source toolbox for developing SDMX-based applications. Many useful tools have been developed to process SDMX (reading, writing, validation, transformation, mapping) and these can be mixed and matched in systems regardless of the author of the component in the knowledge that they all obey the same API.

SDMX Source is developed by Metadata Technology and is used by many big institutions, including Eurostat who has adopted SDMX Source as the underlying framework for its tools and applications. Building an application on this framework ensures full SDMX compliance for both import and export of information.

ISO Standard

SDMX is here to stay. The sponsors are international organisations (BIS, ECB, Eurostat,IMF, OECD, UN, World Bank) and the SDMX standard has ISO status (international standard 17369). SDMX is maintained actively, responding to requests for new functions through an open process.

Return on Investment

It is often said that it is difficult to justify the investment in a specific standard if it is to be used to do only one thing. SDMX can do far more than just act as a common format for exchanging statistical data and metadata. If you use SDMX as the model on which your data collection, data reporting, and data dissemination systems are built, then the benefits will roll in. The more you use the power of SDMX in your systems, the more benefit you will gain.