Skip to Content

Changes for page 2 What is SDMX

Last modified by Yuri on 2025/10/22 20:54
From version 2.1
edited by Helena
on 2025/05/23 16:55
Change comment: There is no comment for this version
To version 5.1
edited by Elena
on 2025/06/16 15:29
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -xwiki:XWiki.helena
1 +xwiki:XWiki.elenasemanticproorg
Tags
... ... @@ -1,0 +1,1 @@
1 +Category|Code list|Component|Concept|Content-Oriented Guidelines|Data set|Global registry|Metadata repository|Reference metadata|SDMX Information Model|SDMX Registry|SDMX Technical Specification|SDMX-EDI|Statistical data and metadata exchange|Statistical subject-matter domain
Content
... ... @@ -6,11 +6,11 @@
6 6  
7 7  This chapter provides some background on the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Initiative, the issues which [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] addresses, the history of the standards and guidelines which have come out of this initiative, and the areas in which [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is playing a role today. This chapter also provides some guidance about how prospective users should think about [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] as it relates to their own part of the statistical process.
8 8  
9 -[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] offers a wide variety of technical tools, and these – like any tools – if used well, they produce positive results.. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] also offers a set of guidelines regarding the application of harmonized statistical (% style="color:#e74c3c" %)concepts(%%) to [[data sets>>doc:sdmx:Glossary.Data set.WebHome]], and how these can be represented. Other guidelines address the classification of statistical data and domains, and the harmonization of relevant terminology. Additionally, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] represents a framework for the process of harmonization within domains. All of these different aspects are considered here.
9 +[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] offers a wide variety of technical tools, and these – like any tools – if used well, they produce positive results. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] also offers a set of guidelines regarding the application of harmonized statistical[[ concepts>>doc:Glossary.Concept.WebHome]] to [[data sets>>doc:sdmx:Glossary.Data set.WebHome]], and how these can be represented. Other guidelines address the classification of statistical data and domains, and the harmonization of relevant terminology. Additionally, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] represents a framework for the process of harmonization within domains. All of these different aspects are considered here.
10 10  
11 11  == 2.2 Background – Official Statistics ==
12 12  
13 -[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] comes out of the world of official statistics. If you work for a national or international statistical agency, you already understand “official statistics”, but many people who work with statistical data may not understand this domain, so we will provide a brief, high-(% style="color:#e74c3c" %)level(%%) description.
13 +[[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] comes out of the world of official statistics. If you work for a national or international statistical agency, you already understand “official statistics”, but many people who work with statistical data may not understand this domain, so we will provide a brief, high-level description.
14 14  
15 15  “Official statistics” are the data which is collected and disseminated by a set of governmental and international organizations to provide the factual basis for making policy and supporting research. Some countries have a “national statistical office” (NSO) while others may have several governmental organizations which are charged with collecting statistical data for governmental use. Most countries also have central banks or similar organizations which collect and disseminate financial and economic data.
16 16  
... ... @@ -18,7 +18,7 @@
18 18  
19 19  These national organizations typically report their statistics to a set of supra-national organizations, representing either regions of the globe (examples include Eurostat and the European Central Bank) or domains (examples include the World Health Organization, the Food and Agriculture Organization, UNESCO, and the World Bank). Many of these organizations belong to the UN, or are treaty organizations.
20 20  
21 -All of these organizations exchange, report, and disseminate data in a chain which can be understood as starting at the lowest (% style="color:#e74c3c" %)level(%%) within each country, and resulting in highlevel [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] which are “aggregated” as they move through various (% style="color:#e74c3c" %)levels(%%) to reach the international (% style="color:#e74c3c" %)level(%%).
21 +All of these organizations exchange, report, and disseminate data in a chain which can be understood as starting at the lowest level within each country, and resulting in highlevel [[data sets>>doc:sdmx:Glossary.Data set.WebHome]] which are “aggregated” as they move through various levels to reach the international level.
22 22  
23 23  The system of official statistics is this network of reported data, according to legal requirements or other types of agreements. There are several important meetings, conferences, and initiatives within this system, so that all organizations adopt similar approaches and techniques, and to coordinate reporting: the Conference of European Statisticians is an important meeting, as is the United Nations Statistical Commission meeting. Ultimately the goal is to measure important phenomenon occurring in the world, and to report the data to policy makers, students, journalists, and other users to help inform their activity. The data is “official” because it comes with the reputation of the world’s governments and international institutions behind it.
24 24  
... ... @@ -49,7 +49,7 @@
49 49  Thus, the goals of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] initiative were ones which were broadly agreed across the sponsoring organizations, and within the official statistics community generally. It is important to understand that there were some firm foundations on which [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] was building:
50 50  
51 51  1. An existing standard for exchanging statistical data, known as GESMES/TS, was already in use among several of the sponsor organizations and their nationallevel counterparties. This was based not on modern Web technologies such as XML, but used the older UN/EDIFACT syntax.
52 -1. The work on the “metadata common vocabulary” was based on many years of harmonization work within the community, notably Eurostat’s (% style="color:#e74c3c" %)//Concept//(%%)// and Definitions Database (CODED)// and the //OECD Glossary of Terms//.
52 +1. The work on the “metadata common vocabulary” was based on many years of harmonization work within the community, notably Eurostat’s (% style="color:#e74c3c" %)//Concept//(%%)// and Definitions Database (CODED)// and the //OECD Glossary of Terms//.
53 53  
54 54  The formation of the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Initiative can be understood as a recognition by the sponsor organizations that working together to address these issues, and that coordinating business approaches using modern, standards-based technology, was the best way forward. In one sense, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] evolved from earlier work, but indicated the increased commitment the sponsors had toward reaching its goals. It also represents a comingtogether of efforts around harmonizing statistical content and terminology, and for deploying technology to support statistical processes.
55 55  
... ... @@ -67,19 +67,19 @@
67 67  
68 68  Within the content-oriented work, there are a set of work products: //The Content Oriented Guidelines, //and 5 annexes:
69 69  
70 -1. Cross-Domain (% style="color:#e74c3c" %)Concepts
71 -1. Cross-Domain Codelists
70 +1. Cross-Domain [[Concepts>>doc:Glossary.Concept.WebHome]]
71 +1. Cross-Domain [[Codelists>>doc:Glossary.Code list.WebHome]]
72 72  1. Statistical Subject-Matter Domains
73 73  1. Metadata Common Vocabulary
74 -1. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-ML for the [[Content-Oriented Guidelines>>doc:sdmx:Glossary.Content-Oriented Guidelines.WebHome]] ((% style="color:#e74c3c" %)Concepts(%%), [[Code Lists>>doc:sdmx:Glossary.Code list.WebHome]], [[Category>>doc:sdmx:Glossary.Category.WebHome]] Scheme)
74 +1. [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-ML for the [[Content-Oriented Guidelines>>doc:sdmx:Glossary.Content-Oriented Guidelines.WebHome]] ([[Concepts>>doc:Glossary.Concept.WebHome]], [[Code Lists>>doc:sdmx:Glossary.Code list.WebHome]], [[Category>>doc:sdmx:Glossary.Category.WebHome]] Scheme)
75 75  
76 76  These are discussed in more detail in the first annex to this user guide.
77 77  
78 -The [[SDMX Technical Specifications>>doc:sdmx:Glossary.SDMX Technical Specification.WebHome]] are now in (% style="color:#e74c3c" %)version(%%) 2.1, but both (% style="color:#e74c3c" %)version(%%) 1.0 and (% style="color:#e74c3c" %)version(%%) 2.0 were implemented. The 1.0 (% style="color:#e74c3c" %)version(%%) of the specifications have relatively limited coverage – a model for data formats and their structures, along with XML and UN/EDIFACT formats for exchanging these. The UN/EDIFACT format was backwardcompatible with GESMES/TS; the XML formats were new. There was also some support provided for [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-based Web services: an XML query document, and a set of guidelines about the use of other related Web-services standards (SOAP and WSDL).
78 +The [[SDMX Technical Specifications>>doc:sdmx:Glossary.SDMX Technical Specification.WebHome]] are now in version 2.1, but both version 1.0 and version 2.0 were implemented. The 1.0 version of the specifications have relatively limited coverage – a model for data formats and their structures, along with XML and UN/EDIFACT formats for exchanging these. The UN/EDIFACT format was backwardcompatible with GESMES/TS; the XML formats were new. There was also some support provided for [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]]-based Web services: an XML query document, and a set of guidelines about the use of other related Web-services standards (SOAP and WSDL).
79 79  
80 -The 2.0 (% style="color:#e74c3c" %)version(%%) of the technical specifications had a greatly-expanded scope. The model was extended to include “[[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]]” as a way of structuring and formatting metadata related to data quality frameworks, methodological metadata, and other types of “footnote” metadata. Thus, XML formats for [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] were added. Further, a set of standard interfaces in XML for interactions with a [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] were added, for cataloguing the location of data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] across the Internet or within an organization, and for maintaining and retrieving structural metadata.
80 +The 2.0 version of the technical specifications had a greatly-expanded scope. The model was extended to include “[[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]]” as a way of structuring and formatting metadata related to data quality frameworks, methodological metadata, and other types of “footnote” metadata. Thus, XML formats for [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] were added. Further, a set of standard interfaces in XML for interactions with a [[SDMX Registry>>doc:sdmx:Glossary.SDMX Registry.WebHome]] were added, for cataloguing the location of data and [[reference metadata>>doc:sdmx:Glossary.Reference metadata.WebHome]] across the Internet or within an organization, and for maintaining and retrieving structural metadata.
81 81  
82 -In (% style="color:#e74c3c" %)version(%%) 2.1, many features of 2.0 have been improved, and the Web-services recommendations have been expanded to include a RESTful interface, standard functions, and error messages. Now, it is possible to develop generically interoperable applications based on the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] standards. Further, the various XML data formats have been simplified based on implementation experience with (% style="color:#e74c3c" %)version(%%) 2.0.
82 +In version 2.1, many features of 2.0 have been improved, and the Web-services recommendations have been expanded to include a RESTful interface, standard functions, and error messages. Now, it is possible to develop generically interoperable applications based on the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] standards. Further, the various XML data formats have been simplified based on implementation experience with version 2.0.
83 83  
84 84  For all types of work products, there have been internal reviews within the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] community, and also public review of the guidelines and standards.
85 85  
... ... @@ -138,6 +138,6 @@
138 138  
139 139  To learn more about available [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] tools, the best place is to consult the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Tools Database, a service provided by the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] sponsors, linked to from www.[[sdmx>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]].org.
140 140  
141 -And support for the standards does not only take the form of tools – The Open Data Foundation hosts the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] User Forum in collaboration with the sponsors, providing a place where the community can interact online, and Eurostat’s CIRCA website provides many types of resources, from training videos to student guides. Many organizations offer [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] in-person training for different (% style="color:#e74c3c" %)levels(%%) of users. The best single point of entry is of course the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] website itself.
141 +And support for the standards does not only take the form of tools – The Open Data Foundation hosts the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] User Forum in collaboration with the sponsors, providing a place where the community can interact online, and Eurostat’s CIRCA website provides many types of resources, from training videos to student guides. Many organizations offer [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] in-person training for different levels of users. The best single point of entry is of course the [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] website itself.
142 142  
143 143  Looking forward, [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] is increasingly coming into use: at the most recent [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] Global conference held in Washington DC in May 2011, Google showed some interest in [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] as a source of data for its Data Explorer; there is now an interest in setting up a [[global registry>>doc:sdmx:Glossary.Global registry.WebHome]] so that all [[SDMX>>doc:sdmx:Glossary.Statistical data and metadata exchange.WebHome]] data and metadata sources can be easily found. Further, we see the strong possibility that the world of corporate statistics may realize the utility of having a strong standards basis around the vast amounts of data collected today to support business intelligence applications.