Skip to Content
Last modified by Helena K. on 2026/06/08 15:16
From version 36.11
edited by Helena K.
on 2026/06/08 13:19
Change comment: There is no comment for this version
To version 35.7
edited by Helena K.
on 2026/06/08 12:14
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -18,51 +18,51 @@
18 18  
19 19  **Section 1** **Framework for [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Technical Standards** – this document providing an introduction to the technical standards.
20 20  
21 -**Section 2** **SDMX Information Model** - the [[SDMX information model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]] is a standardised object model for modelling [[statistical domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]] centring on the structure of their data and [[metadata sets>>doc:sdmx:Glossary 2\.1.Metadata set.WebHome]], the coding schemes used for [[classification>>doc:sdmx:Glossary 2\.1.Statistical Classification.WebHome]], and the rules for controlling the exchange of data and metadata between organisations. This document provides a UML specification with supporting narrative.
21 +**Section 2** **[[SDMX Information Model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]]** - the [[SDMX information model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]] is a standardised object model for modelling [[statistical domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]] centring on the structure of their data and [[metadata sets>>doc:sdmx:Glossary 2\.1.Metadata set.WebHome]], the coding schemes used for [[classification>>doc:sdmx:Glossary 2\.1.Statistical Classification.WebHome]], and the rules for controlling the exchange of data and metadata between organisations. This document provides a UML specification with supporting narrative.
22 22  
23 -**Section 5** **SDMX Registry Specification** – an [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] ‘registry’ acts as a repository for [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] and provisioning information, and a registry of data and metadata sources. This document sets out the specification.
23 +**Section 5** **[[SDMX Registry>>doc:sdmx:Glossary 2\.1.SDMX Registry.WebHome]] Specification** – an [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] ‘registry’ acts as a repository for [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] and provisioning information, and a registry of data and metadata sources. This document sets out the specification.
24 24  
25 -**Section 6** **SDMX Technical Notes** – detailed technical guidance for implementors of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] standard.
25 +**Section 6** **[[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Technical Notes** – detailed technical guidance for implementors of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] standard.
26 26  
27 -The following are published on the GitHub repository of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Standards Technical Working Group ([[__https:~~/~~/github.com/sdmx-twg__>>https://https:github.comsdmx-twg||rel="noopener noreferrer" target="_blank"]]): sdmx-twg/sdmx-rest – REST API
27 +The following are published on the GitHub repository of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Standards Technical Working Group ([[__https:~~/~~/github.com/sdmx-twg__>>https://https:github.comsdmx-twg||rel="noopener noreferrer" target="_blank"]]): [[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-rest – REST API
28 28  
29 29  Technical specifications for the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] RESTful web services application programming interfaces (API).
30 30  
31 31  (% class="wikigeneratedid" id="Hsdmx-twg2Fsdmx-ml2013SDMX-ML" %)
32 -**sdmx-twg/sdmx-ml – SDMX-ML**
32 +**[[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx-ml>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] – [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]]**
33 33  Technical specifications for the XML transmission format including XSD schemas, documentation and samples for data, structure and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] messages.
34 34  
35 35  (% class="wikigeneratedid" id="Hsdmx-twg2Fsdmx-json2013SDMX-JSON" %)
36 -**sdmx-twg/sdmx-json – SDMX-JSON**
36 +**[[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx-json>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] – [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]]**
37 37  Technical specifications for the JSON transmission format including documentation, schemas and samples for data, structure and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] messages.
38 38  
39 39  (% class="wikigeneratedid" id="Hsdmx-twg2Fsdmx-csv2013SDMX-CSV" %)
40 -**sdmx-twg/sdmx-csv – SDMX-CSV**
40 +**[[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx-csv>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] – [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]]**
41 41  Technical specifications for the [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] transmission format for ‘comma-separated values’ (CSV) data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]].
42 42  
43 43  The following sections are obsolete:
44 44  
45 -Section 3 - [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] - replaced by the sdmx-twg/sdmx-ml GitHub repository
45 +Section 3 - [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] - replaced by the [[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx-ml>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] GitHub repository
46 46  
47 47  Section 4 - [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]]
48 48  
49 -Section 7 - API - replaced by the sdmx-twg/sdmx-rest GitHub repository [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]]
49 +Section 7 - API - replaced by the [[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-twg/[[sdmx>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-rest GitHub repository [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]]
50 50  
51 51  In July 2020 the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 specifications were revised to add support for the [[Validation and Transformation Language>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] ([[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]]). For 3.0, the [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] specification has been updated to align with changes to the information model and other modifications to the Standard such as the introduction of Semantic Versioning for the versioning of [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]]. Section 2 (Information Model) sets out details of the ‘Transformation and Expressions’ package for defining and managing [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 2.0 programs and Section 6 (Technical Notes) provides detailed guidance on implementing and using [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] with [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]].
52 52  
53 53  = 2 Change History =
54 54  
55 -The 2.0 version of this standard represented a significant increase in scope, and also provided more complete support in those areas covered in the version 1.0 specification. Version 2.0 of this standard is backward-compatible with version 1.0, so that existing implementations can be easily migrated to conformance with version 2.0.
55 +The 2.0 (% style="color:#e74c3c" %)version(%%) of this standard represented a significant increase in scope, and also provided more complete support in those areas covered in the (% style="color:#e74c3c" %)version(%%) 1.0 specification. (% style="color:#e74c3c" %)Version(%%) 2.0 of this standard is backward-compatible with (% style="color:#e74c3c" %)version(%%) 1.0, so that existing implementations can be easily migrated to conformance with (% style="color:#e74c3c" %)version(%%) 2.0.
56 56  
57 -The 2.1 version of this standard represents a set of changes resulting from several years of implementation experience with the 2.0 standard. The changes do not represent a major increase in scope or functionality, but do correct some bugs, and add functionalities in some cases. Major changes in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] include a much stronger alignment of the XML Schemas with the Information Model, to emphasize inheritance and object-oriented features, and increased precision and flexibility in the attachment of metadata reports to specific objects in the [[SDMX Information Model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]].
57 +The 2.1 (% style="color:#e74c3c" %)version(%%) of this standard represents a set of changes resulting from several years of implementation experience with the 2.0 standard. The changes do not represent a major increase in scope or functionality, but do correct some bugs, and add functionalities in some cases. Major changes in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] include a much stronger alignment of the XML Schemas with the Information Model, to emphasize inheritance and object-oriented features, and increased precision and flexibility in the attachment of metadata reports to specific objects in the [[SDMX Information Model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]].
58 58  
59 -The 3.0 version incorporates new features, improvements and changes arising from the collective knowledge gained from a decade of operating experience with the 2.1 standard. In pursuit of modernisation and simplification, features considered obsolete have been deprecated – in particular the EDI transmission format, the lesser-used XML data messages and the SOAP web services API. Many areas remain backwardly compatible with 2.1, but there are some breaking changes where the information model has been redesigned to better support practical use case. Structure mapping and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] are examples. The opportunity has been taken to revise the RESTful web services API which is also not backwardly compatible, but benefits from a rationalisation and better organisations of resources, and a much richer data query URL syntax.
59 +The 3.0 (% style="color:#e74c3c" %)version(%%) incorporates new features, improvements and changes arising from the collective knowledge gained from a decade of operating experience with the 2.1 standard. In pursuit of modernisation and simplification, features considered obsolete have been deprecated – in particular the EDI transmission format, the lesser-used XML data messages and the SOAP web services API. Many areas remain backwardly compatible with 2.1, but there are some breaking changes where the information model has been redesigned to better support practical use case. Structure mapping and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] are examples. The opportunity has been taken to revise the RESTful web services API which is also not backwardly compatible, but benefits from a rationalisation and better organisations of resources, and a much richer data query URL syntax.
60 60  
61 -The 3.1 version provides supports for data models to increase dimensionality over time without impacting existing data collections. The Data Constraint model was adjusted to separate concerns of data reporting and data dissemination.
61 +The 3.1 (% style="color:#e74c3c" %)version(%%) provides supports for data models to increase dimensionality over time without impacting existing data collections. The Data (% style="color:#e74c3c" %)Constraint(%%) model was adjusted to separate concerns of data reporting and data dissemination.
62 62  
63 63  == 2.1 Major Changes from 1.0 to 2.0 ==
64 64  
65 -* **[[Reference Metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]]**: In addition to describing and specifying data structures and formats (along with related [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]), the version 2.0 specification also provides for the exchange of metadata which is distinct from the [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] in the 1.0 version. This [[category>>doc:sdmx:Glossary 2\.1.Category.WebHome]] includes “reference” metadata (regarding data quality, methodology, and similar types – it can be configured by the user to include whatever concepts require reporting); metadata related to data provisioning ([[release calendar>>doc:sdmx:Glossary 2\.1.Release policy - release calendar.WebHome]] information, description of the data and metadata provided, etc.); and metadata relevant to the exchange of categorization schemes.
65 +* **[[Reference Metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]]**: In addition to describing and specifying data structures and formats (along with related [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]), the (% style="color:#e74c3c" %)version(%%) 2.0 specification also provides for the exchange of metadata which is distinct from the [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] in the 1.0 (% style="color:#e74c3c" %)version(%%). This [[category>>doc:sdmx:Glossary 2\.1.Category.WebHome]] includes “reference” metadata (regarding data quality, methodology, and similar types – it can be configured by the user to include whatever (% style="color:#e74c3c" %)concepts(%%) require reporting); metadata related to data provisioning ([[release calendar>>doc:sdmx:Glossary 2\.1.Release policy - release calendar.WebHome]] information, description of the data and metadata provided, etc.); and metadata relevant to the exchange of categorization schemes.
66 66  * **[[SDMX Registry>>doc:sdmx:Glossary 2\.1.SDMX Registry.WebHome]]**: Provision is made in the 2.0 standard for standard communication with registry services, to support a data-sharing model of statistical exchange. These services include registration of data and metadata, querying of registered data and metadata, and [[subscription>>doc:sdmx:Glossary 2\.1.Subscription.WebHome]]/[[notification>>doc:sdmx:Glossary 2\.1.Notification.WebHome]].
67 67  ** **[[Structural Metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]**: The support for exchange of statistical data and related [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] has been expanded. Some support is provided for qualitative data; data cube structures are described; [[hierarchical code lists>>doc:sdmx:Glossary 2\.1.Hierarchical code list.WebHome]] are supported; relationships between data structures can be expressed, providing support for extensibility of data structures; and the description of functional dependencies within cubes are supported.
68 68  
... ... @@ -75,14 +75,14 @@
75 75  * **Optimization for XML Technologies:** Implementation has shown that it is possible to better organize the XML schemas for use within common technology development tools which work with XML. These changes are primarily focused on leveraging the objectoriented features of W3C XML Schema to allow for easier processing of [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] data and metadata.
76 76  * **Consistency between the [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and the [[SDMX Information Model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]]: **Certain aspects of the XML schemas and UML model have been more closely aligned, to allow for easier comprehension of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] model.
77 77  * **Technical Bugs:** Some minor technical bugs have been identified in the registry interfaces and elsewhere. These bugs have been addressed.
78 -* **Support for Non-Time-Series Data in the Generic Format: **One area which has been extended is the ability to express non-time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] data as part of the generic data message.
79 -* **Simplification of the [[data structure definition>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] - specific message types: **Both [[time series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] (version 2.0 Compact) and non-[[time series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] (version 2.0 Cross Sectional) use the same underlying structure for a structure-specific formatted message, which is specific to the [[Data Structure Definition>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] of the [[data set>>doc:sdmx:Glossary 2\.1.Data set.WebHome]].
78 +* **Support for Non-Time-[[Series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] Data in the Generic Format: **One area which has been extended is the ability to express non-time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] data as part of the generic data message.
79 +* **Simplification of the [[data structure definition>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] - specific message types: **Both [[time series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] ((% style="color:#e74c3c" %)version(%%) 2.0 Compact) and non-[[time series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] ((% style="color:#e74c3c" %)version(%%) 2.0 Cross Sectional) use the same underlying structure for a structure-specific formatted message, which is specific to the [[Data Structure Definition>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] of the [[data set>>doc:sdmx:Glossary 2\.1.Data set.WebHome]].
80 80  * **Simplification and better support for the metadata structure: **New use cases have been reported and these are now supported by a re-modelled [[metadata structure definition>>doc:sdmx:Glossary 2\.1.Metadata structure definition.WebHome]].
81 -* **Support for partial [[item schemes>>doc:sdmx:Glossary 2\.1.Item scheme.WebHome]] such as a [[code list>>doc:sdmx:Glossary 2\.1.Code list.WebHome]]: **The concept of a partial (subset) [[item scheme>>doc:sdmx:Glossary 2\.1.Item scheme.WebHome]] such as a partial [[code list>>doc:sdmx:Glossary 2\.1.Code list.WebHome]] for use in exchange scenarios has been introduced**.**
81 +* **Support for partial [[item schemes>>doc:sdmx:Glossary 2\.1.Item scheme.WebHome]] such as a [[code list>>doc:sdmx:Glossary 2\.1.Code list.WebHome]]: **The (% style="color:#e74c3c" %)concept(%%) of a partial (subset) [[item scheme>>doc:sdmx:Glossary 2\.1.Item scheme.WebHome]] such as a partial [[code list>>doc:sdmx:Glossary 2\.1.Code list.WebHome]] for use in exchange scenarios has been introduced**.**
82 82  
83 83  == 2.3 Major Changes from 2.1 to 3.0 ==
84 84  
85 -[[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] version 3.0 introduces new features, improvements and changes to the Standard in the following key areas:
85 +[[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] (% style="color:#e74c3c" %)version(%%) 3.0 introduces new features, improvements and changes to the Standard in the following key areas:
86 86  
87 87  (% class="wikigeneratedid" id="HInformationModel" %)
88 88  **Information Model**
... ... @@ -93,10 +93,10 @@
93 93  * Support for [[code list>>doc:sdmx:Glossary 2\.1.Code list.WebHome]] extension and discriminated union of [[code lists>>doc:sdmx:Glossary 2\.1.Code list.WebHome]]
94 94  * Improvements to structure mapping
95 95  * Improvements to [[code>>doc:sdmx:Glossary 2\.1.Code.WebHome]] [[hierarchies>>doc:sdmx:Glossary 2\.1.Hierarchy.WebHome]] for data discovery
96 -* Improvements to [[constraints>>doc:sdmx:Glossary 2\.1.Constraint.WebHome]]
96 +* Improvements to (% style="color:#e74c3c" %)constraints
97 97  
98 98  (% class="wikigeneratedid" id="HVersioningofStructuralMetadataArtefacts" %)
99 -**Versioning of Structural Metadata Artefacts**
99 +**Versioning of [[Structural Metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] [[Artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]]**
100 100  
101 101  Adoption of the three-number semantic versioning standard for [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] ([[__https:~~/~~/semver.org__>>https://https:semver.org||rel="noopener noreferrer" target="_blank"]])
102 102  
... ... @@ -111,24 +111,24 @@
111 111  (% class="wikigeneratedid" id="HSOAPWebServicesAPI" %)
112 112  **SOAP Web Services API**
113 113  
114 -* The SOAP web services API has been deprecated with version 3.0 standardising on REST
114 +* The SOAP web services API has been deprecated with (% style="color:#e74c3c" %)version(%%) 3.0 standardising on REST
115 115  
116 116  (% class="wikigeneratedid" id="HXML2CJSON2CCSVandEDITransmissionformats" %)
117 117  **XML, JSON, CSV and EDI Transmission formats**
118 118  
119 -The [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]], [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] and [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] specifications have been extended and modified where needed to support the new features and changes such as [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] and microdata Obsolete [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] data message variants including Generic, Compact, Utility and Cross-sectional have been deprecated standardising on Structure Specific Data as the sole XML format for [[data exchange>>doc:sdmx:Glossary 2\.1.Data exchange.WebHome]] The [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]] transmission format for structures and data has been deprecated The organisation of structures into ‘collections’ in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] structure messages has been flattened and simplified The option to reference structures in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] messages using Agency, ID and Version has been deprecated with URN now exclusively used for all non-local referencing purpose
119 +The [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]], [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] and [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] specifications have been extended and modified where needed to support the new features and changes such as [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] and microdata Obsolete [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] data message variants including Generic, Compact, Utility and Cross-sectional have been deprecated standardising on Structure Specific Data as the sole XML format for [[data exchange>>doc:sdmx:Glossary 2\.1.Data exchange.WebHome]] The [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]] transmission format for structures and data has been deprecated The organisation of structures into ‘collections’ in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] structure messages has been flattened and simplified The option to reference structures in [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] messages using Agency, ID and (% style="color:#e74c3c" %)Version(%%) has been deprecated with URN now exclusively used for all non-local referencing purpose
120 120  
121 -Several of the changes are ‘breaking’ meaning that, in specific cases, the version 3.0 specification is not backwardly compatible with earlier versions of the Standard.
121 +Several of the changes are ‘breaking’ meaning that, in specific cases, the (% style="color:#e74c3c" %)version(%%) 3.0 specification is not backwardly compatible with earlier (% style="color:#e74c3c" %)versions(%%) of the Standard.
122 122  
123 123  The principle breaking changes are:
124 124  
125 125  * REST API – The REST API is not backwardly compatible due to modifications to the URLs and query parameters resulting in breaking changes in four of the five main resources.
126 -* SOAP API – Deprecation of the SOAP API means that existing systems designed to use SOAP will not work with version 3.0 registries.
127 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] – [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 and earlier structure, data and metadata XML messages are not valid in version 3.0. Specifically: legacy data messages including Generic, Compact and Utility are no longer supported. The remaining Structure Specific data message has been changed to support new features such as reporting of [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] as part of the [[dataset>>doc:sdmx:Glossary 2\.1.Data set.WebHome]], Structure messages have a number of breaking changes, principally modification to the information model, removal of the agency-version-id option for referencing [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] and changes to the way the structures are organised into ‘collections’ within the message.
128 -* [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] – [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 structure, data and metadata JSON messages are not valid in version 3.0. The data message has been changed to support the improved REST data queries, in particular the ability to retrieve in one operation data from multiple [[datasets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] with potentially different [[Data Structure Definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]]. Breaking changes similar to those for the [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] transmission format have been made to the structure message.
129 -* [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] - The CSV data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] messages are not backwardly compatible with those under version 2.1 due to changes to the structure of the messages needed to support new features such as the improved REST API data queries.
130 -* [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]] – Deprecation of the EDI transmission format means that existing systems designed to send or receive structures or data in EDI will not work with version 3.0 registries.
131 -* Information Model – Several structures have been changed in the version 3.0 model and three removed. For these reasons the version 3.0 model is not directly compatible with version 2.1 or earlier, although conversion of specific [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] is possible under some circumstances. Loss of information during the conversion process however means that in cases like structure mapping, the conversion is not reversible i.e. it is not possible to recreate the 2.1 structure once it has been converted to the 3.0 model.
126 +* SOAP API – Deprecation of the SOAP API means that existing systems designed to use SOAP will not work with (% style="color:#e74c3c" %)version(%%) 3.0 registries.
127 +* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] – [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 and earlier structure, data and metadata XML messages are not valid in (% style="color:#e74c3c" %)version(%%) 3.0. Specifically: legacy data messages including Generic, Compact and Utility are no longer supported. The remaining Structure Specific data message has been changed to support new features such as reporting of [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] as part of the [[dataset>>doc:sdmx:Glossary 2\.1.Data set.WebHome]], Structure messages have a number of breaking changes, principally modification to the information model, removal of the agency-(% style="color:#e74c3c" %)version(%%)-id option for referencing [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] and changes to the way the structures are organised into ‘collections’ within the message.
128 +* [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] – [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 structure, data and metadata JSON messages are not valid in (% style="color:#e74c3c" %)version(%%) 3.0. The data message has been changed to support the improved REST data queries, in particular the ability to retrieve in one operation data from multiple [[datasets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] with potentially different [[Data Structure Definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]]. Breaking changes similar to those for the [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] transmission format have been made to the structure message.
129 +* [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] - The CSV data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] messages are not backwardly compatible with those under (% style="color:#e74c3c" %)version(%%) 2.1 due to changes to the structure of the messages needed to support new features such as the improved REST API data queries.
130 +* [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]] – Deprecation of the EDI transmission format means that existing systems designed to send or receive structures or data in EDI will not work with (% style="color:#e74c3c" %)version(%%) 3.0 registries.
131 +* Information Model – Several structures have been changed in the (% style="color:#e74c3c" %)version(%%) 3.0 model and three removed. For these reasons the (% style="color:#e74c3c" %)version(%%) 3.0 model is not directly compatible with (% style="color:#e74c3c" %)version(%%) 2.1 or earlier, although conversion of specific [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] is possible under some circumstances. Loss of information during the conversion process however means that in cases like structure mapping, the conversion is not reversible i.e. it is not possible to recreate the 2.1 structure once it has been converted to the 3.0 model.
132 132  
133 133  The [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 3.0 Major Changes document provides more information including an analysis of the breaking changes.
134 134  
... ... @@ -139,11 +139,11 @@
139 139  
140 140  * Addition of Dimension Constraint property to a Dataflow
141 141  * Addition of evolving structure property to a Data Structure Definition
142 -* Remove [[version>>doc:sdmx:Glossary 2\.1.Version.WebHome]] property on Categorisation
142 +* Remove version property on Categorisation
143 143  * Simplification of Constraints
144 144  ** Removal of Advanced Release Calendar
145 145  ** Removal of Role, Data Constraints only restrict data that can be reported
146 -** Restrict [[constraint>>doc:sdmx:Glossary 2\.1.Constraint.WebHome]] targets to Identifiable structures (not URLs)
146 +** Restrict constraint targets to Identifiable structures (not URLs)
147 147  ** Addition of Availability Constraint to define actual data
148 148  
149 149  (% class="wikigeneratedid" id="HDocumentation" %)
... ... @@ -306,92 +306,95 @@
306 306  
307 307  == 5.1 SDMX-ML ==
308 308  
309 -[[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] is the XML transmission format specification for exchanging [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]], data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]], and interacting with [[SDMX registry>>doc:sdmx:Glossary 2\.1.SDMX Registry.WebHome]] services. It is designed as a general-purpose format for all automation and data / metadata exchange tasks, and provides the most complete [[coverage>>doc:sdmx:Glossary 2\.1.Coverage.WebHome]].
310 310  
310 +SDMX-ML is the XML transmission format specification for exchanging structural metadata, data and reference metadata, and interacting with SDMX registry services. It is designed as a general-purpose format for all automation and data / metadata exchange tasks, and provides the most complete coverage.
311 +
311 311  There are four distinct types of message:
312 312  
313 -1. //**Structure Definition**~:// For the exchange of [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]. A [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] structure message can carry details of any number and combination of [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]] like DSDs, [[code lists>>doc:sdmx:Glossary 2\.1.Code list.WebHome]] and (% style="color:#e74c3c" %)constraints(%%).
314 -1. //**Structure-specific Data**~:// For the exchange of data. This format is specific to the [[Data Structure Definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] of the [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] (in other terms, it is [[DSD>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]]-specific) and is created by following mappings between the metadata constructs defined in the Structure Definition message and the technical specification of the format. It supports the exchange of large [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] in XML format, provides strict validation of conformance with the [[DSD>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] using a generic XML parser, and supports the transmission of partial [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] ([[incremental updates>>doc:sdmx:Glossary 2\.1.Incremental update.WebHome]]) as well as whole [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]].
314 +1. //**Structure Definition**~:// For the exchange of structural metadata. A SDMX-ML structure message can carry details of any number and combination of structural metadata artefacts like DSDs, code lists and constraints.
315 +1. //**Structure-specific Data**~:// For the exchange of data. This format is specific to the Data Structure Definitions of the data sets (in other terms, it is DSD-specific) and is created by following mappings between the metadata constructs defined in the Structure Definition message and the technical specification of the format. It supports the exchange of large data sets in XML format, provides strict validation of conformance with the DSD using a generic XML parser, and supports the transmission of partial data sets (incremental updates) as well as whole data sets.
315 315  
316 316  Many XML tools and technologies have expectations about the functions performed by an XML schema, one of which is a very direct relationship between the XML constructs described in the XML schema and the tagged data in the XML instance. Strong data typing is also considered normal, supporting full validation of the tagged data. These message types are designed to support validation and other expected XML schema functions.
317 317  
318 -1. //**Generic Metadata**~:// For the exchange of [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] sets. ‘Generic’ means the XML elements and XML [[attributes>>doc:sdmx:Glossary 2\.1.Attribute.WebHome]] are the same regardless of the [[metadata set>>doc:sdmx:Glossary 2\.1.Metadata set.WebHome]].
319 -1. //**Registry**~:// All of the possible interactions with the [[SDMX registry>>doc:sdmx:Glossary 2\.1.SDMX Registry.WebHome]] services are supported using [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] interfaces and REST API calls. Submission of [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] content, data registrations and [[subscriptions>>doc:sdmx:Glossary 2\.1.Subscription.WebHome]] is performed by a synchronous exchange of documents – a “request” message answered by a “response” message.
319 +1. //**Generic Metadata**~:// For the exchange of reference metadata sets. ‘Generic’ means the XML elements and XML attributes are the same regardless of the metadata set.
320 +1. //**Registry**~:// All of the possible interactions with the SDMX registry services are supported using SDMX-ML interfaces and REST API calls. Submission of structural metadata content, data registrations and subscriptions is performed by a synchronous exchange of documents – a “request” message answered by a “response” message.
320 320  
321 321  == 5.2 SDMX-JSON ==
322 322  
323 -[[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] is the JSON transmission format specification for exchanging [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]], data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]]. It provides an alternative to [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and is most suited to applications like web data dissemination.
324 324  
325 -[[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] messages serve the same function as those of the XML formats but have a different structure. For data, an important distinction is that they carry both [[component>>doc:sdmx:Glossary 2\.1.Component.WebHome]] [[codes>>doc:sdmx:Glossary 2\.1.Code.WebHome]] and labels which provides all the information needed to display the content in a single JSON response. The XML Structure-specific Data format by contrast carries only [[code>>doc:sdmx:Glossary 2\.1.Code.WebHome]] IDs thus requiring applications obtain and hold [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] about the [[data set>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] in order to display the content in human-readable form.
325 +SDMX-JSON is the JSON transmission format specification for exchanging structural metadata, data and reference metadata. It provides an alternative to SDMX-ML and is most suited to applications like web data dissemination.
326 326  
327 -[[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] does not include messages for [[subscription>>doc:sdmx:Glossary 2\.1.Subscription.WebHome]] / [[notification>>doc:sdmx:Glossary 2\.1.Notification.WebHome]] or registration registry services - [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] must be used for those purposes.
327 +SDMX-JSON messages serve the same function as those of the XML formats but have a different structure. For data, an important distinction is that they carry both component codes and labels which provides all the information needed to display the content in a single JSON response. The XML Structure-specific Data format by contrast carries only code IDs thus requiring applications obtain and hold structural metadata about the data set in order to display the content in human-readable form.
328 328  
329 +SDMX-JSON does not include messages for subscription / notification or registration registry services - SDMX-ML must be used for those purposes.
330 +
329 329  There are three distinct message types:
330 330  
331 -1. //**Structure**~:// For the exchange [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]. [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] structure messages follow the same principles as for [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] in that a single message can transmit any number and combination of [[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]] [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]]. While the [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] and [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] messages are structured differently, it is possible to freely convert between them.
332 -1. //**Data**: //For the exchange of data. Unlike [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]], the structure of a [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]] data message is not specific to the DSDs of the [[data sets>>doc:sdmx:Glossary 2\.1.Data set.WebHome]] so schema validation will not check for compliance of the data with the DSDs.
333 -1. //Metadata//: For the exchange of [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] sets.
333 +1. //**Structure**~:// For the exchange structural metadata. SDMX-JSON structure messages follow the same principles as for SDMX-ML in that a single message can transmit any number and combination of structural metadata artefacts. While the SDMX-ML and SDMX-JSON messages are structured differently, it is possible to freely convert between them.
334 +1. //**Data**: //For the exchange of data. Unlike SDMX-ML, the structure of a SDMX-JSON data message is not specific to the DSDs of the data sets so schema validation will not check for compliance of the data with the DSDs.
335 +1. //Metadata//: For the exchange of reference metadata sets.
334 334  
335 335  == 5.3 SDMX-CSV ==
336 336  
337 -[[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] is the CSV transmission format specification for exchanging data and [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] only.
338 338  
339 -[[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] provides a simple columnar format for data and metadata that can be readily created and interpreted by standard software tools such as Microsoft Excel. Nevertheless, data and metadata can still be converted between the CSV and the JSON / XML formats without loss.
340 +SDMX-CSV is the CSV transmission format specification for exchanging data and reference metadata only.
340 340  
342 +SDMX-CSV provides a simple columnar format for data and metadata that can be readily created and interpreted by standard software tools such as Microsoft Excel. Nevertheless, data and metadata can still be converted between the CSV and the JSON / XML formats without loss.
343 +
341 341  There are two distinct message types:
342 342  
343 -1. **//Data//**: For the exchange of data. Like [[SDMX-JSON>>doc:sdmx:Glossary 2\.1.SDMX-JSON.WebHome]], [[SDMX-CSV>>doc:sdmx:Glossary 2\.1.SDMX-CSV.WebHome]] can include both [[code>>doc:sdmx:Glossary 2\.1.Code.WebHome]] IDs and labels which is helpful when using the data to create human readable charts and dashboards.
344 -1. **//Metadata//**: For the exchange of [[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]] sets.
346 +1. **//Data//**: For the exchange of data. Like SDMX-JSON, SDMX-CSV can include both code IDs and labels which is helpful when using the data to create human readable charts and dashboards.
347 +1. **//Metadata//**: For the exchange of reference metadata sets.
345 345  
346 346  == 5.4 Formats and Messages Deprecated in Version 3.0 ==
347 347  
348 -The following formats and messages have been deprecated in (% style="color:#e74c3c" %)version(%%) 3.0 to simplify, modernise and rationalise the standard.
351 +The following formats and messages have been deprecated in version 3.0 to simplify, modernise and rationalise the standard.
349 349  
350 -* [[SDMX-EDI>>doc:sdmx:Glossary 2\.1.SDMX-EDI.WebHome]]
351 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 1.0/2.0 Generic (time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]]) data message
352 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 1.0/2.0 Compact (time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]]) data message
353 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 1.0/2.0 Utility (time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]]) data message
354 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 1.0/2.0 Cross-Sectional data message
355 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 2.1 Generic ([[Time Series>>doc:sdmx:Glossary 2\.1.Series.WebHome]]) data messages (for observations, time-[[series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] and cross-sectional data)
356 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] 2.1 Structure Specific [[Time Series>>doc:sdmx:Glossary 2\.1.Series.WebHome]] data message
353 +* SDMX-EDI
354 +* SDMX-ML 1.0/2.0 Generic (time-series) data message
355 +* SDMX-ML 1.0/2.0 Compact (time-series) data message
356 +* SDMX-ML 1.0/2.0 Utility (time-series) data message
357 +* SDMX-ML 1.0/2.0 Cross-Sectional data message
358 +* SDMX-ML 2.1 Generic (Time Series) data messages (for observations, time-series and cross-sectional data)
359 +* SDMX-ML 2.1 Structure Specific Time Series data message
357 357  
358 -The following messages were deprecated in (% style="color:#e74c3c" %)version(%%) 3.0 as a consequence of the deprecation of the SOAP web services:
361 +The following messages were deprecated in version 3.0 as a consequence of the deprecation of the SOAP web services:
359 359  
360 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] Query messages
361 -* [[SDMX-ML>>doc:sdmx:Glossary 2\.1.SDMX-ML.WebHome]] Submit Structure Request messages
363 +* SDMX-ML Query messages
364 +* SDMX-ML Submit Structure Request messages
362 362  
363 363  = 6 Dependencies on SDMX content-oriented guidelines =
364 364  
365 -The technical standards proposed here are designed so that they can be used in conjunction with other [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] guidelines which are more closely tied to the content and semantics of statistical [[data exchange>>doc:sdmx:Glossary 2\.1.Data exchange.WebHome]]. The [[SDMX Information Model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]] works equally well with any statistical (% style="color:#e74c3c" %)concept(%%), but to encourage interoperability, it is also necessary to standardize and harmonize the use of specific (% style="color:#e74c3c" %)concepts(%%) and terminology. To achieve this goal, [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] creates and maintains guidelines for [[cross-domain concepts>>doc:sdmx:Glossary 2\.1.Cross-domain concept.WebHome]], terminology, and structural definitions. There are three major parts to this effort.
368 +The technical standards proposed here are designed so that they can be used in conjunction with other SDMX guidelines which are more closely tied to the content and semantics of statistical data exchange. The SDMX Information Model works equally well with any statistical concept, but to encourage interoperability, it is also necessary to standardize and harmonize the use of specific concepts and terminology. To achieve this goal, SDMX creates and maintains guidelines for cross-domain concepts, terminology, and structural definitions. There are three major parts to this effort.
366 366  
367 367  == 6.1 Cross-Domain Concepts ==
368 368  
369 -The [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[Cross-Domain Concepts>>doc:sdmx:Glossary 2\.1.Cross-domain concept.WebHome]] is a content guideline concerning (% style="color:#e74c3c" %)concepts(%%) which are used across [[statistical domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]]. This list is expected to grow and to be subject to revision as [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] is used in a growing number of domains. The use of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[Cross-Domain Concepts>>doc:sdmx:Glossary 2\.1.Cross-domain concept.WebHome]], where appropriate, provides a framework to further promote interoperability among organisations using the technical standards presented here. The harmonization of statistical (% style="color:#e74c3c" %)concepts(%%) includes not only the definitions of the (% style="color:#e74c3c" %)concepts(%%), and their names, but also, where appropriate, their [[representation>>doc:sdmx:Glossary 2\.1.Representation.WebHome]] with standard [[code lists>>doc:sdmx:Glossary 2\.1.Code list.WebHome]], and the role they play within [[data structure definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] and [[metadata structure definitions>>doc:sdmx:Glossary 2\.1.Metadata structure definition.WebHome]].
372 +The SDMX Cross-Domain Concepts is a content guideline concerning concepts which are used across statistical domains. This list is expected to grow and to be subject to revision as SDMX is used in a growing number of domains. The use of the SDMX Cross-Domain Concepts, where appropriate, provides a framework to further promote interoperability among organisations using the technical standards presented here. The harmonization of statistical concepts includes not only the definitions of the concepts, and their names, but also, where appropriate, their representation with standard code lists, and the role they play within data structure definitions and metadata structure definitions.
370 370  
371 -The intent of this guideline is two-fold: to provide a core set of (% style="color:#e74c3c" %)concepts(%%) which can be used to structure statistical data and metadata, to promote interoperability between systems (“[[structural metadata>>doc:sdmx:Glossary 2\.1.Structural metadata.WebHome]]”, as described above); and to promote the exchange of metadata more widely, with a set of harmonized (% style="color:#e74c3c" %)concept(%%) names and definitions for other types of metadata (“[[reference metadata>>doc:sdmx:Glossary 2\.1.Reference metadata.WebHome]]”, as defined above.)
374 +The intent of this guideline is two-fold: to provide a core set of concepts which can be used to structure statistical data and metadata, to promote interoperability between systems (“structural metadata”, as described above); and to promote the exchange of metadata more widely, with a set of harmonized concept names and definitions for other types of metadata (“reference metadata”, as defined above.)
372 372  
373 373  == 6.2 Metadata Common Vocabulary ==
374 374  
375 -The Metadata Common Vocabulary is an [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] guideline which provides definition of terms to be used for the comparison and mapping of terminology found in [[data structure definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]] and in other aspects of statistical metadata management. Essentially, it provides ISOcompliant definitions for a wide range of statistical terms, which may be used directly, or against which other terminology systems may be (% style="color:#e74c3c" %)mapped(%%). This set of terms is inclusive of the terminology used within the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Technical Standards.
378 +The Metadata Common Vocabulary is an SDMX guideline which provides definition of terms to be used for the comparison and mapping of terminology found in data structure definitions and in other aspects of statistical metadata management. Essentially, it provides ISOcompliant definitions for a wide range of statistical terms, which may be used directly, or against which other terminology systems may be mapped. This set of terms is inclusive of the terminology used within the SDMX Technical Standards.
376 376  
377 -The MCV provides definitions for terms on which the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Cross-Domain Metadata (% style="color:#e74c3c" %)Concepts(%%) work is built.
380 +The MCV provides definitions for terms on which the SDMX Cross-Domain Metadata Concepts work is built.
378 378  
379 379  == 6.3 Statistical Subject-Matter Domains ==
380 380  
381 -The [[Statistical Subject-Matter Domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]] is a listing of the breadth of statistical information for the purposes of organizing widespread statistical exchange and categorization. It acts as a standard scheme against which the categorization schemes of various counterparties can be (% style="color:#e74c3c" %)mapped(%%), to facilitate interoperable data and metadata exchange. It serves another useful purpose, however, which is to allow an organization of corresponding “domain groups”, each of which could define standard [[data structure definitions>>doc:sdmx:Glossary 2\.1.Data structure definition.WebHome]], (% style="color:#e74c3c" %)concepts(%%), etc. within their domains. Such groups already exist within the international community. [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] would use the [[Statistical Subject-Matter Domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]] list to facilitate the efforts of these groups to develop the kinds of content standards which could support the interoperation of [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]]-conformant technical systems within and across [[statistical domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]]. The organisation of the content of such schemes is supported in [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] as a [[Category Scheme>>doc:sdmx:Glossary 2\.1.Category scheme.WebHome]].
384 +The Statistical Subject-Matter Domains is a listing of the breadth of statistical information for the purposes of organizing widespread statistical exchange and categorization. It acts as a standard scheme against which the categorization schemes of various counterparties can be mapped, to facilitate interoperable data and metadata exchange. It serves another useful purpose, however, which is to allow an organization of corresponding “domain groups”, each of which could define standard data structure definitions, concepts, etc. within their domains. Such groups already exist within the international community. SDMX would use the Statistical Subject-Matter Domains list to facilitate the efforts of these groups to develop the kinds of content standards which could support the interoperation of SDMX-conformant technical systems within and across statistical domains. The organisation of the content of such schemes is supported in SDMX as a Category Scheme.
382 382  
383 -[[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[Statistical Subject-Matter Domains>>doc:sdmx:Glossary 2\.1.Statistical subject-matter domain.WebHome]] will be listed and maintained by the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Initiative and will be subject to [[adjustment>>doc:sdmx:Glossary 2\.1.Adjustment.WebHome]].
386 +SDMX Statistical Subject-Matter Domains will be listed and maintained by the SDMX Initiative and will be subject to adjustment.
384 384  
385 385  == 6.4 SDMX Concept Roles ==
386 386  
387 -These guidelines define the standard set of [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] (% style="color:#e74c3c" %)Concept(%%) Roles and their use. This set of standard [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] (% style="color:#e74c3c" %)Concepts(%%) are implemented as a [[cross-domain Concept>>doc:sdmx:Glossary 2\.1.Cross-domain concept.WebHome]] Scheme that defines the set of (% style="color:#e74c3c" %)concept(%%) roles and gives examples on (% style="color:#e74c3c" %)concept(%%) role implementation in [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.0, 2.1 and 3.0. A (% style="color:#e74c3c" %)concept(%%) role gives a particular context to a (% style="color:#e74c3c" %)concept(%%) for easy and systematic interpretation by machine processing and visualization tools. For example, the (% style="color:#e74c3c" %)concepts(%%) REPORTING_AREA and COUNTERPART_AREA are different (% style="color:#e74c3c" %)concepts(%%) but they are both geographical characteristics, therefore they can be associated with the same (% style="color:#e74c3c" %)concept(%%) role ID: "GEO". This allows visualization systems to interpret these (% style="color:#e74c3c" %)concepts(%%) as geographical data in order to generate (% style="color:#e74c3c" %)maps(%%). The implementation of (% style="color:#e74c3c" %)concept(%%) roles is different in (% style="color:#e74c3c" %)versions(%%) 2.0 and 2.1/3.0 of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] technical standard. Specifically for [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 3.0, this set of roles is considered a normative list that must be interpreted in the same way by all organisations. Additional roles may be provided via the standard roles’ mechanism in [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 3.0, i.e., via [[Concept Schemes>>doc:sdmx:Glossary 2\.1.Concept scheme.WebHome]]; the semantics of these roles have to be agreed bilateraly in [[data exchanges>>doc:sdmx:Glossary 2\.1.Data exchange.WebHome]]. The (% style="color:#e74c3c" %)Concept(%%) Roles are available as an [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[Concept Scheme>>doc:sdmx:Glossary 2\.1.Concept scheme.WebHome]] on the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[Global Registry>>doc:sdmx:Glossary 2\.1.Global registry.WebHome]].
390 +These guidelines define the standard set of SDMX Concept Roles and their use. This set of standard SDMX Concepts are implemented as a cross-domain Concept Scheme that defines the set of concept roles and gives examples on concept role implementation in SDMX 2.0, 2.1 and 3.0. A concept role gives a particular context to a concept for easy and systematic interpretation by machine processing and visualization tools. For example, the concepts REPORTING_AREA and COUNTERPART_AREA are different concepts but they are both geographical characteristics, therefore they can be associated with the same concept role ID: "GEO". This allows visualization systems to interpret these concepts as geographical data in order to generate maps. The implementation of concept roles is different in versions 2.0 and 2.1/3.0 of the SDMX technical standard. Specifically for SDMX 3.0, this set of roles is considered a normative list that must be interpreted in the same way by all organisations. Additional roles may be provided via the standard roles’ mechanism in SDMX 3.0, i.e., via Concept Schemes; the semantics of these roles have to be agreed bilateraly in data exchanges. The Concept Roles are available as an SDMX Concept Scheme on the SDMX Global Registry.
388 388  
389 389  = 7 Validation and Transformation Language =
390 390  
391 -For many years the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] initiative has been fostering and supporting the development of a standard calculation [[language>>doc:sdmx:Glossary 2\.1.Language.WebHome]], called [[Validation and Transformation Language>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] ([[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]]). A blueprint for defining calculations was already described in the original [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 specifications (package 13 of the Information Model - “Transformations and Expressions”). It was just a basic framework that required further developments to became operational in order to achieve a calculation [[language>>doc:sdmx:Glossary 2\.1.Language.WebHome]] able to manipulate [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] [[artefacts>>doc:sdmx:Glossary 2\.1.Artefact.WebHome]].
394 +For many years the SDMX initiative has been fostering and supporting the development of a standard calculation language, called Validation and Transformation Language (VTL). A blueprint for defining calculations was already described in the original SDMX 2.1 specifications (package 13 of the Information Model - “Transformations and Expressions”). It was just a basic framework that required further developments to became operational in order to achieve a calculation language able to manipulate SDMX artefacts.
392 392  
393 -These developments started in late 2012 and were put in charge of the [[Validation and Transformation Language>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] Task Force ([[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] TF), which included members of the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] Technical Working Group (TWG) and Statistical Working Group (SWG), besides experts coming from the DDI and GSIM communities. The intent was to define a standard [[language>>doc:sdmx:Glossary 2\.1.Language.WebHome]] to be implemented in [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] and applicable also to GSIM and DDI. This brought to the publication of the [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 1.0 in 2015. Then new requirements came from a number of proofs of (% style="color:#e74c3c" %)concepts(%%) and tests of [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 1.0 made by several organisations and triggered a large improvement of the [[language>>doc:sdmx:Glossary 2\.1.Language.WebHome]]. A new provisional (% style="color:#e74c3c" %)version(%%), the [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 1.1, was released in public consultation in 2017. The high number of [[comments>>doc:sdmx:Glossary 2\.1.Comment.WebHome]] received triggered another phase of intensive work, with the main goal of achieving a more robust and forward compatible (% style="color:#e74c3c" %)version(%%). Finally, the [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 2.0 was published between April and July 2018 (see the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] website).
396 +These developments started in late 2012 and were put in charge of the Validation and Transformation Language Task Force (VTL TF), which included members of the SDMX Technical Working Group (TWG) and Statistical Working Group (SWG), besides experts coming from the DDI and GSIM communities. The intent was to define a standard language to be implemented in SDMX and applicable also to GSIM and DDI. This brought to the publication of the VTL 1.0 in 2015. Then new requirements came from a number of proofs of concepts and tests of VTL 1.0 made by several organisations and triggered a large improvement of the language. A new provisional version, the VTL 1.1, was released in public consultation in 2017. The high number of comments received triggered another phase of intensive work, with the main goal of achieving a more robust and forward compatible version. Finally, the VTL 2.0 was published between April and July 2018 (see the SDMX website).
394 394  
395 -The implementation of the [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 2.0 in [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] started in late 2018 and was published as an incremental revision to the [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] 2.1 standards in July 2020. It allows users to write [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] 2.0 programs for validating and transforming [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] data, to store these programs in a [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] metadata registry and to exchange them through [[SDMX>>doc:sdmx:Glossary 2\.1.Statistical data and metadata exchange.WebHome]] messages, also together the definition of the data structures of the involved data.
398 +The implementation of the VTL 2.0 in SDMX started in late 2018 and was published as an incremental revision to the SDMX 2.1 standards in July 2020. It allows users to write VTL 2.0 programs for validating and transforming SDMX data, to store these programs in a SDMX metadata registry and to exchange them through SDMX messages, also together the definition of the data structures of the involved data.
396 396  
397 -The Transformations and Expressions package for modelling [[VTL>>doc:sdmx:Glossary 2\.1.Validation and transformation language.WebHome]] programs in the [[SDMX information model>>doc:sdmx:Glossary 2\.1.SDMX Information Model.WebHome]] is explained in Section 2 of the Technical Specifications with further detailed usage and implementation guidance given in Section 6.
400 +The Transformations and Expressions package for modelling VTL programs in the SDMX information model is explained in Section 2 of the Technical Specifications with further detailed usage and implementation guidance given in Section 6.
© Semantic R&D Group, 2026